Researchers

IT Services Blog

IT services blog

Geopandas and streamlit to display local tree data in deckgl

by Anson Parker on 2020-12-08T15:19:00-05:00 | Comments

I'm not a good programmer.  85 times and about two hours total.  That's how many edits and saves before I got something out the front door that might pass as an "app" by some definition.  If you're looking for the Tableau experience this is not it... yet

https://share.streamlit.io/alibama/cville-trees/main - here's a link to the app.  It filters trees from the UVa equity atlas http://equity-atlas-uvalibrary.opendata.arcgis.com/ based on species, and allows the user the ability to change the size of the marker for the tree on the map.  Eventually I want to do some canopy analysis, and this is a small start.

The Streamlit components in markets as diverse as molecular structure visualization or building a Voronoi maps of Trader Joe's distributions evidences that their team is considering a big picture - and to be able to get a relative non-programmer like myself using a tool and getting a draft out the door in one day is pretty impressive.  

My workflow went between Sreamlit's discussion boards located here https://discuss.streamlit.io/ and a bit of stackoverflow... given that i now have a Python app akin to what jupyter and voila aspire towards, or Shiny apps in R achieve with a nifty backend in github that really simplifies developing. 

Streamlit Sharing

This is the streamlit sharing app administration dashboard.  There's a list of the existing apps on this page, as well as the menu to create new apps from scratch or in a new repo or an existing templateThe main streamlit dashboard has a list of the existing projects as well as a menu to create new apps either from scratch or from an existing template

 

 

 

 

 

 

once you choose to create a new app from scratch you can connect it to your github repo where you'll be storing your code

Once you go to create a new app from scratch you'll have the opportunity to connect the app over to the github repo and name your main file and branch (if you're using an existing repo with code in it already)

 

 

 

 

 

 

Packages.txt and Requirements.txt files in Streamlit Sharing

this is a screenshot of a streamlit github repo with three files - the streamlit_app.py file that holds the python scripts, the requirements.txt that holds the python packages, and packages.txt file that contains binaries installable through apt-get in basic debian repositoriesStreamlit Sharing is still in beta, however I got my invitation to participate within an hour or so of request, maybe sooner.... i don't recall.

Streamlit Sharing connects github.com's infrastructure to a containerized Streamlit server to allow essentially no-click app creation - hit save in github, the app automatically updates on the streamlit side almost instantly - no other commands necessary.  

This is a screenshot of a streamlit github repo with three files - the streamlit_app.py file that holds the python scripts and references the data.  The requirements.txt file contains python packages that would normally be installed with pip - to be honest i'm not sure how or whether conda is part of this process.  Lastly there is the packages.txt file that contains references to binaries that would be typically installed through apt-get in basic debian repositories.  

In our example we have the following Python libraries in our requirements.txt file

  pydeck For Deck.Gl integration
  pandas https://pandas.pydata.org/ for data processing
  streamlit because we're working with streamlit.... 
  shapely https://pypi.org/project/Shapely/ for polygon management
  fiona Python's GDAL API https://pypi.org/project/Fiona/
  geopandas Geospatial data processing in /ython https://geopandas.org/
  pygeos Extends Geopandas abilitie

 

Our packages.txt file has the following lines

gdal-bin
python-rtree

these two programs give the underlying container the ability to do the heavy geospatial lifting 

Once the packages.txt and requirements.txt files are in the repository they will be automatically discovered during the app baking process.  Adding new sources in to these files may require you to reboot the app - and this brings us in to the first bit of streamlit infrastructure

screenshot of a streamlit environment from the front end of the app with the streamlit menu opened from the bottom left of the screen

 

This is a screenshot of the streamlit app with the app management console open

 

 

 

 

 

 

 

screenshot closeup of the streamlit app control panelThe streamlit control panel provides some utilities for managing your app - it's located on the bottom left of the screen

  1. more in-depth debugging tools - available through the log file download
  2.  below that is the reboot app option - this will reload the requirements.txt and packages.txt file and is necessary to use if you add new binaries in to your app
  3. delete app... it deletes the app...and below that are some documentation and support tools

 

 

 

 

 

Inside the app itself is a pretty light logic load.  I really like deck.gl, however hadn't really worked with it before - it's pretty elegant and comes with some neat pan-tilt tools i haven't seen front and center before... has a modern design flavor to it in general

Anyhow - with 3 lines of code I added one dropdown select menu that manipulates the Geopandas dataframe and generates a data set that the deck gl library can consume

treetype = trees['Common_Name'].drop_duplicates() # select all of the trees from the dataframe and filter by unique values to create a useful dropdown menu list
tree_choice = st.sidebar.selectbox('Tree type:', treetype) # render the streamlit widget on the sidebar of the page using the list we created above for the menu
trees=trees[trees['Common_Name'].str.contains(tree_choice)] # create a dataframe for our deck.gl map to use in the layer as the data source and update it based on the selection made above

and similarly here's a streamlit slider widget for controlling the size of the points inside the deck.gl map

dotradius = st.sidebar.slider('Tree dot radius') # this creates a slider widget called "tree dot radius"

layer = [
    pdk.Layer(
        "GeoJsonLayer",
        data=trees,
        getFillColor=[60, 220, 255],
        getRadius=dotradius, #here's the streamlit slider widget being used to determine the size of the point on the deckgl map
    ),

]

And that's really all i've done so far.  In the next part of this work I'm hoping to start doing some actual geopandas data processing to see where Ash trees land on or near existing buildings as we did in this jupyter notebook + geopandas tutorial here https://guides.hsl.virginia.edu/it-services-blog/zoombites/using-tree-data-in-a-python-3-jupyter-notebook 

Additional Discussion

  • Here's an interesting discussion on streamlit as the next tableau...  https://medium.com/@fabio.annovazzi/is-streamlit-a-threat-to-powerbi-and-tableau-d20af60d594
  • A cool component for using leaflet with streamlit for feedback =https://github.com/andfanilo/streamlit-light-leaflet

Thank You!

An enormous shout out to Randy Zwitch who leaned in on the forums and Martin Fleishman of Geopandas for encouraging me to do something useful like documentation ;)  It's a pleasure to be a part of an internationally diverse community - something that Open Source continues to deliver.


 Add a Comment

0 Comments.

  Return to Blog
This post is closed for further discussion.

Skip to Main Content

Claude Moore Health Sciences Library
1350 Jefferson Park Avenue P.O. Box 800722
Charlottesville, VA 22908 (Directions)

facebook twitter instagram
© 2021 by the Rector and Visitors of the University of Virginia
Copyright & Privacy