Better documentation on loading big data and on x/y vs lat/long plotting

In all the examples/documentation the data to be plotted is read into a single df. For most this means that we are working with data which can fit in memory. However, for my case, I really want to use datashader to plot big data, but I have no chance on reading my entire dataset into a single df. (Its stored in a PostgresSQL db). I have no doubt this is possible, however It would be great to see some examples on how this is possible. (Piecewise combine plots, just make several plots that overlay with eachother etc)

An other thing that is lacking, in my eye, is a clear destinction on when to use x,y coordinates and when to use lat,long.

Thank you.

When data does not fit into memory I use Dask. So in your case you’d create a Dask.DataFrame and use that to create a datashader image.

When to use x/y values vs when to use latitude & longitude depends on your goal. If you want to display a datashader image on its own the choice is yours; datashader doesn’t care about units. However, if you want to overlay the datashader image on a web map then you need to convert your latitude and longitude values to meters first.

https://datashader.org/user_guide/Geography.html