feat: use geopandas for GEOGRAPHY columns if geopandas is installed

[tswast]

This would technically be a breaking change, but it might make sense to do while we are changing default dtypes in #786 for https://issuetracker.google.com/144712110

If the GeoPandas library is installed (meaning, GeoPandas should be considered an optional "extra"), it may make sense to use the extension dtypes provided by GeoPandas by default on GEOGRAPHY columns.

[tswast]

Since GeoSeries using the extension type (https://jorisvandenbossche.github.io/blog/2019/08/13/geopandas-extension-array-refactor/), I believe that means it can be used in a normal pandas DataFrame (ignoring the GeoDataFrame class)

[tswast]

This should be easier once #786 is in, as that pull request adds logic to introspect the BigQuery schema to determine what dtype to use + do any conversion steps on the dataframe (e.g. time zone updates) before returning it.

[tswast]

Oh, and I did some more reading regarding GeoDataFrame vs GeoSeries. To use GeoDataFrame requires selecting one specific column to be the "geometry" column. I don't really think it makes sense for us to add arguments just for that. Instead, we can create regular DataFrame with any GEOGRAPHY columns converted to GeoSeries. The developer should then be able to construct a GeoDataFrame from that if they desire.

[jimfulton]

+1

[jimfulton]

Oh, and I did some more reading regarding GeoDataFrame vs GeoSeries. To use GeoDataFrame requires selecting one specific column to be the "geometry" column. I don't really think it makes sense for us to add arguments just for that. Instead, we can create regular DataFrame with any GEOGRAPHY columns converted to GeoSeries. The developer should then be able to construct a GeoDataFrame from that if they desire.

Actually, not so much. If you try to put a GeoSeries in a regular Pandas data frame, it gets converted to a Pandas series.

GeoDataFrames go though a bunch of hijinks to give the illusion of storing a GeoSeries.

[jimfulton]

Digging a little deeper:

• A GeoSeries is a wrapper for a Series containing shapely objects (of possibly various kinds) and providing lots of geographic analytical capabilities. A series also carries along a projection.
• A GeoDataFrame lets you store a GeoSeries along with a bunch of other attributes. It provides analytic functions that effectively delegate to the series. (I'm sure I'm simplifying somewhat, but not a lot.)
  • GeoDataFrames intrinsically only allow one series. The model is that a row in a table models a feature consisting of a geography and some other attributes.
• If we don't wan't to deal with picking a single geography column, we could provide some value by converting shapely objects.
• I imagine that it's very common for a table to have a single geography column, in which case, we could determine the geographic highlander ourselves.

I have a suggestion, but I'll ponder it a bit more. :)

[jimfulton]

IMO the leveraging of geography types should be explicit. (The behavior should not change based on what's installed other than providing a clear error when what's requested isn't possible due to a lack of a dependency.)

Some options (not all mutually exclusive):

1. Add a to_geodataframe query-job method.

   • This would have the same arguments as to_dataframe.
   • There must be at least one GEOGRAPHY column.
   • If there's more than one GEOGRAPHY column, then the geography_column option must be used to control which column is used to support the geographic analysis methods.
     • Other GEOGRAPHY columns are output as shapely objects and can be easily converted to GeoSeries.
   • Error if geopandas isn't installed.

2. Add a geodataframe option to to_dataframe. If there is only one GEOGRAPHY column, then this can be True. If there is more than one, then this must be the name of the column to use, in which the others are converted as shapely objects.

   • Error if geopandas isn't installed.

3. Add a geography_as_object option to to_dataframe. If true, then geography objects are output as shapely objects.

   • Error if shapley isn't installed.

BTW, there should be some non-reference documentation on working with pandas and geopandas. It's elegant but non-obvious, IMO, that you get data into Pandas via job objects, which are elegant and strange in themselves. :)

[jimfulton]

For loading data with load_table_from_dataframe, I propose:

1. When using parquet format and loading a geography column, inspect the first value (if there are any):

   • If it's binary, use the parquet binary type. If the values are WKB, this will just work. I tested it. :)
   • If shapely is installed and it's a shapely object, convert the pandas column to WKB and use a pandas binary column.

2. When using CSV and loading a geography column, inspect the first value (if there are any):

   • If shapely is installed and it's a shapely object, convert the pandas column to wkt.

I think this will make load_table_from_dataframe just work with GeoDataFrames.

[jimfulton]

Do we want to_geodataframe_iterable? It seems that to_dataframe_iterable is missing at least one to_dataframe feature, date_as_object=False.