Disagreement between Spatial Index Report in GeoPandas and QGIS

[DimitrisMantas]

I have a GeoPackage files containing a Polygon sheet index which I want to intersect with some geometry to search and download relevant assets.

When loading this index into QGIS and checking its properties, it is reported that it has a spatial index. I can also see rtee_-named tables in the underlying SQLite database. However, GeoPandas insists that the dataset has no spatial index when using the has_sindex method unless I specifically “create” it with sindex.

That being said, the intersection I need to compute takes virtually the same amount of time regardless of whether I do that or not which makes me think that it in fact exists, it’s just that GeoPandas has trouble recognizing it although the underlying libraries can use it just fine.

[theroggy]

GeoPandas is an in-memory analytics library... so all data will typically be loaded in-memory for further processing. Because it does all processing in memory, it will also create its own in-memory spatial index the first time you apply a spatial operation that needs one. Some more info in the documentation of has_sindex.

So, GeoPandas will typically not use the spatial index available in the GeoPackage. An exception to this is at the stage when you are reading the data (to memory) using geopandas.read_file:

• If you use the parameters bbox= or mask= you can limit the data being loaded to the rows that intersect with the bbox/geometry mask specified. At this stage the spatial index in the file will be used to speed up locating the relevant rows.
• Similarly, when you use geopandas.read_file with the engine="pyogrio" option, you can use the extra parameters listed here. When you use where= or sql= to filter/proces/... the data being read, you can also (explicitly) use the spatial index available in a GeoPackage to speedup the spatial filtering.

[DimitrisMantas]

Huh, my guess was that read_file (which I’m using to open the sheet index in the first place) would somehow parse the GeoPackage index to an in-memory object, so all clear on that end.

My understanding of your reply is that if there’s no in-memory index, GeoPandas will automatically create one when necessary. However, if I call has_sindex on the sheet index after I’m done with the overlay operation, the return value is still False.

Am I missing something? Unless such operations are programed to not require a spatial index in the first place…

[theroggy]

What overlay operation are you using?

And on which dataframe are you checking the has_index, because the result of the overlay will be a copy of the data that won't have an index anymore until you do a spatial operation on that again...

[DimitrisMantas]

All clear, I was checking the original data frame. Thanks!