-
Notifications
You must be signed in to change notification settings - Fork 18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DO NOT MERGE: Refactor dataframe where tests, reduce IF conditional compilation blocks #386
base: main
Are you sure you want to change the base?
Conversation
After splicing in changes from #349 to correctly release the Arrow stream, I'm no longer able to easily reproduce the segfaults on repeated reruns of CI. Given that these were intermittent, it's hard to say for certain, but per the GDAL docs, we should have been releasing the stream:
|
Clarification, it seems that the |
I merged #349, so let's see if the failures now disappear on main and other PRs. Although to be honest I also don't fully understand how the changes would fix the crashes ..
Indeed, that's the idea. When we pass the struct to pyarrow, they will "move" it and take ownership of it (i.e. being responsible of releasing it when done reading the data) by setting the release callback to NULL, to ensure we don't call that anymore. |
Hmm, still a crash on the merge commit on main .. https://github.com/geopandas/pyogrio/actions/runs/8718790747/job/23916758253 |
Split up dataframe read with
where
test to try and probe at #384Also tries to remove
IF
blocks marked as deprecated by Cython, except for those still needed for GDAL version specific functions.