New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Invalid data and segfault when reading past the size of file with fromfile on Ubuntu 16.04 #12300
Comments
Just to note, reproducable on 1.15.3. I guess we know the size, so this should just raise an error, or read the whole file. If this works silently on some systems, maybe we should put a release note just in case (I still would say we can just fix it). EDIT: I would tend to error, just thought whole file might be an option because of indexing, but indexing is a bit special in this regard. |
Agreed, raising an error sounds like a good idea. There still is something to be said about partial reads, which could be handled in two ways:
I don't know which scenarios fits better with the numpy philosophy, but the first option sounds more useful. |
I think an error is most reasonable. What I am not sure about right now is if fromfile supports file like objects that do not have a known size, or what currently happens in the case of non-empty sep kwarg. @amuresan the code for fromfile is in C, but if you have a bit of time, we are always very happy about pull requests, and it seems like a reasonable difficulty to dabble a bit into the C (Python) API. |
I believe the problem here is actually that on ubuntu you are getting a A PR with a fix is here: #12354 |
There is a problem with the way in which we handle errors which occur in the call to `PyArray_FromFile` in `np.fromfile`. The problem here is twofold. 1. The return value isn't checked, therefore if we reach the fail block, we will attempt a DECREF on a NULL and go down in flames. 2. The cleanup code on the filepointers (most notabily the call to `npy_PyFile_DupClose2`) assumes that there is no error set to work. This PR addresses these issues 1. By adding a NULL check to the fail block to ensure we don't attempt a DECREF on a NULL. 2. By saving the error state before attempting the cleanup code on the file descriptor, and then restoring it after. Fixes: numpy#12300
fromfile
invalid data and sometimes segfault if reading past the end of a file i.e. it does not check if reading will go past the file end. This issue leads to a segfault on Ubuntu 16.04, but seems to not segfault on OSX.Reproducing code example:
Error message:
Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)
Numpy/Python version information:
platform linux -- Python 3.6.6, pytest-3.8.2, py-1.6.0, pluggy-0.7.1
The text was updated successfully, but these errors were encountered: