np.fromfile() raises non-helpful error when reading non-numerical text

[Doom312]

When I try to read strings from file to numpy array Python crashes.

Reproducing code example:

import sys
print(sys.platform)
print(sys.version)

import numpy as np
print(np.version.full_version)

np.fromfile('data\\foo.txt', dtype='str')

win32
3.7.1 (default, Oct 28 2018, 08:39:03) [MSC v.1912 64 bit (AMD64)]
1.15.4

The file contains:

[data\foo.txt content -- this line is not there]
dsadas
asdasdads
asdasda
dasdasda

Error message:

There is no error message just the Python has stopped working window:

---

Summary by @seberg:

As noted below, this is now working, but gives a strange ValueError(for 'S' dtype without a size).

[charris]

Is this problem new in Python 3.7.1?

[mattip]

Strange no-one hit this before. Using np.fromfile with

• a dtype with variable itemsize (str, unicode, void) and
• not specifying count

does not work, it passes -1 to PyArray_NewFromDescr_int. However it should raise an error, not crash the interpreter.

Where did you get numpy? Are you running python inside an IDE or jupyter, or directly from the command line?

[mattip]

Does

with open('data\\foo.txt', 'rt') as fid:
    for line in fid:
        print line

print the file contents?

[cgohlke]

I think this is fixed by #12354. The crash is due to a DECREF on a NULL at https://github.com/numpy/numpy/blob/maintenance/1.15.x/numpy/core/src/multiarray/multiarraymodule.c#L2178

[mattip]

While this no longer crashes, it raises ValueError: negative dimensions are not allowed. np.loadtxt with no dtype is no better, it returns an uninitialized float64 ndarray.