Segmentation fault (core dumped)

[hafiz031]

I am trying to serialize a fasttext model using jsonpickle. It seems to be that, the serialization and de-serialization works without posing any error. But after de-serialization, when I want to make prediction on the de-serialized version of the model by calling predict() method, it throws an error mentioned in the title.

Following is a sample program to reproduce it:

import jsonpickle
import fasttext

model = fasttext.load_model("path/to/.bin/model/file")

# The following works fine before serialization and de-serialization
print(model.predict("SOME STRING FOR TEXT CLASSIFICATION"))
"""Output be like:
(('__label__5310',), array([1.00001001]))
"""

# Now do the same after serializing and de-serializing
serialized_model = jsonpickle.dumps(model)
# De-serialization also done without showing any error
de_serialized_model = jsonpickle.loads(serialized_model)
# Now try to make a prediction, at this point the error will occur
print(de_serialized_model.predict("SOME STRING FOR TEXT CLASSIFICATION")
"""Following is the error:
Segmentation fault (core dumped)
"""

Why this is happening? But surprising enough, both the original version and the version after serialization and de-serialization have the same type:

print(type(model)) # <class 'fasttext.FastText._FastText'>
print(type(de_serialized_model)) # <class 'fasttext.FastText._FastText'>

Thanks!

[Theelx]

I'm not sure, could this be a bug in fasttext? I don't believe jsonpickle uses any c-extensions, so I'd be surprised if the segmentation fault was jsonpickle's fault. That said, if you can provide the specific model file, as well as the version of jsonpickle and fasttext you were using, I can test it myself.

[hafiz031]

@Theelx Fasttext is basically a pybind package. Its back-end is written in C++ and than using pybind a set of python API s are provided. Is it the root cause?

[Theelx]

I'd expect so. In my experience with Cython, pybind, and handwritten c-extensions, I've never seen a segmentation fault come from code that is pure-python, like jsonpickle is. If you could generate a coredump file and step through it with gdb or something, I may be able to help pinpoint if the issue is jsonpickle or fasttext. Keep in mind that fasttext seems to be 2 years old now, and there may be internal issues with python's C API depending on if you use a newer version of python.