Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG-REPORT] HDF5 file remains locked after close #2410

Open
intelligibledata opened this issue Jan 11, 2024 · 0 comments
Open

[BUG-REPORT] HDF5 file remains locked after close #2410

intelligibledata opened this issue Jan 11, 2024 · 0 comments

Comments

@intelligibledata
Copy link

Thank you for reaching out and helping us improve Vaex!

Before you submit a new Issue, please read through the documentation. Also, make sure you search through the Open and Closed Issues - your problem may already be discussed or addressed.

Description
I need to change the data in a column in an existing HDF5 file and write it back to the file. The problem is that as soon as I use and existing column from the df to change the data the HDF5 file gets locked and I cannot write to it or delete/replace it without closing the application (which I do not want to do since its a dash board). I reduced the problem to the following script:

`import vaex as vx
original_file=""
tempfile = ""
)
df = vx.open(original_file)
dftemp = df.copy()
dftemp["new column1"] = vx.vconstant("test", dftemp.shape[0])
#dftemp["new column2"] = dftemp["lockprofile"]
#dftemp.drop(dftemp["index"])
dftemp.export_hdf5(tempfile, progress=True)

dftemp.close()
df.close()`

With the 2 lines commented out this script works correctly and both files are unlocked after running this script in jupyter. When uncommenting one of the 2 other lines I get the following error: "could not close memmap ... dataset_mmap.py:94"

In the example above I tried to write to another file since writing to the same file is also not possible due to the lock on the file.

Also when you do not write the data to another file the lock is place on the hdf5 file as soon as you use an existing column to add a new column and never released until the application is closed.

Software information

  • Vaex version (import vaex; vaex.__version__): 4.17.0
  • Vaex was installed via: pip
  • OS: Windows 10 enterprise

Additional information
Please state any supplementary information or provide additional context for the problem (e.g. screenshots, data, etc..).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant