h5py requirement of stardist v0.7.3 not compatible with tensorflow >2.5

[ElpadoCan]

Describe the bug
Tensorflow > 2.5 has the requirement h5py~=3.1.0, but stardist 0.7.3 requires h5py<3. I know this comes from CSBDeep package, but is there anything that can be done? I cannot currently upgrade to python >= 3.9.

To reproduce
To reproduce simply install tensorflow 2.5.2 and then stardist. Stardist installation will downgrade h5py and pip will show the following error

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. tensorflow 2.5.2 requires h5py~=3.1.0, but you have h5py 2.10.0 which is incompatible.

Expected behavior
I think h5py installation should be handled by tensorflow.

Data and screenshots

Environment (please complete the following information):

• Python version [3.8.5]
• StarDist version [0.7.3]
• CSBDeep version [0.6.3]
• TensorFlow version [2.52]
• OS: [Windows 10 - 64bit]
• GPU memory (if applicable): no GPU

[maweigert]

Does the pip error actually have any consequences?

For me

pip install tensorflow==2.5.2
pip install  h5py==2.10.0
pip install stardist

installs stardist 0.7.3 (despite raising the same error/warning as you get) and everything works fine afterwards.

[uschmidt83]

Yes, the h5py version issue is annoying. In case you haven't done so, you may read the CSBDeep release notes to explain why we did this.

As @maweigert pointed out above, you can check if everything still works fine with the older version 2.10.0 of h5py, regardless of claimed version requirements by TensorFlow. However, it could very well be that the model serialization issues in Keras have been fixed now to work with h5py 3.1.0, which you can test by saving and loading models in CSBDeep/StarDist.

[uschmidt83]

The bug is presumably fixed in TensorFlow > 2.4, if one can believe this: tensorflow/tensorflow#44467 (comment)

That means CSBDeep/StarDist should work with h5py >= 3.0 in this case. If this were true, I still wouldn't know how to update the CSBDeep package requirements, since the required version of h5py would be conditional on the version of TensorFlow. My hope is that everybody just moves on to Python 3.9 eventually, where this issue is not relevant.

[ElpadoCan]

Thanks for the quick answers!

At the moment I'm using StarDist only for prediction which does not require h5py so everything is fine, the problem is that if I put both tensorflow>=2.4 and stardist in the requirements of the project that I want to deploy pip would start that useless backtracking and it would try ALL possible tensorflow versions to try to solve this compatibility issue and it would never succeed. Essentially adding stardist to the requirements of my project broke my environment installation (damn pip).

Now my question is, what is best to do in this case? How do I deploy my project with both tensorflow and stardist? Thanks!

[ElpadoCan]

The bug is presumably fixed in TensorFlow > 2.4, if one can believe this: tensorflow/tensorflow#44467 (comment)

That means CSBDeep/StarDist should work with h5py >= 3.0 in this case. If this were true, I still wouldn't know how to update the CSBDeep package requirements, since the required version of h5py would be conditional on the version of TensorFlow. My hope is that everybody just moves on to Python 3.9 eventually, where this issue is not relevant.

Maybe I can ask on CSBDeep Github page to remove that requirement, if possible?

[uschmidt83]

How do I deploy my project with both tensorflow and stardist?

Can you run a script to install the requirements? In the case, you can first install TensorFlow, and then later CSBDeep/StarDist, which should work with just a warning.

Maybe I can ask on CSBDeep Github page to remove that requirement, if possible?

We also maintain the CSBDeep library, so the answer would be the same ;)

I'm reluctant to remove it at the moment...

[ElpadoCan]

How do I deploy my project with both tensorflow and stardist?

Can you run a script to install the requirements? In the case, you can first install TensorFlow, and then later CSBDeep/StarDist, which should work with just a warning.

Maybe I can ask on CSBDeep Github page to remove that requirement, if possible?

We also maintain the CSBDeep library, so the answer would be the same ;)

I'm reluctant to remove it at the moment...

Sorry, I didn't know you maintain also CSBDeep, I already asked, feel free to ignore it.

I also plan to move to Python 3.9 as soon as possible, but we have several users on Python 3.8 using our software and it would be too much to ask all of them to migrate.

I will look into using a script to install the requirements, never tried before. Thanks anyway!

[uschmidt83]

I will look into using a script to install the requirements, never tried before. Thanks anyway!

I think that would be the easiest solution for now.