simple optional path to download HDF5 from conda and statically link

[pmarks]

We've been maintaining this as separate branch for a while: it works really well for us to get reliable builds on diverse machines without requiring an HDF5 installation & to have unit test in downstream crates work seamlessly.

Is this something you'd be open to? Any suggestions on how you'd like to approach it differently?

[mulimoen]

Will this clash with zlib-sys?

[mulimoen]

This is a whole lot of (optional) dependencies to add to the project to solve something package managers should be able to do. We also have the static build for easy bundling, although this adds a lot of build-time. Would a solution be to adapt the build script to allow HDF5_DIR to accept a statically built library?

If we go ahead with a pure conda option we should add a new crate (hdf5-conda?) and instead plumb it like hdf5-src.

[aldanor]

Ok, given this is the last of the old hanging PRs, what's the deal with this?

[mulimoen]

A build time installation of a package only supporting x64 seems a bit backwards to me. Building from source remains a viable option which works on most platforms, and is very easy to configure, although expensive to compile and hard to use for other packages. Half of the problem could be solved with #106.

@pmarks or @luiz10x Is this used by you in the current form?

[pmarks]

@pmarks or @luiz10x Is this used by you in the current form?

Yes, we're still using this mode -- it's been the best way for us to make tools that use hdf5 'just work', without requiring the user to install anything (including cmake)

A build time installation of a package only supporting x64 seems a bit backwards to me.

It doesn't actually install the package, it just downloads it and extracts the static library & instructs cargo to link to it.

[aldanor]

@pmarks I think if we're to include it in the base distribution, we better make it flexible enough. Few questions:

• why is zlib package is downloaded separately from conda? Is that the only way to do it? can't we use zlib-sys?
• why anaconda and not conda-forge?
• how do you select HDF5 version?
• how do you switch between nompi/openmpi/mpich builds?
• why the need to specify OUT_DIR if it's just a temporary build artifact?

[aldanor]

Regardless, I think this is too long of a discussion and there's too many unknowns at the moment to resolve it quickly, so let's consider it for 0.9 so it doesn't hold off releasing the 0.8 which is already a humongous release. (The questions listed above need to be answered first anyhow - and then we can think of how to proceed if we all agree on something)

[pmarks]

Agreed, it needs more thought so shouldn't be considered for 0.8.

* why is zlib package is downloaded separately from conda? Is that the only way to do it? can't we use zlib-sys?

We probably can, I'll try. I probably didn't know how to do this when I originally made the patch.

* why anaconda and not conda-forge?

I don't have a strong opinion. I thought anaconda was more 'official', but I don't really know the difference.

* how do you select HDF5 version?

It's hardcoded right now, but easy enough to support more versions. Do people think it's valuable to be able to select a version in the conda mode?

* how do you switch between nompi/openmpi/mpich builds?

conda has all these, so maybe it's possible? There's a note in the readme saying mpi isn't support in static mode - is that still true? Would that preclude using mpi in this case?

* why the need to specify `OUT_DIR` if it's just a temporary build artifact?

I thought OUT_DIR was meant to be the scratch space to use if you need to write anything from within build.rs. See here. I've seen folks refactoring build.rs script to move temp files into OUT_DIR so I was following that pattern. Are there drawbacks to doing it this way?

[mulimoen]

I've had a go at this which resulted in the following branch. This moves the extraction to a helper crate, decreasing dev-dependencies. It seems dependencies for hdf5-c must also be downloaded and linked appropriately (on non-linux), which might conflict badly with libraries installed from elsewhere. Is support for macos and windows necessary, or could somebody with experience maybe try to fix this for these platforms?