Create extendible dataset with H5Easy

[jjcasmar]

I want to know if its possible to write an Eigen::Matrix using H5Easy, in such way that the resulting dataset can be extended?

I would like to write something like

Eigen::Map<const Eigen::Matrix<float, Eigen::Dynamic, 3, Eigen::RowMajor>> pointsMap{
    vertices.data()->data(),  //
    static_cast<Eigen::Index>(vertices.size()),
    3};

Eigen::Map<const Eigen::Matrix<float, Eigen::Dynamic, 3, Eigen::RowMajor>> points1Map{
    vertices1.data()->data(),  //
    static_cast<Eigen::Index>(vertices1.size()),
    3};

H5Easy::DumpOptions options{H5Easy::Compression(de::hdf::CompressionLevel)};

H5Easy::dump(parent.getFile(), fmt::format("Cache/POINTS"), pointsMap, options);
H5Easy::dump(parent.getFile(), fmt::format("Cache/POINTS"), points1Map, options);

so that I have a single Cache/POINTS dataset with both Eigen maps concatenated.

If its not possible, how should be done using HighFive "normal" API?

[1uc]

I can't spot anything in Easy that would allow you to do this.

For HighFive, you can look at this example:
https://github.com/BlueBrain/HighFive/blob/master/src/examples/create_extensible_dataset.cpp

You add compression by adding it to props.

The unfortunate twist is that you can't use Eigen matrices directly (we're working on it). However, since you have row-major matrices, you might be able to write directly from the pointer.

You'd use the following line instead:

dataset.select({0, 0}, {3, 1}).write_raw(ptr);

You need to double check that Eigen doesn't pad the inner dimension.

[jjcasmar]

Thanks, I have test this approach and works fine. Eigen doesn't pad anything unless you explicitely ask for, all the data is continuos. In any case, I wasn't actually using Eigen matrices, but Eigen maps, since I was using Easy and was really easy to dump directly the Eigen::Map. I will use the write_raw directly for my new use case.

Thanks