Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bad_alloc when running C++ example (writeSubarray, bufView = view) #213

Open
bartoszukm opened this issue Dec 19, 2022 · 0 comments
Open

Comments

@bartoszukm
Copy link

bartoszukm commented Dec 19, 2022

Hello,

I just wanted to start using z5 library in my C++ project. I copied a code from example:

#include "json.hpp"
#include "xtensor/xarray.hpp"

// factory functions to create files, groups and datasets
#include "z5/factory.hxx"
// handles for z5 filesystem objects
#include "z5/filesystem/handle.hxx"
// io for xtensor multi-arrays
#include "z5/multiarray/xtensor_access.hxx"
// attribute functionality
#include "z5/attributes.hxx"

int main() {

  // get handle to a File on the filesystem
  z5::filesystem::handle::File f("data.zr");
  // if you wanted to use a different backend, for example AWS, you
  // would need to use this instead:
  // z5::s3::handle::File f;

  // create the file in zarr format
  const bool createAsZarr = true;
  z5::createFile(f, createAsZarr);

  // create a new zarr dataset
  const std::string dsName = "data";
  std::vector<size_t> shape = { 1000, 1000, 1000 };
  std::vector<size_t> chunks = { 100, 100, 100 };
  auto ds = z5::createDataset(f, dsName, "float32", shape, chunks);

  // write array to roi
  z5::types::ShapeType offset1 = { 50, 100, 150 };
  xt::xarray<float>::shape_type shape1 = { 150, 200, 100 };
  xt::xarray<float> array1(shape1, 42.0);
  z5::multiarray::writeSubarray<float>(ds, array1, offset1.begin());

  // read array from roi (values that were not written before are filled with a fill-value)
  z5::types::ShapeType offset2 = { 100, 100, 100 };
  xt::xarray<float>::shape_type shape2 = { 300, 200, 75 };
  xt::xarray<float> array2(shape2);
  z5::multiarray::readSubarray<float>(ds, array2, offset2.begin());

  // get handle for the dataset
  const auto dsHandle = z5::filesystem::handle::Dataset(f, dsName);

  // read and write json attributes
  nlohmann::json attributesIn;
  attributesIn["bar"] = "foo";
  attributesIn["pi"] = 3.141593;
  z5::writeAttributes(dsHandle, attributesIn);

  nlohmann::json attributesOut;
  z5::readAttributes(dsHandle, attributesOut);
  
  return 0;
}

and unfortunately I get std::bad_alloc and Signal: SIGABRT (Aborted) in a line z5::multiarray::writeSubarray<float>(ds, array1, offset1.begin());.

As I debugged the code, it is a line 366 in xtensor_access.hxx:

// could also implement smart view for this,
// but this would be kind of hard and premature optimization
bufView = view;

I use xtensor-0.24.2, xtensor-blas-0.20.0 and xtl-0.7.4 in my project. The z5 is in version z5-2.0.16. g++ (Ubuntu 10.3.0-1ubuntu1~20.04) 10.3.0.

I tried slightly change a main code, as createAsZarr = false or put more threads in writeSubarray. The outcome remains the same.

I have not linked against the compression codecs, but I believe it is not a problem here.

Could you please help me or give any guidance what is the nature of this problem?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant