ValueError: Big-endian buffer not supported on little-endian compiler

[Elisa-Visentin]

Hi, sorry for disturbing you... I am running gammapy on the same fits files. While I use gammapy 0.20.1 it works, while if I use version 1.0 it fails with the following error:

ValueError                                Traceback (most recent call last)
Cell In[9], line 21
     18 counts.fill_events(observation.events)
     20 # Run the makers to the observation data
---> 21 dataset = dataset_maker.run(dataset_empty.copy(name=str(obs_id)), observation)
     22 dataset_on_off = bkg_maker.run(dataset, observation)
     23 dataset_on_off = safe_mask_maker.run(dataset_on_off, observation)

File ~/mambaforge/envs/gammapy-v1.0/lib/python3.8/site-packages/gammapy/makers/map.py:411, in MapDatasetMaker.run(self, dataset, observation)
    409         edisp = self.make_edisp(dataset.edisp.edisp_map.geom, observation)
    410     else:
--> 411         edisp = self.make_edisp_kernel(
    412             dataset.edisp.edisp_map.geom, observation
    413         )
    415     kwargs["edisp"] = edisp
    417 return dataset.__class__(name=dataset.name, **kwargs)

File ~/mambaforge/envs/gammapy-v1.0/lib/python3.8/site-packages/gammapy/makers/map.py:295, in MapDatasetMaker.make_edisp_kernel(self, geom, observation)
    291 exposure = self.make_exposure_irf(geom.squash(axis_name="energy"), observation)
    293 use_region_center = getattr(self, "use_region_center", True)
--> 295 return make_edisp_kernel_map(
    296     edisp=observation.edisp,
    297     pointing=observation.pointing_radec,
    298     geom=geom,
    299     exposure_map=exposure,
    300     use_region_center=use_region_center,
    301 )

File ~/mambaforge/envs/gammapy-v1.0/lib/python3.8/site-packages/gammapy/makers/utils.py:351, in make_edisp_kernel_map(edisp, pointing, geom, exposure_map, use_region_center)
    348 # Create temporary EDispMap Geom
    349 new_geom = geom.to_image().to_cube([migra_axis, geom.axes["energy_true"]])
--> 351 edisp_map = make_edisp_map(
    352     edisp, pointing, new_geom, exposure_map, use_region_center
    353 )
    355 return edisp_map.to_edisp_kernel_map(geom.axes["energy"])

File ~/mambaforge/envs/gammapy-v1.0/lib/python3.8/site-packages/gammapy/makers/utils.py:299, in make_edisp_map(edisp, pointing, geom, exposure_map, use_region_center)
    296 offset = coords.skycoord.separation(pointing)
    298 # Compute EDisp values
--> 299 data = edisp.evaluate(
    300     offset=offset,
    301     energy_true=coords["energy_true"],
    302     migra=coords["migra"],
    303 )
    305 if not use_region_center:
    306     data = np.average(data, axis=2, weights=weights)

File ~/mambaforge/envs/gammapy-v1.0/lib/python3.8/site-packages/gammapy/irf/core.py:281, in IRF.evaluate(self, method, **kwargs)
    278     if coord is not None:
    279         coords_default[key] = u.Quantity(coord, copy=False)
--> 281 data = self._interpolate(coords_default.values(), method=method)
    283 if self.interp_kwargs["fill_value"] is not None:
    284     idxs = self.axes.coord_to_idx(coords_default, clip=False)

File ~/mambaforge/envs/gammapy-v1.0/lib/python3.8/site-packages/gammapy/utils/interpolation.py:120, in ScaledRegularGridInterpolator.__call__(self, points, method, clip, **kwargs)
    118     points = np.broadcast_arrays(*points)
    119     points_interp = np.stack([_.flat for _ in points]).T
--> 120     values = self._interpolate(points_interp, method, **kwargs)
    121     values = self.scale.inverse(values.reshape(points[0].shape))
    122 else:

File ~/mambaforge/envs/gammapy-v1.0/lib/python3.8/site-packages/scipy/interpolate/_rgi.py:336, in RegularGridInterpolator.__call__(self, xi, method)
    332 if (ndim == 2 and hasattr(self.values, 'dtype') and
    333         self.values.ndim == 2):
    334     # a fast path
    335     out = np.empty(indices.shape[1], dtype=self.values.dtype)
--> 336     result = evaluate_linear_2d(self.values,
    337                                 indices,
    338                                 norm_distances,
    339                                 self.grid,
    340                                 out)
    341 else:
    342     result = self._evaluate_linear(indices, norm_distances)

File _rgi_cython.pyx:19, in scipy.interpolate._rgi_cython.evaluate_linear_2d()

ValueError: Big-endian buffer not supported on little-endian compiler

Notebook: https://github.com/cta-observatory/magic-cta-pipe/blob/v0.3.1/notebooks/check_flux_from_dl3.ipynb

Error in cell 8

I don't know how to solve this issue (I don't even know exactly what this error is: I just know that computers write bytes in little or big endian order, i.e., most significant bit to the left-right; but here I am using the same computer, just two different environments).

Can somebody help me? Thanks

PS: I cannot share the fits files because they contain non-published data, sorry

[registerrier]

Hi @Elisa-Visentin , I suppose this is due to the scipy version which had an issue. See scipy/scipy#17727

Can you check which version of scipy is installed and make sure this is not 1.10?

[Elisa-Visentin]

Yes, it is 0.10.0. Sorry!!! And thank you