You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I would like to add to the documentation information about which file loaders support the dask-distributed backend. Mostly just add an extra column here
Currently I believe that this is only the zspy and the new file loader #11 but we can think about adding in support for the hspy file format as well as any of the other binary files.
Describe the context
I have defined a function in #11 that works as a drop in replacement for np.memmap and allows for distributed loading of some data. This is particularly useful for large data sets as well as does a much better job handling the available resources.
Additional information
Using the dask-distributed scheduler is the preferred way to interact with dask in most cases. Supporting distributed schedulers at the loading level is important for larger datasets and allows for much better scalable preformance.
The text was updated successfully, but these errors were encountered:
Yes, sounds good. This table is created manually and it would better to have this information available in the specification and generate table automatically as we do with hs.print_known_signal_types() for example.
Describe the functionality you would like to see.
I would like to add to the documentation information about which file loaders support the
dask-distributed
backend. Mostly just add an extra column hereCurrently I believe that this is only the
zspy
and the new file loader #11 but we can think about adding in support for thehspy
file format as well as any of the other binary files.Describe the context
I have defined a function in #11 that works as a drop in replacement for
np.memmap
and allows for distributed loading of some data. This is particularly useful for large data sets as well as does a much better job handling the available resources.Additional information
Using the
dask-distributed
scheduler is the preferred way to interact withdask
in most cases. Supporting distributed schedulers at the loading level is important for larger datasets and allows for much better scalable preformance.The text was updated successfully, but these errors were encountered: