Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor analysis functions to not rely on API provided by Spectrum1D #1065

Open
Cadair opened this issue Jun 14, 2023 · 3 comments
Open

Refactor analysis functions to not rely on API provided by Spectrum1D #1065

Cadair opened this issue Jun 14, 2023 · 3 comments

Comments

@Cadair
Copy link
Member

Cadair commented Jun 14, 2023

At this weeks SunPy community meeting we were discussing spectral analysis of solar data. One of the things we were discussing is how there is lots of functionality in specutils that could be of use to solar physicists, but for various reasons they might not want to use the Spectrum1D class.

Looking down the doc page for Spectral Cubes got me all excited because of the functional approach to analysis, the inheritance of NDCube, and relaxation of the position of the spectral axis, made me think I could apply some functions (like moment) to a generic NDCube or my dkist.Dataset subclass.
However, this lead me to the fact that it uses .flux which in turn calls u.Quantity on the data which would cause all the Dask array backing a dkist.Dataset to be loaded into RAM.

More generically though, for the analysis functions provided by specutils to be generally applicable to data from different sources (instruments / missions) and different packages (dkist / ndcube / sunraster / specutils) I strongly feel that all the analysis code should be written for and tested against the base NDCube API.

@aragilar
Copy link
Contributor

What does NDCube not have from Spectrum1D?

@Cadair
Copy link
Member Author

Cadair commented Jun 15, 2023

The largest things are probably .flux and .spectral_axis, but a variety of other things which are implemented in Spectrum1D. I would like to see things like .spectral_axis become a dynamic function based on the WCS which could be applied to any NCube input in analysis functions, (with shortcuts such as decorators etc if that would be helpful).

Calling back to #651 I am unconvinced from my quick pass through the code that previous discussions about automatically casting from one ndcube subtype to another are a good idea for two main reasons: 1) the conversion is likely to be imperfect (i.e. wont round trip correctly) and 2) a lot of places in specutils assumes that .data is a numpy array, and that's an assumption which is becoming less and less viable.

@rosteen
Copy link
Contributor

rosteen commented Nov 9, 2023

@Cadair Apologies for the extreme delay in replying to this, I've had a tab open with it haunting my conscience for months while hoping that maybe @eteq would pick up my slack 😅. I don't have any fundamental objections to this, but (perhaps obviously) I don't currently have the bandwidth to think about doing this upstreaming myself or to think through potential pitfalls. I'm wary of current specutils getting confused by this stuff moving upstream, but I can also see a path to retaining the current specutils analysis functions in a way that calls the upstream functions without making our users import them directly from NDCube.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants