Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wavecorr offsets are ignored in spectral resampling #7794

Open
jdavies-st opened this issue Aug 7, 2023 · 11 comments · May be fixed by #8376
Open

wavecorr offsets are ignored in spectral resampling #7794

jdavies-st opened this issue Aug 7, 2023 · 11 comments · May be fixed by #8376

Comments

@jdavies-st
Copy link
Collaborator

jdavies-st commented Aug 7, 2023

So wavecorr corrects for wavelength offset due to non-centering of sources in the MSA (or fixed) slit(let). Due to the nature of fixed centers of the MSA, just about every source is offset to some degree.

But only the point sources are corrected, i.e. all the extra-galactic sources, if labelled EXTENDED, are not corrected at all.

Anyway, the correction is not an update to the meta.wcs GWCS object of each 2D extracted cutout in the _cal.fits file, but an updated wavelength array in that _cal file. This updated wavelength array is not always then used downstream, specifically in resample_spec, where the wavelength array is ignored in the building of the output wcs and specifically in the resampling in reproject when producing the drizzle pixmap, where it just uses the GWCS objects to generate the pixel-to-pixel mapping.

def reproject(wcs1, wcs2):

Not sure if there is an easy fix to this, given that the wavelength offset is interpolated from a model and only written to the wavelength array, and not currently part of the GWCS transform. But it should probably be documented, both for point sources and for extended, point-like sources, i.e. high-redshift galaxies.

@jdavies-st jdavies-st added the bug label Aug 7, 2023
@nden nden added the wavecorr label Aug 7, 2023
@nden
Copy link
Collaborator

nden commented Aug 7, 2023

There's no easy way to include it in the WCS model because one of the inputs is the computed wavelength. And you are right, it is valid only for point sources. This should be made clear in the documentation.
Should explore having a special reproject function for NRS calling also wavecorr.
@stscijgbot

@jdavies-st
Copy link
Collaborator Author

jdavies-st commented Aug 7, 2023

I see. Is it fundamentally different from the velocity aberration correction? I.e. V2 and V3 are inputs to and are modified by

# Compute differential velocity aberration (DVA) correction:
va_corr = pointing.dva_corr_model(
va_scale=input_model.meta.velocity_aberration.scale_factor,
v2_ref=input_model.meta.wcsinfo.v2_ref,
v3_ref=input_model.meta.wcsinfo.v3_ref
) & Identity(1)

and the wavelength is just passed through via Identity(1). Could one do something similar for a wavelength offset on just the wavelengths and passing through V2 and V3? I guess one would need an interpolation astropy model, and it might computationally expensive as well.

@stscijgbot-jp
Copy link
Collaborator

This issue is tracked on JIRA as JP-3330.

@stscijgbot-jp
Copy link
Collaborator

Comment by Sarah Kendrew on JIRA:

I'm not aware of this correction being applied to MIRI LRS data but the discussion is relevant for both slit & slitless operation (also e.g. in the context of JP-3307) so tagging Greg Sloan  & Katherine Murray to put it on their radar.   

@stscijgbot-jp
Copy link
Collaborator

Comment by Nadia Dencheva on JIRA:

This is an internal comment about the code, Mihai Cara

When building the output_wcs for a Nirspec resampled observation, the combined wavelength array is computed in _find_nirspec_output_sampling_wavelengths. The function is written to handle three modes: median, fast and accurate but the mode parameter is never passed and in practice the default median mode is always used. The other two modes are also not tested and it's unclear if they improve resampling in any way.

I propose removing the unreachable code which will simplify the solution for using the output of wavecorr in resample. Mihai Cara , if no objections, I'll remove the code.

@mcara
Copy link
Member

mcara commented Oct 7, 2023

'accurate' was the my original fix to the code that was originally developed by @jdavies-st (see #6747). However, as @hbushouse described it in https://jira.stsci.edu/browse/JP-2612, it is "painfully slow". I wish all of these methods were tested to see if there is any important difference between different methods.

I am not against removing unused code.

@stscijgbot-jp
Copy link
Collaborator

Comment by Nadia Dencheva on JIRA:

I made a simplistic attempt to fix this issue by using the corrected wavelengths in resample_spec.

#8153

This doesn't work. 

Resample requires that the WCS transform for an observation roundtrips. Since the wavelength correction is not part of the model but applied separately, the WCS does not roundtrip. To give some details:

The instrument model is anchored at the center of the slit (0,0). In addition the spectral and spatial coordinates are not independent. To roundtrip the transform the input wavelength should be the one computed at (0, 0). 

I think the correct thing to do is to include the wavelength correction in the model so that the offset can be removed in the inverse. This doesn't seem possible with the current wavecorr reference file. The offset is a 2D lookup table which depends on the wavelength of a pixel computed using the instrument model and the x coordinate of the source in the slit. The 2D array is interpolated to get the offset for a wavelength at a specific input pixel and the result added to the wavelength. Since this is a 2D interpolation there's no general analytical inverse. Since the position of the source for a specific observation is known, this can be rewritten as a 1D interpolation model which has an analytical inverse. The issue with this is that there are degenerate values. Please correct me if any of the above is wrong.

Meanwhile I am setting this issue on hold.

@stscijgbot-jp
Copy link
Collaborator

Comment by Melanie Clarke on JIRA:

Thanks for the update Nadia Dencheva

I have briefly discussed this issue with Peter Zeidler and Charles Proffitt.  We agree that if it's possible, it would be best to add the correction to the model so that the transforms have all the information.  We can discuss how we might approach that and let you know what we come up with.

@hayescr hayescr linked a pull request Mar 20, 2024 that will close this issue
7 tasks
@stscijgbot-jp
Copy link
Collaborator

Comment by Christian Hayes on JIRA:

We took a look into the wavelength corrections on the NIRSpec team and have a potential solution for adding the corrections into the WCS model in a draft PR here:

#8376

I've also attached some slides that were discussed with the NIRSpec calibration team that briefly describe the proposed solution and show some examples.  In particular, slide 4 motivates updating the sign of the corrections in the PR (the corrections are currently subtracted, but should instead be added).  Slides 8 and 9 show the spec2 x1d products for a few spectrally dithered observations using the current version of the wavelength correction (labeled as Old, which is not propagated through the resampling) and the proposed updates in the PR (labeled New), which shows the result of adding the wavelength corrections to the WCS.

[^wavecorr_updates_nirspec_cal_presentation.pdf]

@stscijgbot-jp
Copy link
Collaborator

Comment by Nadia Dencheva on JIRA:

Looking at the plots in the original zero point correction technical report, it's clear that the correction includes the sign and should be added. Unfortunately the same report says it should be subtracted and that's what was implemented.

The implementation matches the description. I tested the new implementation with a prism MOS observation and the differences are relatively large, as mentioned in the attached note. If the team considers these acceptable then the implementation is OK. 

Have you tested it with a FS observation?

Have you run tests with resample using the new implementation?

Are you planning to make this note accessible in a more formal way so that the next developer knows what is implemented and why it's different from the technical memo? 

 

@stscijgbot-jp
Copy link
Collaborator

Comment by Christian Hayes on JIRA:

Yes, I've tested this with FS observations (for a multiple different dispersers) and it corrects the wavelengths as expected.  I've also ran FS and MOS observations through resampling using the new implementation and can confirm that the wavelength correction from the new implementation is included in the WCS and propagated through to the resampled spectra (both the s2d and x1d spectra).  There are a few examples of the FS and resampled (x1d) spectra with the new implementation in the above presentation, in case you would like some examples.

We hadn't put anything together yet, but yes, it would be good to more formally document this implementation and what motivated it.  We will work on writing this up.

 

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment