Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add test and fix conservative interpolation algorithm #635

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

jbusecke
Copy link
Contributor

@jbusecke jbusecke commented May 3, 2024

Hey folks, I was just messing around with an example how to transform some tracer (e.g. temperature) from one depth bin layout to another in a way to make sure that the vertical integral is conserved.
This should be easy in theory by doing something like this:

# get cell thickness from outer coordinate
dz = ds.z_outer.diff('z_outer').rename({'z_outer':'z'})

Define the target bin (edges) and compute their thickness
z_bins = xr.DataArray([...], dims=['lev'])
dz_regridded = z_bins.diff('lev')

# convert to vertical integral of tracer
ds_extensive = ds * dz 
# conservatively transform
ds_extensive_transformed = grid.transform(ds_extensive, 'Z', target_data=ds.z_outer, method='conservative')
# convert back to tracer concentrations
ds_transformed = ds_extensive_transformed / dz_regridded

and this should then preserve the vertical integral of the tracer!

I just found out it does NOT if there are nans in the input data (which is pretty common 😝).

What happened was that in this line, any nan value in the input would result in the output bin being nan.

The simple test I added maybe shows this better:

    phi = np.array([1, 2, np.nan])
    theta = np.array([30, 40, 50, 60])
    target = np.array([30, 50])
    # this is basically transforming some arbitrary data into a single output cell (equivalent to a sum over the axis dimension)
    out = interp_1d_conservative(phi, theta, target)

The current code returns nan for this, whereas the fix implemented here returns 3. To me that seems like the desired behavior.

Would love to hear from @rabernat @dcherian @TomNicholas if that all seems right to you?

I also changes another behavior here, and we should discuss if this is desired at all, and if so, if I should factor it out to another PR.
The fixed behavior basically will set every output bin to zero if none of the input cells have any data (e.g. when both the input and output depths cells are below the sea floor. This seems problematic to me since we now have lost any indication/separation between a cell that happens to have a value of zero, vs a cell that does not have any data at all.

I changed this behavior to set cells to nan if there was never any input data that matched them. It is worth noting that the current behavior does 'mask out' cells with all nan input (but also masks out others!), so in a sense combining these changes would not really change the way the output is presented to the user, it just fixes the above bug.

Let me know if you can think of any issues arising from this.

  • Tests added
  • Passes pre-commit run --all-files
  • User visible changes (including notable bug fixes) are documented in whats-new.rst
  • New functions/methods are listed in api.rst
  • Add example for 'conservative depth to depth transform to docs'

Comment on lines +128 to +131
if np.isnan(output[j]):
output[j] = phi[i]
else:
output[j] += phi[i]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am unsure how much of a performance hit we might get from this additional if/else statement...

@jbusecke
Copy link
Contributor Author

jbusecke commented May 3, 2024

Also i realize that 'easy' in the above example is a bit of a stretch and we might want to support doing this within xgcm, but for now I want to focus on the bug.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant