Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for allowing a weight file to be written when it does not exist when using filename argument. #234

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

jr3cermak
Copy link

This might address issues #153 and #202. I had been using xESMF version 0.3.0 and now improving code to use the latest 0.7.0. In version 0.3.0, with Regridding(reuse_weights=True and filename="...") and the filename does not exist, the weights file is created on the first pass and reused in subsequent uses. In version 0.7.0, we encounter this error when the weights file does not exist. We do not pre-generate a weights file,

File "/home/cermak/src/xESMF/xesmf/smm.py", line 74, in _parse_coords_and_values
    raise IOError(f'Weights file not found on disk.\n{indata}')
OSError: Weights file not found on disk.

This patch restores the previous behavior from 0.3.0.

The logic is just a little complex.

If reuse_weights is False, compute new weights each time unless the weights argument is specified.
The weights argument, if a filename, MUST exist.

If reuse_weights is True, use the weights argument if provided. Override weights argument, if filename is provided. If the weights argument is a filename, it MUST exist. If a filename argument is provided, override the weights argument. If the provided filename exists, use it for the weights otherwise compute new weights and trigger saving the weights to the filename in the first pass. Subsequent calls will utilize the new and existing filename.

Maybe add some wording to the function arguments to further clarify use.

@jr3cermak
Copy link
Author

If this PR is approved, the CI test would have to be changed.

The logic would permit that weight file to be created in this case where reuse_weights=True, filename='fakewgts.nc'. It would be an error if 'fakewgts.nc' was not created. If the arguments were reuse_weights=True, weights='fakewgts.nc', that should generate an OSError.

        # check fails on non-existent file
        with pytest.raises(OSError):
>           regridder_reuse = xe.Regridder(
                ds_in, ds_out, method, reuse_weights=True, filename='fakewgts.nc'
            )
E           Failed: DID NOT RAISE <class 'OSError'>

@jr3cermak jr3cermak mentioned this pull request Jan 29, 2023
@raphaeldussin
Copy link
Contributor

@jr3cermak IMHO setting reuse_weights=True to compute and save weights is confusing. The current behavior allows to save weights when reuse_weights=False and filename="something.nc" and then one can re-load the weights using reuse_weights=True and weights="something.nc"

@jr3cermak
Copy link
Author

ok. Is see how the reuse_weights option is worded now. We have arrived at a solution that seems to satisfy both 0.3.0 and 0.7.0. For those that need it:

grd = xesmf.Regridder(...., 
  reuse_weights=os.path.isfile(desired_weight_file),
  filename=desired_weight_file
)

instead of needing to do something like:

if not(os.path.isfile("something.nc")):
  grd = xesmf.Regridder(reuse_weights=False, filename="something.nc")
else:
  grd = xesmf.Regridder(reuse_weights=True, weights="something.nc")

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants