Catch and report PyAPS Copernicus Climate Store credential error

[andretheronsa]

Not sure if it is bug report or feature request.

ERA5 PYAPS Tropospheric delay correction currently fails with a hard to diagnose ZeroDivisionError in the progress bar class when the user's login credentials for https://cds.climate.copernicus.eu/ is not correct (or if the user has not accepted the T&Cs). This makes a simple problem (user registration) very hard to figure out.

Going through the code and error stack below reveals that the division error is a result of the date_list2grib_file function not returning a list of data since it does not have access to the service. The progress bar then uses the length of this empty list to set the endpoint of the progress bar resulting in the error.

I think one solution would be to catch the specific error in the try clause (line 398) of the dload_grib_files function in tropo_pyaps3.py.

calcualting` delay for each date using PyAPS (Jolivet et al., 2011; 2014) ...
number of grib files used: 0
Traceback (most recent call last):
File "/home/app/app.py", line 10, in
smallbaselineApp.main()
File "/home/python/MintPy/mintpy/smallbaselineApp.py", line 1069, in main
app.run(steps=inps.runSteps, plot=inps.plot)
File "/home/python/MintPy/mintpy/smallbaselineApp.py", line 1019, in run
self.run_tropospheric_delay_correction(sname)
File "/home/python/MintPy/mintpy/smallbaselineApp.py", line 751, in run_tropospheric_delay_correction
tropo_pyaps3.main(scp_args.split())
File "/home/python/MintPy/mintpy/tropo_pyaps3.py", line 570, in main
get_delay_timeseries(inps, atr)
File "/home/python/MintPy/mintpy/tropo_pyaps3.py", line 523, in get_delay_timeseries
prog_bar = ptime.progressBar(maxValue=num_date)
File "/home/python/MintPy/mintpy/utils/ptime.py", line 277, in init
self.reset()
File "/home/python/MintPy/mintpy/utils/ptime.py", line 282, in reset
self.update_amount(0) # Build progress bar string
File "/home/python/MintPy/mintpy/utils/ptime.py", line 296, in update_amount
percentDone = (diffFromMin / np.float(self.span)) * 100.0
ZeroDivisionError: float division by zero

[yunjunz]

Hi @andretheronsa, I agree that better handling of the error message is desired. I implemented it to not raise Error message in order to support the time-series with the latest acquisition, whose ERA5 data is not available yet. The current way is ignoring it and continue, maybe raising a warning will support both scenarios.

I will put it in the to-do list and try it laterward in 2 months. Feel free to try it if you want to.

A workaround solution is to run the test notebook from pyaps3:
https://github.com/yunjunz/pyaps3/blob/master/examples/TestECMWF.ipynb

[pbrotoisworo]

Hello @yunjunz

I share the MintPy workflow sometimes and I also encounter this problem because they forget to set the pyaps config.

I do this to check for pyaps credentials. Maybe MintPy can implement something similar?
The cfg file can be interpreted by the standard configparser library.

import os
import pyaps3
from configparser import ConfigParser

cfg_path = os.path.join(os.path.dirname(pyaps3.__file__), 'model.cfg')
cfg = ConfigParser()
cfg.read(cfg_path)
key = cfg.get('CDS', 'key')

if not key:
    raise ValueError('pyaps3 credentials are empty!')

[yunjunz]

We could add this in tropo_pyaps3.py#L544. It should at least address one of the two FAQs. Could you issue a PR for it @pbrotoisworo?

[coastalgeo]

I am new to InSAR processing, and ran across this issue as well. With some guidance from the very kind folks at ASF, they directed me to this issue. After some additional digging as to why my MintPy ! smallbaselineApp.py --work-dir {mintpy_dir} {config_file} failed with the ZeroDivisionError, I did the following:

• Reinstalled PyAPS from git repo to start fresh.

• The only key needed for my specific process is [CDS] (key = UID:key) in ~/tools/PyAPS/pyaps3/model.cfg

• For newbies like me, login to the Copernicus website and click on your name in the top right, scroll down and you should see your UID and API key and you should combine them: 123456:ffffffff-ffff-ffff-ffff-ffffffffffff

• The most basic error on my part was to confirm acceptance of the Copernicus products; I could never find it and it never came up in the initial registration/login: https://cds.climate.copernicus.eu/cdsapp/#!/terms/licence-to-use-copernicus-products

Once that was done, the processing I needed worked.