Reference point errors

[AuRevoirzzz]

Hi,
I encountered an error when processing Sentinel-1 data with MintPy during the step of selecting ground control points (GCPs). Regardless of whether I manually selected points within the area of interest, points outside the area of interest, or allowed MintPy to automatically choose the points, different errors occurred each time. Additionally, even when I set mintpy.reference.maskFile = no in the cfg file, an error was still reported.
here is my cfg file code

# vim: set filetype=cfg:
# mintpy.load.processor      = aria  #[isce, aria, hyp3, gmtsar, snap, gamma, roipac], auto for isce
#---------interferogram datasets:
# mintpy.load.unwFile        = ../stack/unwrapStack.vrt
# mintpy.load.corFile        = ../stack/cohStack.vrt
# mintpy.load.connCompFile   = ../stack/connCompStack.vrt

#---------geometry datasets:
mintpy.load.processor        = isce
##---------for ISCE only:
mintpy.load.metaFile         = ../reference/IW*.xml
mintpy.load.baselineDir      = ../baselines
##---------interferogram datasets:
mintpy.load.unwFile          = ../merged/interferograms/*/filt_*.unw
mintpy.load.corFile          = ../merged/interferograms/*/filt_*.cor
mintpy.load.connCompFile     = ../merged/interferograms/*/filt_*.unw.conncomp
##---------geometry datasets:
mintpy.load.demFile          = ../merged/geom_reference/hgt.rdr
mintpy.load.lookupYFile      = ../merged/geom_reference/lat.rdr
mintpy.load.lookupXFile      = ../merged/geom_reference/lon.rdr
mintpy.load.incAngleFile     = ../merged/geom_reference/los.rdr
mintpy.load.azAngleFile      = ../merged/geom_reference/los.rdr
mintpy.load.shadowMaskFile   = ../merged/geom_reference/shadowMask.rdr

##---------multilook (optional):
## multilook while loading data with the specified method, to reduce dataset size
## method - nearest, mean and median methods are applicable to interferogram/ionosphere/offset stack(s), except for:
##   connected components and all geometry datasets, for which nearest is hardwired.
## Use mean / median method with caution! It could smoothen the noise for a better SNR, but it could also smoothen the
##   unwrapping errors, breaking the integer 2pi relationship, which is used in the unwrapping error correction.
##   If you really want to increase the SNR, consider re-generate your stack of interferograms with more looks instead.
mintpy.multilook.method     = nearest
mintpy.multilook.ystep      = 2    #[int >= 1], auto for 1 - no multilooking
mintpy.multilook.xstep      = 8    #[int >= 1], auto for 1 - no multilooking

# mintpy.reference.lalo        = auto
mintpy.reference.maskFile      = no

########## 4. correct_unwrap_error (optional)
## supported methods:
## a. phase_closure          - suitable for highly redundant network
## b. bridging               - suitable for regions separated by narrow decorrelated features, e.g. rivers, narrow water bodies
## c. bridging+phase_closure - recommended when there is a small percentage of errors left after bridging
mintpy.unwrapError.method = bridging

## phase_closure options:
## numSample - a region-based strategy is implemented to speedup L1-norm regularized least squares inversion.
##     Instead of inverting every pixel for the integer ambiguity, a common connected component mask is generated,
##     for each common conn. comp., numSample pixels are radomly selected for inversion, and the median value of the results
##     are used for all pixels within this common conn. comp.
# mintpy.unwrapError.numSample       = auto  #[int>1], auto for 100, number of samples to invert for common conn. comp.

## bridging options:
## ramp - a phase ramp could be estimated based on the largest reliable region, removed from the entire interferogram
##     before estimating the phase difference between reliable regions and added back after the correction.
## bridgePtsRadius - half size of the window used to calculate the median value of phase difference
# mintpy.unwrapError.ramp            = auto  #[linear / quadratic], auto for no; recommend linear for L-band data
# mintpy.unwrapError.bridgePtsRadius = auto  #[1-inf], auto for 50, half size of the window around end points

########## 8. correct_troposphere (optional but recommended)
mintpy.troposphericDelay.method = gacos
mintpy.troposphericDelay.gacosDir = ../aps

########## 9. deramp (optional)
## Estimate and remove a phase ramp for each acquisition based on the reliable pixels.
## Recommended for localized deformation signals, i.e. volcanic deformation, landslide and land subsidence, etc.
## NOT recommended for long spatial wavelength deformation signals, i.e. co-, post- and inter-seimic deformation.
mintpy.deramp          = quadratic  #[no / linear / quadratic], auto for no - no ramp will be removed

and these are error imformations
when the reference point inside the area of intrest

******************** step - reference_point ********************

generate_mask.py /media/aurevoir/kaiding_mintpy/Process/mintpy/inputs/ifgramStack.h5 --nonzero -o /media/aurevoir/kaiding_mintpy/Process/mintpy/maskConnComp.h5 --update
input ifgramStack file: /media/aurevoir/kaiding_mintpy/Process/mintpy/inputs/ifgramStack.h5
--------------------------------------------------
update mode: ON
1) output file /media/aurevoir/kaiding_mintpy/Process/mintpy/maskConnComp.h5 already exists.
2) output file is NOT newer than input dataset: connectComponent.
run or skip: run.
calculate the common mask of pixels with non-zero connectComponent value
[==================================================] 295/295    1s /     0s
delete exsited file: /media/aurevoir/kaiding_mintpy/Process/mintpy/maskConnComp.h5
create HDF5 file: /media/aurevoir/kaiding_mintpy/Process/mintpy/maskConnComp.h5 with w mode
create dataset /mask of bool       in size of (698, 341)           with compression=None
finished writing to /media/aurevoir/kaiding_mintpy/Process/mintpy/maskConnComp.h5
time used: 00 mins 2.0 secs.

temporal_average.py /media/aurevoir/kaiding_mintpy/Process/mintpy/inputs/ifgramStack.h5 --dataset coherence -o /media/aurevoir/kaiding_mintpy/Process/mintpy/avgSpatialCoh.h5 --update
--------------------------------------------------
update mode: ON
1) output file /media/aurevoir/kaiding_mintpy/Process/mintpy/avgSpatialCoh.h5 already exists.
2) output file is NOT newer than input dataset: coherence.
run or skip: run.
calculate the temporal average of coherence in file /media/aurevoir/kaiding_mintpy/Process/mintpy/inputs/ifgramStack.h5 ...
[==================================================] lines 698/698 
delete exsited file: /media/aurevoir/kaiding_mintpy/Process/mintpy/avgSpatialCoh.h5
create HDF5 file: /media/aurevoir/kaiding_mintpy/Process/mintpy/avgSpatialCoh.h5 with w mode
create dataset /coherence of float32    in size of (698, 341)           with compression=None
finished writing to /media/aurevoir/kaiding_mintpy/Process/mintpy/avgSpatialCoh.h5
time used: 00 mins 0.3 secs


reference_point.py /media/aurevoir/kaiding_mintpy/Process/mintpy/inputs/ifgramStack.h5 -t /media/aurevoir/kaiding_mintpy/Process/mintpy/smallbaselineApp.cfg -c /media/aurevoir/kaiding_mintpy/Process/mintpy/avgSpatialCoh.h5 --lookup /media/aurevoir/kaiding_mintpy/Process/mintpy/inputs/geometryRadar.h5
--------------------------------------------------
reading reference info from template: /media/aurevoir/kaiding_mintpy/Process/mintpy/smallbaselineApp.cfg
input reference point in lat/lon: (30.6675, 102.04)
input reference point in y/x: (448, 93)
mask: maskConnComp.h5
Traceback (most recent call last):
  File "/home/aurevoir/anaconda3/envs/isce2_env/bin/smallbaselineApp.py", line 10, in <module>
    sys.exit(main())
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/cli/smallbaselineApp.py", line 208, in main
    run_smallbaselineApp(inps)
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/smallbaselineApp.py", line 1117, in run_smallbaselineApp
    app.run(steps=inps.runSteps)
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/smallbaselineApp.py", line 880, in run
    self.run_reference_point(sname)
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/smallbaselineApp.py", line 306, in run_reference_point
    mintpy.cli.reference_point.main(iargs)
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/cli/reference_point.py", line 141, in main
    inps = read_reference_input(inps)
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/reference_point.py", line 360, in read_reference_input
    raise ValueError(msg)
ValueError: input reference point is in masked OUT area defined by maskConnComp.h5!

when the reference point outside the area of intrest

******************** step - reference_point ********************

generate_mask.py /media/aurevoir/kaiding_mintpy/Process/mintpy/inputs/ifgramStack.h5 --nonzero -o /media/aurevoir/kaiding_mintpy/Process/mintpy/maskConnComp.h5 --update
input ifgramStack file: /media/aurevoir/kaiding_mintpy/Process/mintpy/inputs/ifgramStack.h5
--------------------------------------------------
update mode: ON
1) output file /media/aurevoir/kaiding_mintpy/Process/mintpy/maskConnComp.h5 already exists.
2) output file is newer than input dataset: connectComponent.
run or skip: skip.

temporal_average.py /media/aurevoir/kaiding_mintpy/Process/mintpy/inputs/ifgramStack.h5 --dataset coherence -o /media/aurevoir/kaiding_mintpy/Process/mintpy/avgSpatialCoh.h5 --update
--------------------------------------------------
update mode: ON
1) output file /media/aurevoir/kaiding_mintpy/Process/mintpy/avgSpatialCoh.h5 already exists.
2) output file is newer than input dataset: coherence.
run or skip: skip.

reference_point.py /media/aurevoir/kaiding_mintpy/Process/mintpy/inputs/ifgramStack.h5 -t /media/aurevoir/kaiding_mintpy/Process/mintpy/smallbaselineApp.cfg -c /media/aurevoir/kaiding_mintpy/Process/mintpy/avgSpatialCoh.h5 --lookup /media/aurevoir/kaiding_mintpy/Process/mintpy/inputs/geometryRadar.h5
--------------------------------------------------
reading reference info from template: /media/aurevoir/kaiding_mintpy/Process/mintpy/smallbaselineApp.cfg
/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/numpy/core/fromnumeric.py:3464: RuntimeWarning: Mean of empty slice.
  return _methods._mean(a, axis=axis, dtype=dtype,
/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/numpy/core/_methods.py:184: RuntimeWarning: invalid value encountered in divide
  ret = um.true_divide(
Traceback (most recent call last):
  File "/home/aurevoir/anaconda3/envs/isce2_env/bin/smallbaselineApp.py", line 10, in <module>
    sys.exit(main())
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/cli/smallbaselineApp.py", line 208, in main
    run_smallbaselineApp(inps)
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/smallbaselineApp.py", line 1117, in run_smallbaselineApp
    app.run(steps=inps.runSteps)
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/smallbaselineApp.py", line 880, in run
    self.run_reference_point(sname)
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/smallbaselineApp.py", line 306, in run_reference_point
    mintpy.cli.reference_point.main(iargs)
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/cli/reference_point.py", line 141, in main
    inps = read_reference_input(inps)
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/reference_point.py", line 339, in read_reference_input
    inps.ref_x) = coord.geo2radar(np.array(inps.ref_lat),
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/objects/coord.py", line 306, in geo2radar
    az, rg = self._get_lookup_row_col(
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/objects/coord.py", line 188, in _get_lookup_row_col
    raise RuntimeError(f'No corresponding coordinate found for y/x: {y}/{x}')
RuntimeError: No corresponding coordinate found for y/x: 30.88/101.88

when I set mintpy.reference.maskFile = no in the cfg file

******************** step - correct_unwrap_error ********************

unwrap_error_bridging.py /media/aurevoir/kaiding_mintpy/Process/mintpy/inputs/ifgramStack.h5 --template /media/aurevoir/kaiding_mintpy/Process/mintpy/smallbaselineApp.cfg --update
read options from template file: smallbaselineApp.cfg
--------------------------------------------------
update mode: ON
1) output dataset: unwrapPhase_bridging exists
2) output dataset is newer than input dataset: unwrapPhase
3) NOT all key configuration parameters are the same: ['waterMaskFile', 'connCompMinArea', 'ramp', 'bridgePtsRadius'].
	connCompMinArea	: no --> 2500.0
	bridgePtsRadius	: no --> 50
run or skip: run.
--------------------------------------------------
correct unwrapping error in /media/aurevoir/kaiding_mintpy/Process/mintpy/inputs/ifgramStack.h5 with bridging ...
open /media/aurevoir/kaiding_mintpy/Process/mintpy/inputs/ifgramStack.h5 with r+ mode
input  dataset: unwrapPhase
output dataset: unwrapPhase_bridging
access /unwrapPhase_bridging of np.float32 in size of (295, 698, 341)
[======================= 62% ==>                   ] 20210127_20210208   10s /     6sTraceback (most recent call last):
  File "/home/aurevoir/anaconda3/envs/isce2_env/bin/smallbaselineApp.py", line 10, in <module>
    sys.exit(main())
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/cli/smallbaselineApp.py", line 208, in main
    run_smallbaselineApp(inps)
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/smallbaselineApp.py", line 1117, in run_smallbaselineApp
    app.run(steps=inps.runSteps)
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/smallbaselineApp.py", line 886, in run
    self.run_unwrap_error_correction(sname)
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/smallbaselineApp.py", line 351, in run_unwrap_error_correction
    mintpy.cli.unwrap_error_bridging.main(iargs_bridge)
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/cli/unwrap_error_bridging.py", line 137, in main
    run_unwrap_error_bridging(
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/unwrap_error_bridging.py", line 155, in run_unwrap_error_bridging
    cc_obj.label(min_area=cc_min_area)
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/objects/conncomp.py", line 175, in label
    raise ValueError('input reference point is NOT included in the connectComponent.')
ValueError: input reference point is NOT included in the connectComponent.

when allowed MintPy to automatically choose the points

******************** step - correct_unwrap_error ********************

unwrap_error_bridging.py /media/aurevoir/kaiding_mintpy/Process/mintpy/inputs/ifgramStack.h5 --template /media/aurevoir/kaiding_mintpy/Process/mintpy/smallbaselineApp.cfg --update
read options from template file: smallbaselineApp.cfg
--------------------------------------------------
update mode: ON
1) output dataset: unwrapPhase_bridging exists
2) output dataset is newer than input dataset: unwrapPhase
3) NOT all key configuration parameters are the same: ['waterMaskFile', 'connCompMinArea', 'ramp', 'bridgePtsRadius'].
	connCompMinArea	: no --> 2500.0
	bridgePtsRadius	: no --> 50
run or skip: run.
--------------------------------------------------
correct unwrapping error in /media/aurevoir/kaiding_mintpy/Process/mintpy/inputs/ifgramStack.h5 with bridging ...
open /media/aurevoir/kaiding_mintpy/Process/mintpy/inputs/ifgramStack.h5 with r+ mode
input  dataset: unwrapPhase
output dataset: unwrapPhase_bridging
access /unwrapPhase_bridging of np.float32 in size of (295, 698, 341)
[======================= 62% ==>                   ] 20210127_20210208   10s /     6sTraceback (most recent call last):
  File "/home/aurevoir/anaconda3/envs/isce2_env/bin/smallbaselineApp.py", line 10, in <module>
    sys.exit(main())
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/cli/smallbaselineApp.py", line 208, in main
    run_smallbaselineApp(inps)
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/smallbaselineApp.py", line 1117, in run_smallbaselineApp
    app.run(steps=inps.runSteps)
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/smallbaselineApp.py", line 886, in run
    self.run_unwrap_error_correction(sname)
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/smallbaselineApp.py", line 351, in run_unwrap_error_correction
    mintpy.cli.unwrap_error_bridging.main(iargs_bridge)
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/cli/unwrap_error_bridging.py", line 137, in main
    run_unwrap_error_bridging(
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/unwrap_error_bridging.py", line 155, in run_unwrap_error_bridging
    cc_obj.label(min_area=cc_min_area)
  File "/home/aurevoir/anaconda3/envs/isce2_env/lib/python3.8/site-packages/mintpy/objects/conncomp.py", line 175, in label
    raise ValueError('input reference point is NOT included in the connectComponent.')
ValueError: input reference point is NOT included in the connectComponent.

I am seeking assistance to resolve this issue.
THANKS!

[AuRevoirzzz]

when I set mintpy.reference.maskFile = no in the cfg file and allow MintPy to automatically choose the points. They have same information about reference pionts and The reference point code is

******************** step - reference_point ********************

generate_mask.py /media/aurevoir/kaiding_mintpy/Process/mintpy/inputs/ifgramStack.h5 --nonzero -o /media/aurevoir/kaiding_mintpy/Process/mintpy/maskConnComp.h5 --update
input ifgramStack file: /media/aurevoir/kaiding_mintpy/Process/mintpy/inputs/ifgramStack.h5
--------------------------------------------------
update mode: ON
1) output file /media/aurevoir/kaiding_mintpy/Process/mintpy/maskConnComp.h5 already exists.
2) output file is newer than input dataset: connectComponent.
run or skip: skip.

temporal_average.py /media/aurevoir/kaiding_mintpy/Process/mintpy/inputs/ifgramStack.h5 --dataset coherence -o /media/aurevoir/kaiding_mintpy/Process/mintpy/avgSpatialCoh.h5 --update
--------------------------------------------------
update mode: ON
1) output file /media/aurevoir/kaiding_mintpy/Process/mintpy/avgSpatialCoh.h5 already exists.
2) output file is newer than input dataset: coherence.
run or skip: skip.

reference_point.py /media/aurevoir/kaiding_mintpy/Process/mintpy/inputs/ifgramStack.h5 -t /media/aurevoir/kaiding_mintpy/Process/mintpy/smallbaselineApp.cfg -c /media/aurevoir/kaiding_mintpy/Process/mintpy/avgSpatialCoh.h5 --lookup /media/aurevoir/kaiding_mintpy/Process/mintpy/inputs/geometryRadar.h5
--------------------------------------------------
reading reference info from template: /media/aurevoir/kaiding_mintpy/Process/mintpy/smallbaselineApp.cfg
no input reference y/x.
REF_Y/X exists in input file, skip updating.
REF_Y: 532
REF_X: 222
reference point selection method: None
--------------------------------------------------

[AuRevoirzzz]

I have identified the source of the error; there was no reliable reference point with relatively high coherence within my area of interest. After setting mintpy.unwrapError.method = no, the software functioned properly. Thank you!