Auto-levels fail when NaN is in data

[pllim]

For image with NaNs in the data (e.g., use data file given in astropy/astropy#6485), auto-levels fail with the following traceback and defaults to (0, 0). The correct behavior should be to grab only valid data (e.g., using np.isfinite()) and calculate from that (EDIT: Although nanmax and nanmin should have caught that, so not sure why they don't seem to work properly here).

2017-08-28 14:03:02,654 | W | AutoCuts.py:173 (calc_histogram) | NaN's found in data, using workaround for histogram
2017-08-28 14:03:02,661 | W | nanfunctions.py:242 (nanmin) | .../ginga/AutoCuts.py:177: RuntimeWarning: All-NaN axis encountered
  minval = numpy.nanmin(data)

2017-08-28 14:03:02,662 | W | nanfunctions.py:350 (nanmax) | .../ginga/AutoCuts.py:178: RuntimeWarning: All-NaN slice encountered
  maxval = numpy.nanmax(data)

2017-08-28 14:03:02,662 | E | ImageView.py:799 (_image_set_cb) | Failed to initialize image: range parameter must be finite.
2017-08-28 14:03:02,711 | E | ImageView.py:804 (_image_set_cb) | Traceback:
  File ".../ginga/ImageView.py", line 796, in _image_set_cb
    self.apply_profile_or_settings(image)
  File ".../ginga/ImageView.py", line 884, in apply_profile_or_settings
    self.auto_levels()
  File ".../ginga/ImageView.py", line 2619, in auto_levels
    loval, hival = autocuts.calc_cut_levels(image)
  File ".../ginga/AutoCuts.py", line 153, in calc_cut_levels
    bnch = self.calc_histogram(data, pct=self.pct, numbins=self.numbins)
  File ".../ginga/AutoCuts.py", line 187, in calc_histogram
    density=False)
  File ".../numpy/lib/function_base.py", line 670, in histogram
    'range parameter must be finite.')

2017-08-28 14:03:15,068 | W | nanfunctions.py:350 (nanmax) | .../ginga/BaseImage.py:249: RuntimeWarning: All-NaN slice encountered
  self.maxval = np.nanmax(data)

2017-08-28 14:03:15,069 | W | nanfunctions.py:242 (nanmin) | .../ginga/BaseImage.py:250: RuntimeWarning: All-NaN axis encountered
  self.minval = np.nanmin(data)

[pllim]

Update: I think this has something to do with how Ginga is "slicing" the data. nanmin and nanmax seem to work fine by themselves:

>>> import numpy as np
>>> from astropy.io import fits
>>> dat = fits.getdata('NGC_628_NA_MOM2_THINGS.FITS')
>>> dat.shape
(1, 1, 1024, 1024)
>>> np.nanmin(dat), np.nanmax(dat)
(0.0, 40194.199)

[pllim]

I am stumped. I manually ran the logic block that deals with NaN in Histogram.calc_histogram() in AutoCuts.py using the dat I read in from above and I could not reproduce the error from Ginga.

Update: Although total_px is calculated wrong for the data shape of (1, 1, 1024, 1024) here but it should not affect the LOC that is reported in the traceback.

Update to the update: When the file is read into AstroImage, seems like data.shape is reduced into (1024, 1024), so in that case total_px is correct.

[ejeschke]

@pllim, I've run into these issues before and dealt with them in each kind of auto cut in AutoCuts. I take it you are using the "histogram" method?

[ejeschke]

I take it you are using the "histogram" method?

Do you get the same problem using other methods like "zscale", "stddev" or "minmax"?

[ejeschke]

This might be significant: "RuntimeWarning: All-NaN axis encountered". Is there really an axis that has all NaNs?

[ejeschke]

Is there really an axis that has all NaNs?

If there is a slice that is all NaNs, I'm not sure how it could calculate cuts--to ignore them all is to have nothing.

[pllim]

Is there really an axis that has all NaNs?

That's the weird thing. It is a 4-D data but not really a cube because two of the dimensions have size 1. Like I mentioned above, I couldn't find out why I even see that error when I ran the suspected logic block manually with the same data.

Perhaps you can download the data and see if you can re-produce it on your end?

[ejeschke]

@pllim, if you open Preferences and set the auto cuts method to "histogram" or "stddev" with this image you will see the usecrop option. Turn this to False and the images will display correctly.

The issue is that these two methods have an optimization to prevent long calculation times by using a crop of the image by default. I think with this particular image there are too many NaNs in the crop.

[ejeschke]

@pllim, I believe this is fixed in commit ac89ad3. Can you test it? Tnx.

[pllim]

Much better now. Thanks!