Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix UnderflowError in _try_rescale_float when maxVal == 0. #2881

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

nup002
Copy link

@nup002 nup002 commented Nov 14, 2023

In ImageItem.py, an UnderflowError exception is thrown in _try_rescale_float by numpy.nextafter() if the histogram levels minVal and maxVal are both 0. This PR fixes the problem by instead setting maxVal to the machine precision if maxVal is 0.

Tested and verified to resolve the issue without introducing new ones.

Fix UnderflowError in _try_rescale_float when maxVal == 0.
@@ -640,7 +640,7 @@ def _try_rescale_float(self, image, levels, lut):

minVal, maxVal = levels
if minVal == maxVal:
maxVal = xp.nextafter(maxVal, 2*maxVal)
maxVal = xp.nextafter(maxVal, 2*maxVal) if maxVal != 0 else numpy.finfo(maxVal).eps
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a dumb question:
Is it actually intentional that the existing code moves maxval to the next more negative number when maxval is already < 0? That makes rng negative below, and that seems conceptually a little strange.

If the intention is to clamp rng to a small positive number, then wouldn't
maxVal = math.nextafter( minVal, math.inf )
be what we want? (under all variations of maxVal <0 / ==0 / > 0)

Additional comment:
if minVal and maxVal are just simple numbers, then there's probably no need to potentially invoke the cupy version of nextafter here. I suspect that nextafter from the math library should be enough.

@@ -640,7 +640,7 @@ def _try_rescale_float(self, image, levels, lut):

minVal, maxVal = levels
if minVal == maxVal:
maxVal = xp.nextafter(maxVal, 2*maxVal)
maxVal = xp.nextafter(maxVal, 2*maxVal) if maxVal != 0 else numpy.finfo(maxVal).eps
rng = maxVal - minVal
rng = 1 if rng == 0 else rng
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This (pre-existing) line seems redundant if the code above works as intended. If it is indeed still needed, then there's a kind of problem that gets past the check minVal == maxVal... Possibly something originating from mixed or non-float datatypes?

Is the real thing we want to avoid float(maxval - minval) == 0 ? Then the most robust path might be to directly check that. Something like

rng = float(maxVal - minVal)
if rng == 0.:
   maxVal = math.nextafter( minVal, math.inf )
   rng = float(maxVal - minVal)

maybe?

@pijyoi
Copy link
Contributor

pijyoi commented Nov 14, 2023

The code chunk actually comes from functions::makeARGB():

if isinstance(levels, xp.ndarray) and levels.ndim == 2:
# we are going to rescale each channel independently
if levels.shape[0] != data.shape[-1]:
raise Exception("When rescaling multi-channel data, there must be the same number of levels as channels (data.shape[-1] == levels.shape[0])")
newData = xp.empty(data.shape, dtype=int)
for i in range(data.shape[-1]):
minVal, maxVal = levels[i]
if minVal == maxVal:
maxVal = xp.nextafter(maxVal, 2*maxVal)
rng = maxVal-minVal
rng = 1 if rng == 0 else rng
newData[...,i] = rescaleData(data[...,i], scale / rng, minVal, dtype=dtype)
data = newData
else:
# Apply level scaling unless it would have no effect on the data
minVal, maxVal = levels
if minVal != 0 or maxVal != scale:
if minVal == maxVal:
maxVal = xp.nextafter(maxVal, 2*maxVal)
rng = maxVal-minVal
rng = 1 if rng == 0 else rng
data = rescaleData(data, scale/rng, minVal, dtype=dtype)

Back then, the goal was to make the code behave the same as makeARGB().

I would replace it with the following:

minVal, maxVal = levels
# if minVal == maxVal:
#     maxVal = xp.nextafter(maxVal, 2*maxVal)
rng = maxVal - minVal
rng = 1 if rng == 0 else rng

The goal is simply to avoid a division by zero shortly after.

This simple pattern can be found in:

  1. # normalize and quantize
    mn, mx = self.getLevels()
    rng = mx - mn
    if rng == 0:
    rng = 1
    scale = len(lut) / rng
    Z = fn.rescaleData(Z, scale, mn, dtype=int, clip=(0, len(lut)-1))
  2. rng = z_max - z_min
    if rng == 0:
    rng = 1
    norm = fn.rescaleData(self.z, scale / rng, z_min,
    dtype=int, clip=(0, len(lut)-1))

@NilsNemitz
Copy link
Contributor

Well, that looks like a dramatic reduction in complexity!

And unless I am missing some side effect in the rescale functions, the exact value of the combined scaling factor scale/rng should not matter when the output range is zero - or it should at least be very obvious if it does.

@j9ac9k
Copy link
Member

j9ac9k commented Mar 7, 2024

I belive the issue this PR was meant to address is now addressed in #2944 ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants