DOC: docstring for numpy.finfo.eps is incorrect

[mdickinson]

The docstring for numpy.finfo currently defines the eps attribute as:

The smallest representable positive number such that 1.0 + eps != 1.0. [...]

That definition is not correct, at least in the common case of the IEEE 754 binary formats. For example, with the IEEE 754 binary64 format under the usual round-ties-to-even rounding mode, the stated definition gives a value of 2**-53 + 2**-105 (approx. 1.1102230246251568e-16), which is a little over half of the correct value of 2**-52 (approx 2.220446049250313e-16).

In [36]: eps = 2**-53 + 2**-105

In [37]: eps
Out[37]: 1.1102230246251568e-16

In [38]: 1.0 + eps != 1.0
Out[38]: True

In [39]: 1.0 + np.nextafter(eps, -np.inf) != 1.0
Out[39]: False

In [40]: np.finfo(float).eps
Out[40]: 2.2204460492503131e-16

Some possible rewordings:

The smallest positive number such that 1.0 + eps is representable.

Or:

The difference between 1.0 and the smallest representable float larger than 1.0.

Or it could be defined in terms of nextafter as:

The value of np.nextafter(1.0, np.inf) - 1.0.

For comparison, section 5.2.4.2.2, paragraph 11 of the C99 standard defines the various *_EPSILON macros (DBL_EPSILON, FLT_EPSILON, ...) as:

the difference between 1 and the least value greater than 1 that is representable in the given floating point type, b^{1− p}

The docstring for epsneg is similarly incorrect.

[mdickinson]

Ah, looking at the source, I see that eps is actually computed as the smallest power of 2 such that 1 + eps != eps. That's not the same thing as the difference between 1.0 and the next representable float up from 1.0, though the two things do happen to coincide for IEEE 754 and the round-ties-to-even rounding mode. So my suggested rewordings above are invalid. Perhaps something like:

The smallest power of 2 such that 1.0 + eps is representable.

would be better? Technically, that 2 should be replaced with the floating-point radix, but that's probably unnecessarily pedantic. Does NumPy currently have any support for radix 10 or radix 16 floats?

[As an aside, I much prefer the C99-style definition of epsilon, since it involves only the format and doesn't depend on the semantics of floating-point addition. In particular, the C99 definition doesn't depend on the rounding mode currently in effect, while the 1 + eps != 1 definition does.]

[joferkington]

A pointer to np.spacing and np.nextafter in the "See Also" section would also be potentially useful.

[gwhammett]

Here we are almost 4 years later and this hasn't been fixed. I don't think that #14618 will be sufficient to close this issue.

To clarify the bug report, the current definition of eps in the docstring is off by about a factor of 2 from the value of eps returned by np.finfo. (The numbers I give below are for implementations with IEEE-754 64 bit floating point standard.) I.e.,

np.finfo(1.0).eps = 2**-52 = 2.220446049250313e-16

However, the docstring defines eps as

"The smallest representable positive number such that 1.0 + eps != 1.0."

The actual value of eps that would satisfy the definition 1.0+eps_min != 1.0 is:

eps_min = 2**-53+2**-105 = 1.1102230246251568e-16

which is almost 1/2 of the present value of np.finfo(1.0).eps. We probably don't want to change the value of np.finfo(1.0).eps, to keep backwards compatibility, but the docstring definition of eps needs to be corrected to match what is calculated. The source code for np.finfo(1.0)eps uses numpy.MachAr, and looking at the documentation there, which allows other bases besides 2, I think the best definition of of eps is one of the early options that @mdickinson previously suggested. I would also add an example value to the description:

"The difference between 1.0 and the smallest representable float larger than 1.0. (For 64 bit binary floats in the IEEE-754 standard, eps = 2**-52 ≅ 2.22e-16.)"

The docstring for epsneg is similarly incorrect.

[c-f-h]

I also just stumbled over this inaccuracy. This ought to be fixed.

For reference, Matlab documents its eps (which has the same value) as: "the distance from 1.0 to the next larger double-precision number, that is, 2^-52."

[rossbar]

The current definition is incorrect and confusing due to rounding issues. @gwhammett the proposed definition is both correct and more descriptive and encourage a PR to close this issue.