Remove bitcount field of the mpf tuple?

[skirpichev]

Our docs says:

More technically, mpmath represents a floating-point number as a 4-tuple (sign, man, exp, bc) where sign is 0 or 1 (indicating positive vs negative) and the mantissa is nonnegative; bc (bitcount) is the size of the absolute value of the mantissa as measured in bits. Though redundant, keeping a separate sign field and explicitly keeping track of the bitcount significantly speeds up arithmetic (the bitcount, especially, is frequently needed but slow to compute from scratch due to the lack of a Python built-in function for the purpose).

Since CPython 3.1 there is a such function. It's the int.bit_length() method.

Some timings:

$ python -m timeit -r11 -s 'from mpmath.libmp.libintmath import python_bitcount as bc' 'bc(1000)'
500000 loops, best of 11: 965 nsec per loop
$ python -m timeit -r11 -s 'from mpmath.libmp.libintmath import python_bitcount as bc' 'bc(10)'
500000 loops, best of 11: 930 nsec per loop
$ python -m timeit -r11 -s 'from mpmath.libmp.libintmath import python_bitcount as bc' \
  -s 'from math import factorial;n=factorial(100)' 'bc(n)'
100000 loops, best of 11: 3.18 usec per loop

vs

$ python -m timeit -r11 '(1000).bit_length()'
5000000 loops, best of 11: 75.6 nsec per loop
$ python -m timeit -r11 '(10).bit_length()'
5000000 loops, best of 11: 74.9 nsec per loop
$ python -m timeit -r11 -s 'from math import factorial;n=factorial(100)' 'n.bit_length()'
5000000 loops, best of 11: 97.2 nsec per loop

So, using the bit_length() is clearly better and it's available for all backends. Does it make sense to keep now the bc field of the mpf tuple or it should be deprecated?

[oscarbenjamin]

The tricky thing is that the mpf tuple is used as a conversion format in many places so there would still be a need to generate the 4-tuple with bit count often.

Otherwise though I agree that a 3 tuple is better and it can save a lot of wasted arithmetic (and bugs) trying to keep the bit count in sync.

[skirpichev]

The tricky thing is that the mpf tuple is used as a conversion format in many places so there would still be a need to generate the 4-tuple with bit count often.

The proposed change is obviously not a backward compatible, so we will accept 4-tuples for a while, e.g. in _mpf_ properties.

Otherwise though I agree that a 3 tuple is better and it can save a lot of wasted arithmetic (and bugs)

But maybe we miss some scenarios where "caching" the bitcount has some benefits? (Same for the sign bit, BTW.)

[skirpichev]

BTW, simpler format (2-tuple) was planned in #178. So, I guess Fredrik is ok with this idea.

[skirpichev]

Currently, we use bc for special numbers (nan, inf, etc), but this seems redundant as well.

[skirpichev]

It seems, some mpmath users (well, the SymPy of course!) actually do tuple unpacking of mpf's. I think we should deprecate this in the next release. from_man_exp/to_man_exp() functions should be used instead.

[skirpichev]

Ok. The plan is to deprecate direct manipulations with _mpf_. People should use from_man_exp()/to_man_exp() functions for finite numbers and dedicated imports for special forms like fnan, fnzero, finf and fninf. Then, after a release, we could change internal format of the _mpf_.

Unfortunately, we can't emit a warning at runtime.

I'll make a patch for the Diofant to show how it's possible. It seems, very few places are affected. I think it's valid to the SymPy as well.

Edit: diofant/diofant#1389 BTW, maybe we should change to_man_exp() to output a signed mantissa (like the mpfr.as_mantissa_exp()). That's an incompatible change, than might be proceed via optional kwarg (e.g. named as signed, which currently will be set to False).

[skirpichev]

Ok, I guess, there are no fans of the bc field. That's missing here is a warning in release notes for v1.4, that direct access to mpf tuple is deprecated.