Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

neither ignore_nan=True nor allow_nan=False handle Decimals correctly #149

Open
garborg opened this issue Dec 8, 2016 · 6 comments
Open

Comments

@garborg
Copy link

garborg commented Dec 8, 2016

import math
from decimal import Decimal
import numpy as np
import simplejson

fmath = [1/3, math.nan, 0.2]
fnp = np.array([1/3, np.NaN, 0.2])
dmath = [Decimal(v).quantize(Decimal('0.01')) for v in fmath]
dnp= [Decimal(v).quantize(Decimal('0.01')) for v in fnp]

print(simplejson.dumps(fmath, ignore_nan=True))
print(simplejson.dumps(fnp.tolist(), ignore_nan=True))
print(simplejson.dumps(dmath, ignore_nan=True))
print(simplejson.dumps(dnp, ignore_nan=True))
[0.3333333333333333, null, 0.2]
[0.3333333333333333, null, 0.2]
[0.33, NaN, 0.20]
[0.33, NaN, 0.20]
@etrepum
Copy link
Member

etrepum commented Dec 8, 2016

If this issue is important to you, the best way to expedite it is to submit a PR with tests

@etrepum
Copy link
Member

etrepum commented Dec 8, 2016

The text here is definitely a great start for writing some tests, btw

@garborg garborg changed the title ignore_nan doesn't work on Decimals neither ignore_nan=True nor allow_nan=False handle Decimals correctly Dec 8, 2016
@garborg
Copy link
Author

garborg commented Dec 8, 2016

Do you think that'd mean digging in to the C code? If not, I'll try to circle back to it when I'm not under the gun.

A second inconsistency (change the issue title to reflect it):

simplejson.dumps(decimal.Decimal('NaN'), allow_nan=False) # NaN
simplejson.dumps(np.NaN, allow_nan=False) # exception

@etrepum
Copy link
Member

etrepum commented Dec 9, 2016

I don't recall much C code that deals with decimal, but without writing the tests and Python implementation or auditing the C implementation I couldn't say for sure whether or not any digging there is required. A PR with failing tests (or half-failing when C extensions are enabled) is going to speed up the process if this is something you need.

@garborg
Copy link
Author

garborg commented Jan 6, 2017

This isn't in our workflow anymore, so while I think it'd be good to fix, it's off my radar for now.

@etrepum
Copy link
Member

etrepum commented Jan 6, 2017

To be honest when this feature originally went in, I had no idea the Decimal even had a representation for special values since I had never seen one in the wild, so I didn't think to look into it.

Maybe the easiest thing to do would be to add something like d if d.is_normal() else float(d) in that code path, were anyone to want to look into this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants