Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

System.Text.Json cannot parse int if it looks like a float 123.0 #40596

Closed
MingweiSamuel opened this issue Aug 9, 2020 · 10 comments
Closed

System.Text.Json cannot parse int if it looks like a float 123.0 #40596

MingweiSamuel opened this issue Aug 9, 2020 · 10 comments

Comments

@MingweiSamuel
Copy link

MingweiSamuel commented Aug 9, 2020

The following throws an exception:

System.Text.Json.JsonSerializer.Deserialize<int>("123.0");

Is there a way to prevent the exception since 123.0 is still an integer mathematically?

If there is not a solution already, maybe this should be a case in JsonNumberHandling #30255

I am using netcoreapp3.1


For additional context, I'm calling an API documented to return an int32 field, however the JSON response has 2808.0. It's always an integer but just contains a dot zero. Is the API violating their documentation? I don't know I just want to parse the JSON. So looks like I'll be using a custom JsonConverter<int>.
RiotGames/developer-relations#345
MingweiSamuel/Camille#39

public class IntConverter : System.Text.Json.Serialization.JsonConverter<int>
{
    public override int Read(ref System.Text.Json.Utf8JsonReader reader, Type type, System.Text.Json.JsonSerializerOptions options)
    {
        var valDouble = reader.GetDouble();
        var valInt = (int) valDouble;
        if (valDouble == valInt)
            return valInt;
        return reader.GetInt32();
    }

    public override void Write(System.Text.Json.Utf8JsonWriter writer, int value, System.Text.Json.JsonSerializerOptions options)
    {
        writer.WriteNumberValue(value);
    }
}

Test method Camille.RiotApi.Test.ChampionTest.FloatVsInt threw exception: 
    System.Text.Json.JsonException: The JSON value could not be converted to System.Int32. Path: $ | LineNumber: 0 | BytePositionInLine: 5. ---> System.FormatException: Either the JSON value is not in a supported format, or is out of bounds for an Int32.
  Stack Trace: 
    Utf8JsonReader.GetInt32()
    JsonConverterInt32.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options)
    JsonPropertyInfoNotNullable`4.OnRead(ReadStack& state, Utf8JsonReader& reader)
    JsonPropertyInfo.Read(JsonTokenType tokenType, ReadStack& state, Utf8JsonReader& reader)
    JsonSerializer.ReadCore(JsonSerializerOptions options, Utf8JsonReader& reader, ReadStack& readStack)
    --- End of inner exception stack trace ---
    ThrowHelper.ReThrowWithPath(ReadStack& readStack, Utf8JsonReader& reader, Exception ex)
    JsonSerializer.ReadCore(JsonSerializerOptions options, Utf8JsonReader& reader, ReadStack& readStack)
    JsonSerializer.ReadCore(Type returnType, JsonSerializerOptions options, Utf8JsonReader& reader)
    JsonSerializer.Deserialize(String json, Type returnType, JsonSerializerOptions options)
    JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options)
@Dotnet-GitSync-Bot Dotnet-GitSync-Bot added area-System.Text.Json untriaged New issue has not been triaged by the area owner labels Aug 9, 2020
@poizan42
Copy link
Contributor

Per the spec JSON does not distinguish between floats and integers, there are only numbers. So looks like this is a spec violation.

@svick
Copy link
Contributor

svick commented Aug 10, 2020

@poizan42 The JSON spec does not say anything about how to map JSON values to the .Net type system.

@poizan42
Copy link
Contributor

@svick nope, but 123.0 and 123 are the same number. Since it's the same value in JSON, whatever .NET does to map them it must give the same result for both. There simply isn't any provision that allows for a parser to handle them differently. If you want to map a json number to a .net Int32 then that mapping must act on the json number and not on its textual representation.

@tannergooding
Copy link
Member

but 123.0 and 123 are the same number

This is only true for a subset of numbers, not all numbers. For example, 9007199254740993.0 and 9007199254740993 are different values according to double as whole integrals above 2^53 start losing precision.

If we go strictly by the "de facto" JSON implementation, which is Javascript; there are not integrals only IEEE 754 binary64 (System.Double) and you cannot actually represent a number of integral values and others may round to Infinity.

@tannergooding
Copy link
Member

Additionally worth noting, the official JSON spec: http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf, details:

JSON is agnostic about the semantics of numbers. In any programming language, there can be a variety of
number types of various capacities and complements, fixed or floating, binary or decimal. That can make
interchange between different programming languages difficult. JSON instead offers only the representation of
numbers that humans use: a sequence of digits. All programming languages know how to make sense of digit
sequences even if they disagree on internal representations. That is enough to allow interchange.

It explicitly does not specify how the values should be interpreted and instead just gives a sequence of digits that is up to the individual language to interpret and expose as necessary.

@poizan42
Copy link
Contributor

poizan42 commented Aug 10, 2020

but 123.0 and 123 are the same number

This is only true for a subset of numbers, not all numbers. For example, 9007199254740993.0 and 9007199254740993 are different values according to double as whole integrals above 2^53 start losing precision.

No, they are both the same double value 1*2^53? But at least RFC8259 specifically allows for limited precision:

This specification allows implementations to set limits on the range
and precision of numbers accepted. Since software that implements
IEEE 754 binary64 (double precision) numbers [IEEE754] is generally
available and widely used, good interoperability can be achieved by
implementations that expect no more precision or range than these
provide, in the sense that implementations will approximate JSON
numbers within the expected precision.

ECMA-404 seems to allow for more freedom in how implementations interpret numbers than RFC8259, or at least more clearly states it, still I'm not seeing it justified anywhere that the mere existence of a decimal point can be used to decide which internal representation to use.

Interpretations of specifications aside, if the goal is to maximize interoperability then it might be better if internal representation was decided based on which datatype is requested or which one best holds the value rather than on the presence of a decimal point.

@Clockwork-Muse
Copy link
Contributor

.... okay, then System.Text.Json.JsonSerializer.Deserialize<int>("34359738367.0"); does what?

@poizan42
Copy link
Contributor

.... okay, then System.Text.Json.JsonSerializer.Deserialize<int>("34359738367.0"); does what?

Throws an exception indicating that the number is out of range.

@svick
Copy link
Contributor

svick commented Aug 10, 2020

@poizan42

I'm not seeing it justified anywhere that the mere existence of a decimal point can be used to decide which internal representation to use

I think it's the opposite: I'm not seeing it mandated anywhere that the existence of a decimal point can't be used to decide which representation to use.

Also, that interpretation wouldn't work well for decimal: currently JsonSerializer.Deserialize<decimal>("0.0") and JsonSerializer.Deserialize<decimal>("0") produce different values and I think that's the desired behavior.

@layomia layomia removed the untriaged New issue has not been triaged by the area owner label Aug 12, 2020
@layomia layomia added this to the Future milestone Aug 12, 2020
@eiriktsarpalis
Copy link
Member

I'd be inclined to close this as by-design behavior. Per @svick's original comment this is an issue of mapping JSON to .NET types rather than a spec conformance concern. Recommendation is to either update the model to something that permits floating point representations or register a custom converter for System.Int32 as suggested by the OP. cc @layomia

@dotnet dotnet locked as resolved and limited conversation to collaborators Nov 19, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

8 participants