Skip to content

Commit

Permalink
Additional security hardening improvements:
Browse files Browse the repository at this point in the history 
* Remove unused namedtuple_as_object and tuple_as_array arguments from
  simplejson.load (SJ-PT-23-102)
* Remove vestigial _one_shot code from iterencode (SJ-PT-23-103)
* Change default of allow_nan from True to False and add allow_nan
  to decoder (SJ-PT-23-107)
  • Loading branch information
@etrepum
etrepum committed Apr 6, 2023
1 parent 440a5e4 commit 2cbc419
Show file tree
Hide file tree
Showing 8 changed files with 152 additions and 121 deletions.
5 changes: 5 additions & 0 deletions CHANGES.txt
View file
Expand Up @@ -9,6 +9,11 @@ Version 3.19.0 released 2023-04-XX
* Fix inconsistencies with error messages between the C and Python
implementations (SJ-PT-23-100)
* Remove unused unichr import from encoder (SJ-PT-23-101)
* Remove unused namedtuple_as_object and tuple_as_array arguments from
simplejson.load (SJ-PT-23-102)
* Remove vestigial _one_shot code from iterencode (SJ-PT-23-103)
* Change default of allow_nan from True to False and add allow_nan
to decoder (SJ-PT-23-107)

Version 3.18.4 released 2023-03-14

Expand Down
91 changes: 64 additions & 27 deletions index.rst
View file
Expand Up @@ -160,7 +160,7 @@ Basic Usage
-----------

.. function:: dump(obj, fp, skipkeys=False, ensure_ascii=True, \
check_circular=True, allow_nan=True, cls=None, \
check_circular=True, allow_nan=False, cls=None, \
indent=None, separators=None, encoding='utf-8', \
default=None, use_decimal=True, \
namedtuple_as_object=True, tuple_as_array=True, \
Expand Down Expand Up @@ -191,7 +191,7 @@ Basic Usage
is highly optimized.

.. function:: dumps(obj, skipkeys=False, ensure_ascii=True, \
check_circular=True, allow_nan=True, cls=None, \
check_circular=True, allow_nan=False, cls=None, \
indent=None, separators=None, encoding='utf-8', \
default=None, use_decimal=True, \
namedtuple_as_object=True, tuple_as_array=True, \
Expand Down Expand Up @@ -225,13 +225,17 @@ Basic Usage
reference check for container types will be skipped and a circular
reference will result in an :exc:`OverflowError` (or worse).

If *allow_nan* is false (default: ``True``), then it will be a
If *allow_nan* is false (default: ``False``), then it will be a
:exc:`ValueError` to serialize out of range :class:`float` values
(``nan``, ``inf``, ``-inf``) in strict compliance of the original
JSON specification. If *allow_nan* is true, their JavaScript equivalents
will be used (``NaN``, ``Infinity``, ``-Infinity``). See also *ignore_nan*
for ECMA-262 compliant behavior.

.. versionchanged:: 3.19.0
The default for *allow_nan* was changed to False for better spec
compliance.

If *indent* is a string, then JSON array elements and object members
will be pretty-printed with a newline followed by that string repeated
for each level of nesting. ``None`` (the default) selects the most compact
Expand Down Expand Up @@ -324,7 +328,7 @@ Basic Usage
.. function:: load(fp, encoding='utf-8', cls=None, object_hook=None, \
parse_float=None, parse_int=None, \
parse_constant=None, object_pairs_hook=None, \
use_decimal=None, **kw)
use_decimal=None, allow_nan=False, **kw)
Deserialize *fp* (a ``.read()``-supporting file-like object containing a JSON
document) to a Python object using this
Expand Down Expand Up @@ -367,7 +371,7 @@ Basic Usage
.. function:: loads(s, encoding='utf-8', cls=None, object_hook=None, \
parse_float=None, parse_int=None, \
parse_constant=None, object_pairs_hook=None, \
use_decimal=None, **kw)
use_decimal=None, allow_nan=False, **kw)
Deserialize *s* (a :class:`str` or :class:`unicode` instance containing a JSON
document) to a Python object. :exc:`JSONDecodeError` will be
Expand Down Expand Up @@ -419,10 +423,6 @@ Basic Usage
suitable alternative parser is specified
(e.g. :class:`decimal.Decimal`)

*parse_constant*, if specified, will be called with one of the following
strings: ``'-Infinity'``, ``'Infinity'``, ``'NaN'``. This can be used to
raise an exception if invalid JSON numbers are encountered.

If *use_decimal* is true (default: ``False``) then *parse_float* is set to
:class:`decimal.Decimal`. This is a convenience for parity with the
:func:`dump` parameter.
Expand All @@ -443,12 +443,28 @@ Basic Usage
Subclassing is not recommended. You should use *object_hook* or
*object_pairs_hook*. This is faster and more portable than subclassing.


*allow_nan*, if True (default false), will allow the parser to
accept the non-standard floats
``NaN``, ``Infinity``, and ``-Infinity``.

.. versionchanged:: 3.19.0

This argument was added to make it possible to use the legacy behavior
now that the parser is more strict about compliance to the standard.

*parse_constant*, if specified, will be
called with one of the following strings: ``'-Infinity'``,
``'Infinity'``, ``'NaN'``. It is not recommended to use this feature,
as it is rare to parse non-compliant JSON containing these values.


Encoders and decoders
---------------------

.. class:: JSONDecoder(encoding='utf-8', object_hook=None, parse_float=None, \
parse_int=None, parse_constant=None, \
object_pairs_hook=None, strict=True)
object_pairs_hook=None, strict=True, allow_nan=False)

Simple JSON decoder.

Expand Down Expand Up @@ -476,7 +492,8 @@ Encoders and decoders
| null | None | None |
+---------------+-----------+-----------+

It also understands ``NaN``, ``Infinity``, and ``-Infinity`` as their
When *allow_nan* is True, it also understands
``NaN``, ``Infinity``, and ``-Infinity`` as their
corresponding ``float`` values, which is outside the JSON spec.

*encoding* determines the encoding used to interpret any :class:`str` objects
Expand Down Expand Up @@ -516,15 +533,24 @@ Encoders and decoders
suitable alternative parser is specified
(e.g. :class:`decimal.Decimal`)

*parse_constant*, if specified, will be called with one of the following
strings: ``'-Infinity'``, ``'Infinity'``, ``'NaN'``. This can be used to
raise an exception if invalid JSON numbers are encountered.
*parse_constant*, if specified, will be
called with one of the following strings: ``'-Infinity'``,
``'Infinity'``, ``'NaN'``. It is not recommended to use this feature,
as it is rare to parse non-compliant JSON containing these values.

*strict* controls the parser's behavior when it encounters an invalid
control character in a string. The default setting of ``True`` means that
unescaped control characters are parse errors, if ``False`` then control
characters will be allowed in strings.

*allow_nan* when True (not the default), the decoder will allow
``NaN``, ``Infinity``, and ``-Infinity`` as their corresponding floats.

.. versionchanged:: 3.19.0
This argument was added to make it behave closer to the spec by
default. The previous behavior can be restored by setting this to
False.

.. method:: decode(s)

Return the Python representation of the JSON document *s*. See
Expand All @@ -546,7 +572,7 @@ Encoders and decoders
document is not valid.

.. class:: JSONEncoder(skipkeys=False, ensure_ascii=True, \
check_circular=True, allow_nan=True, sort_keys=False, \
check_circular=True, allow_nan=False, sort_keys=False, \
indent=None, separators=None, encoding='utf-8', \
default=None, use_decimal=True, \
namedtuple_as_object=True, tuple_as_array=True, \
Expand Down Expand Up @@ -587,7 +613,8 @@ Encoders and decoders
wrapped in another type with an appropriate `for_json` method to
transform the keys during encoding.

It also understands ``NaN``, ``Infinity``, and ``-Infinity`` as their
When *allow_nan* is True, it also understands
``NaN``, ``Infinity``, and ``-Infinity`` as their
corresponding ``float`` values, which is outside the JSON spec.

To extend this to recognize other objects, subclass and implement a
Expand All @@ -613,12 +640,16 @@ Encoders and decoders
prevent an infinite recursion (which would cause an :exc:`OverflowError`).
Otherwise, no such check takes place.

If *allow_nan* is true (the default), then ``NaN``, ``Infinity``, and
If *allow_nan* is true (not the default), then ``NaN``, ``Infinity``, and
``-Infinity`` will be encoded as such. This behavior is not JSON
specification compliant, but is consistent with most JavaScript based
encoders and decoders. Otherwise, it will be a :exc:`ValueError` to encode
specification compliant. Otherwise, it will be a :exc:`ValueError` to encode
such floats. See also *ignore_nan* for ECMA-262 compliant behavior.

.. versionchanged:: 3.19.0
This default is now False to make it behave closer to the spec.
The previous behavior can be restored by setting this to
False.

If *sort_keys* is true (not the default), then the output of dictionaries
will be sorted by key; this is useful for regression tests to ensure that
JSON serializations can be compared on a day-to-day basis.
Expand Down Expand Up @@ -730,7 +761,7 @@ Encoders and decoders
:meth:`iterencode`.

.. class:: JSONEncoderForHTML(skipkeys=False, ensure_ascii=True, \
check_circular=True, allow_nan=True, \
check_circular=True, allow_nan=False, \
sort_keys=False, indent=None, separators=None, \
encoding='utf-8', \
default=None, use_decimal=True, \
Expand Down Expand Up @@ -840,22 +871,28 @@ Infinite and NaN Number Values

The RFC does not permit the representation of infinite or NaN number values.
Despite that, by default, this module accepts and outputs ``Infinity``,
``-Infinity``, and ``NaN`` as if they were valid JSON number literal values::
``-Infinity``, and ``NaN`` as if they were valid JSON number literal values
if the allow_nan flag is enabled::

>>> # Neither of these calls raises an exception, but the results are not valid JSON
>>> json.dumps(float('-inf'))
>>> json.dumps(float('-inf'), allow_nan=True)
'-Infinity'
>>> json.dumps(float('nan'))
>>> json.dumps(float('nan'), allow_nan=True)
'NaN'
>>> # Same when deserializing
>>> json.loads('-Infinity')
>>> json.loads('-Infinity', allow_nan=True)
-inf
>>> json.loads('NaN')
>>> json.loads('NaN', allow_nan=True)
nan
>>> # ignore_nan uses the ECMA-262 behavior to serialize these as null
>>> json.dumps(float('-inf'), ignore_nan=True)
'null'
>>> json.dumps(float('nan'), ignore_nan=True)
'null'

In the serializer, the *allow_nan* parameter can be used to alter this
behavior. In the deserializer, the *parse_constant* parameter can be used to
alter this behavior.
behavior. In the deserializer, the *allow_nan* and
*parse_constant* parameters can be used to alter this behavior.


Repeated Names Within an Object
Expand Down

0 comments on commit 2cbc419

Please sign in to comment.