Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement recommended fixes from OSTIF audit #313

Merged
merged 7 commits into from Apr 6, 2023
Merged

Implement recommended fixes from OSTIF audit #313

merged 7 commits into from Apr 6, 2023

Conversation

etrepum
Copy link
Member

@etrepum etrepum commented Apr 4, 2023 •
edited

Implement security hardening measures based on a source code audit of simplejson 3.18.4 by X41 D-Sec GmbH and sponsored by the OSTIF.

  • Fix invalid handling of unicode escape sequences in the pure Python
    implementation of the decoder (SJ-PT-23-01)
  • Fix missing reference count decrease if PyOS_string_to_double raises
    an exception in Python 2.x; was probably unreachable (SJ-PT-23-02)
  • Backport the integer string length limitation from Python 3.11 to
    limit quadratic number parsing (SJ-PT-23-03)
  • Fix inconsistencies with error messages between the C and Python
    implementations (SJ-PT-23-100)
  • Remove unused unichr import from encoder (SJ-PT-23-101)
  • Remove unused namedtuple_as_object and tuple_as_array arguments from
    simplejson.load (SJ-PT-23-102)
  • Remove vestigial _one_shot code from iterencode (SJ-PT-23-103)
  • Change default of allow_nan from True to False and add allow_nan
    to decoder (SJ-PT-23-107)

Several suggested improvements were not implemented in this release and will be considered in the future:

  • SJ-PT-23-104: Type Hints Not Used - Implementing type hints with annotations is not possible for as long as Python 2 is supported. Using stub files or revisiting this in the future when Python 2 support is removed will be considered in a later release.
  • SJ-PT-23-105: Deprecated Python Versions Supported - Without a way to get usage metrics, it's hard to say how many people are still using recent versions of simplejson and Python 2, so I would prefer to maintain support for a while longer.
  • SJ-PT-23-108: Support of Duplicate Key Names - I haven't had a lot of requests to provide this feature, and anyone looking to do this can implement it with object_pairs_hook. The default behavior of "last key wins" is consistent with JavaScript's JSON implementation.
  • SJ-PT-23-106: Unsigned Git Commits - PR merges were already verified since I update them with the GitHub UX but I will also start signing my commits with an SSH key. I have also enabled a tag protection rule for *. I did not enable a branch protection rule to require all commits to be signed since that would prevent accepting third party contributions without first rebasing myself.

The full public report is available here: https://www.x41-dsec.de/static/reports/X41-OSTIF-simplejson-CodeRview-2023-04-18.pdf
See also:

@etrepum
SJ-PT-23-01: Fix invalid handling of unicode escape sequences in Pyth…
7c2ccb7 
…on decoder
@etrepum
SJ-PT-23-02: Fix missing reference count decrease
1e495c1
@etrepum
SJ-PT-23-03: Backport integer string length limitation to limit quadr…
59dac4e 
…atic parsing
@etrepum
SJ-PT-23-100: Fix inconsistencies in error messages between C and Pyt…
dbd0aa3 
…hon implementations
@etrepum
SJ-PT-23-101: Remove unused unichr import from encoder
440a5e4
@etrepum
Additional security hardening improvements:
2cbc419 
* Remove unused namedtuple_as_object and tuple_as_array arguments from
  simplejson.load (SJ-PT-23-102)
* Remove vestigial _one_shot code from iterencode (SJ-PT-23-103)
* Change default of allow_nan from True to False and add allow_nan
  to decoder (SJ-PT-23-107)
@etrepum
Update CHANGES.txt
ec4a3d5
@etrepum etrepum enabled auto-merge April 6, 2023 16:53
@etrepum etrepum merged commit 1a4995d into master Apr 6, 2023
12 checks passed
This was referenced Apr 6, 2023
Merged
Merged
Merged
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Apr 7, 2023
py-simplejson: update to 3.19.1
e2fe0e1 
Version 3.19.1 released 2023-04-06

* This release contains security hardening measures based on recommendations
  by a security audit sponsored by OSTIF and conducted by X41 D-Sec GmbH.
  Several of these measures include changing defaults to be more strict,
  by default simplejson will now only consume and produce compliant JSON,
  but the flags still exist for any backwards compatibility needs.
  No high priority issues were discovered, the reference count
  leak is thought to be unreachable since the digits of the float are
  checked before PyOS_string_to_double is called.
  A link to the public version of this report will be included in a
  future release of simplejson. The following fixes were implemented in
  one PR: simplejson/simplejson#313
* Fix invalid handling of unicode escape sequences in the pure Python
  implementation of the decoder (SJ-PT-23-01)
* Fix missing reference count decrease if PyOS_string_to_double raises
  an exception in Python 2.x; was probably unreachable (SJ-PT-23-02)
* Backport the integer string length limitation from Python 3.11 to
  limit quadratic number parsing (SJ-PT-23-03)
* Fix inconsistencies with error messages between the C and Python
  implementations (SJ-PT-23-100)
* Remove unused unichr import from encoder (SJ-PT-23-101)
* Remove unused namedtuple_as_object and tuple_as_array arguments from
  simplejson.load (SJ-PT-23-102)
* Remove vestigial _one_shot code from iterencode (SJ-PT-23-103)
* Change default of allow_nan from True to False and add allow_nan
  to decoder (SJ-PT-23-107)
This was referenced Apr 8, 2023
Merged
Open
kraj pushed a commit to YoeDistro/meta-openembedded that referenced this pull request Apr 10, 2023
@wangmingyu84 @kraj
python3-simplejson: upgrade 3.18.4 -> 3.19.1
3c49711 
Changelog:
============
* This release contains security hardening measures based on recommendations
  by a security audit sponsored by OSTIF and conducted by X41 D-Sec GmbH.
  Several of these measures include changing defaults to be more strict,
  by default simplejson will now only consume and produce compliant JSON,
  but the flags still exist for any backwards compatibility needs.
  No high priority issues were discovered, the reference count
  leak is thought to be unreachable since the digits of the float are
  checked before PyOS_string_to_double is called.
  A link to the public version of this report will be included in a
  future release of simplejson. The following fixes were implemented in
  one PR: simplejson/simplejson#313
* Fix invalid handling of unicode escape sequences in the pure Python
  implementation of the decoder (SJ-PT-23-01)
* Fix missing reference count decrease if PyOS_string_to_double raises
  an exception in Python 2.x; was probably unreachable (SJ-PT-23-02)
* Backport the integer string length limitation from Python 3.11 to
  limit quadratic number parsing (SJ-PT-23-03)
* Fix inconsistencies with error messages between the C and Python
  implementations (SJ-PT-23-100)
* Remove unused unichr import from encoder (SJ-PT-23-101)
* Remove unused namedtuple_as_object and tuple_as_array arguments from
  simplejson.load (SJ-PT-23-102)
* Remove vestigial _one_shot code from iterencode (SJ-PT-23-103)
* Change default of allow_nan from True to False and add allow_nan
  to decoder (SJ-PT-23-107)

Signed-off-by: Wang Mingyu <wangmy@fujitsu.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
This was referenced May 31, 2023
Open
Merged
Merged
@renovate renovate bot mentioned this pull request Jul 8, 2023
Merged
1 task
@winterbot winterbot mentioned this pull request Aug 14, 2023
Merged
1 task
@renovate renovate bot mentioned this pull request Aug 31, 2023
Merged
1 task
@renovate renovate bot mentioned this pull request Oct 23, 2023
Closed
1 task
daregit pushed a commit to daregit/yocto-combined that referenced this pull request May 22, 2024
@wangmingyu84 @kraj
src/meta-openembedded:python3-simplejson: upgrade 3.18.4 -> 3.19.1
7ee58e4 
Changelog:
============
* This release contains security hardening measures based on recommendations
  by a security audit sponsored by OSTIF and conducted by X41 D-Sec GmbH.
  Several of these measures include changing defaults to be more strict,
  by default simplejson will now only consume and produce compliant JSON,
  but the flags still exist for any backwards compatibility needs.
  No high priority issues were discovered, the reference count
  leak is thought to be unreachable since the digits of the float are
  checked before PyOS_string_to_double is called.
  A link to the public version of this report will be included in a
  future release of simplejson. The following fixes were implemented in
  one PR: simplejson/simplejson#313
* Fix invalid handling of unicode escape sequences in the pure Python
  implementation of the decoder (SJ-PT-23-01)
* Fix missing reference count decrease if PyOS_string_to_double raises
  an exception in Python 2.x; was probably unreachable (SJ-PT-23-02)
* Backport the integer string length limitation from Python 3.11 to
  limit quadratic number parsing (SJ-PT-23-03)
* Fix inconsistencies with error messages between the C and Python
  implementations (SJ-PT-23-100)
* Remove unused unichr import from encoder (SJ-PT-23-101)
* Remove unused namedtuple_as_object and tuple_as_array arguments from
  simplejson.load (SJ-PT-23-102)
* Remove vestigial _one_shot code from iterencode (SJ-PT-23-103)
* Change default of allow_nan from True to False and add allow_nan
  to decoder (SJ-PT-23-107)

Signed-off-by: Wang Mingyu <wangmy@fujitsu.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
1 participant
@etrepum