Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make double parsing locale independent (and faster) #1993

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

lkeegan
Copy link
Member

@lkeegan lkeegan commented Nov 24, 2023

  • use std::from_chars instead of strtod to parse doubles if available (c++17 required)
    • otherwise falls back to previous strtod implementation
    • std::from_chars is locale independent and faster than strtod
  • add more double parsing tests
  • add double parsing google benchmarks
  • resolves parsing of doubles depends on locale #1566

@rikardn
Copy link
Contributor

rikardn commented Nov 24, 2023

std::from_chars has the potential of being the fastest possible parsing function. If I remember correctly the implementation of this in libstdc++ was very delayed or even never implemented. I remember using a third party implementation once. Has it been implemented now in both gcc and clang?

@lkeegan
Copy link
Member Author

lkeegan commented Nov 24, 2023

@lkeegan
Copy link
Member Author

lkeegan commented Nov 24, 2023

std::from_chars has the potential of being the fastest possible parsing function. If I remember correctly the implementation of this in libstdc++ was very delayed or even never implemented. I remember using a third party implementation once. Has it been implemented now in both gcc and clang?

According to https://github.com/fastfloat/fast_float their implementation is now part of gcc (12) and llvm (not sure exactly what version)

@rikardn
Copy link
Contributor

rikardn commented Nov 24, 2023

Ok, nice. That was the implementation I was using. I think clang 15 supports it.

Copy link
Contributor

@certik certik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thanks!

@certik certik requested a review from isuruf November 24, 2023 15:52
- use `std::from_chars` instead of `strtod` to parse doubles if available (c++17 required)
  - otherwise falls back to previous `strtod` implementation
  - `std::from_chars` is locale independent and faster than `strtod`
- add more double parsing tests
- add double parsing google benchmarks
- resolves symengine#1566
…g comes from parser and will always be parsable to a double
@lkeegan lkeegan force-pushed the fix_1566_locale_independent_parsing branch from 89ee5c8 to 9c09875 Compare February 20, 2024 08:39
… to incomplete standard library implementations
@lkeegan
Copy link
Member Author

lkeegan commented Feb 20, 2024

So it seems that just checking for c++17 at compile time was not sufficient as standard library support for from_chars (including parsing floating point values) is incomplete for:

  • libstdc++ < 11
  • macOS < 13.3
  • libc++ < 14(?)

I've switched to just directly bundling and using the fast_float library instead (which is the underlying fast implementation of std::from_chars - thanks @rikardn!) and is available as a single header file.

@lkeegan
Copy link
Member Author

lkeegan commented Apr 10, 2024

@isuruf would you have time to take a look at this at some point? It would be nice to resolve #1566

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

parsing of doubles depends on locale
3 participants