Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

#425: Improve performance of INI/BC serialization #426

Open
wants to merge 80 commits into
base: main
Choose a base branch
from
Open

#425: Improve performance of INI/BC serialization #426

wants to merge 80 commits into from

Conversation

priscavdsluis
Copy link
Contributor

No description provided.

priscavdsluis and others added 30 commits November 1, 2022 16:33
@priscavdsluis
#308: Change data block default spacing from 4 to 2
2fe68d9
@priscavdsluis
#308: Make initial setup for custom serialization configuration for u…
694f4a5 
…sers.
@priscavdsluis
#308: Remove serialize override and use the INIModel one (parent)
eda3896
@priscavdsluis
autoformat: isort & black
6ee3f03
@priscavdsluis
#308: Add _exclude_fields to ParsableFileModel
66c842c
@priscavdsluis
autoformat: isort & black
015280d
@priscavdsluis
#308: Add util method to convert a float to a string with n decimals.
0af0873
@priscavdsluis
autoformat: isort & black
a6ac951
@priscavdsluis
#308: Setup for configuring the floating values in a datablock (e,g. …
686b487 
…bc file)
@priscavdsluis
autoformat: isort & black
4116f86
@priscavdsluis
Merge branch 'main' into feat/308-improve-c-file-formatting
96754b5
@priscavdsluis
Merge branch 'main' into feat/308-improve-c-file-formatting
ab35528
@priscavdsluis
#308: Rename SerializerConfig (in INI module) to INISerializerConfig
6b36673
@priscavdsluis
autoformat: isort & black
36dcb30
@priscavdsluis
#308: Add a BCSerializerConfig
5494b54
@priscavdsluis
autoformat: isort & black
aee953d
@priscavdsluis
Revert "#308: Change data block default spacing from 4 to 2"
20d7627 
This reverts commit 2fe68d9.
@priscavdsluis
#308: Rename and move BCSerializerConfig
165fc64
@priscavdsluis
autoformat: isort & black
385064b
@priscavdsluis
#308: Update argument type
8170fad
@priscavdsluis
#308: Make sure that float precision for INI models is adjustable
0c85c5e
@veenstrajelmer
Merge branch 'main' into feat/308-improve-bc-file-formatting
9d5620d
@priscavdsluis
#308: Make sure number_of_decimals_datablock is not checked for Non…
6093092 
…e for each value in the data block.
@priscavdsluis
autoformat: isort & black
c5482f8
@priscavdsluis
#308: Remove cast, and rename elem to value
cb7a340
@priscavdsluis
Merge branch 'main' into feat/308-improve-bc-file-formatting
6cf624a
@priscavdsluis
#308: Add test_ prefix to test function name.
2ba0d8b
@priscavdsluis
#308: Replace number_of_decimals with float_format
0d8c014
@priscavdsluis
#308: Remove float_to_str function.
6de790d
@priscavdsluis
#308: Make float_forman non-optional and by default empty.
b9ba978
priscavdsluis and others added 27 commits November 14, 2022 09:55
@priscavdsluis
autoformat: isort & black
e602db2
@priscavdsluis
#308: Add formatting for poly file
80f3ff4
@priscavdsluis
autoformat: isort & black
e090c32
@priscavdsluis
#308: Add configuration to test cases.
87a483a
@priscavdsluis
autoformat: isort & black
b312966
@priscavdsluis
Merge branch 'feat/308-improve-bc-file-formatting' into feat/308-impr…
9cd5e96 
…ove-file-float-formatting
@priscavdsluis
#308: Add SerializerConfig as to _get_serializer callable
41bab71
@priscavdsluis
autoformat: isort & black
4c0c353
@priscavdsluis
#308: Add formatting for xyz file
de5de22
@priscavdsluis
autoformat: isort & black
79a19da
@priscavdsluis
#308: Add extra test case
1e9e3ee
@priscavdsluis
#308: Add formatting for dimr file
25f4b1f
@priscavdsluis
autoformat: isort & black
d38d960
@priscavdsluis
#308: Add formatting for bui file
9fc8418
@priscavdsluis
autoformat: isort & black
c7c0d07
@priscavdsluis
Merge branch 'main' into feat/308-improve-bc-file-formatting
5aa1364
@priscavdsluis
Merge branch 'feat/308-improve-bc-file-formatting' into feat/308-impr…
8c4aea6 
…ove-file-float-formatting
@priscavdsluis
#308: Add docs.
efc3711
@priscavdsluis
autoformat: isort & black
ffec907
@priscavdsluis
#308: Add formatting for rr node topology file
c7bbbfc
@priscavdsluis
#308: Fix test
54caffa
@priscavdsluis
autoformat: isort & black
1f3dbfe
@priscavdsluis
Merge pull request #417 from Deltares/feat/308-improve-file-float-for…
aed6dc1 
…matting

#308: Improve file float formatting
@priscavdsluis
#308: Update comment
8e0530c
@priscavdsluis
#425: Use construct function for create IO INI models. This will skip…
423372f 
… validation.
@priscavdsluis
#425: Config IO INI models with validate_assignment = False
59ca1b3
@priscavdsluis
#425: Cast int to str
d169a24
@priscavdsluis priscavdsluis linked an issue Nov 22, 2022 that may be closed by this pull request
Improve performance writing BC/INI files #425
Open
@sonarcloud
Copy link

sonarcloud bot commented Nov 22, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 6 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@priscavdsluis
Copy link
Contributor Author

priscavdsluis commented Nov 22, 2022 •
edited

With these changes the @veenstrajelmer tests will be about 33% faster (from ~30s to ~20s on 100000 rows)
There is still a problem with the AutoStart option (hence the failing tests), which may be solved in a different issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Improve performance writing BC/INI files
2 participants
@priscavdsluis @veenstrajelmer