New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
optimisation: some things seem a bit slower since 1.25 #2122
Comments
Are those all compiled with the same GHC version ? Actually no, those are built with 9.0.2, 9.2.2, 9.4.6 and 9.6.3 respectively. So it's not quite a fair test. They are all ARM binaries, and all optimised builds (I'm fairly certain). |
Profiling and some performance fixes were done as part of #2153. Summary: 1.32.3 is much faster than 1.29-1.32, but still slower than 1.25 (which is somewhat expected since it is providing more powerful features in the journal format).
|
See #2153 for some recent benchmarking data. That issue fixed a major performance drop for unusually large numbers of accounts from 1.29 onward. On this one we'll continue discussing the remaining, lesser performance drop since 1.25. This one can be argued as a wish rather than a bug or regression since absence of (reasonable) performance fluctuations has never been promised. I think as part of resolving this issue though we should implement performance regression testing, and try to avoid or at least document any performance drops going forward. |
Well, it's true that performance was never guaranteed, but I'd still consider it a regression, since hledger became slower, not faster since 1.25. With my ledger file, with 17k transaction and 470 accounts, I get similar results as in your last comment, runtime for balance is somewhere around ~1,5 times slower in git head than in 1.25. It may not be much, but it's enough to cross the subjective threshold of annoyance. I know that there have been a lot of features added to hledger since 1.25 that explain the general slowdown. But then, there is this philosophical question should a feature add its weight in terms of performance even if not used? Could I get a much faster stripped down version with only basic features for a daily use, and have slower execution only when I actually use some advanced or heavier functions? As far as I'm concerned, I can keep using 1.25 for the next few years until my journal file grows enough for me to reach the annoyance threshold and have to revisit again. Or split the file and get on with the life. |
Noted @markokocic, thanks for your input as one of the (few?) users running large >10k-txn files. I agree it's arguable, and it's at least a bug in our process that it took so long to discover. But I feel like letting this one escape the dreaded black label on a technicality. There's no bounty involved since I reported it first so I'm assuming it's harmless to do so. Certainly it'll be considered a regression if it happens again, and certainly I hope we'll find ways to bring performance back to 1.25 level and beyond. |
PS any ideas for simple robust automated performance testing ? I'm thinking about:
|
PS I recently added memory usage to the I experimented with printing CPU time rather than elapsed time but it seems unlikely to be needed. |
I don't think saving results to file would help, since the run results may vary across runs. Maybe have a script that will do benchmark against ledger, hledger-1.25 and previous release which can be run on demand or on a ci. Btw, I just checked a new Regarding speed, I finally managed to fix my journal to its format can be read directly by toht ledger and hledger, and here's the quick comparison with the latest from git.
I like hledger usability more, but wish it would be possible to approach ledger balance performance as close as possible :) Also, I don't care if the ticket is a regression or whoever reported it, at least not in a sense of bounty, I'd only wish some performance back. |
I tend to see this (register much faster, but print and balance a bit slower) in recent releases:
More notably, right now on this machine I can see ~25k TPS with hledger 1.25
stats
, but only ~15k TPS with 1.32. A new round of profiling could be in order.Related:
The text was updated successfully, but these errors were encountered: