New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
testscr_check_kdb_internal_suite: very slow at least on a7 #3512
Comments
Thank you for creating the issue! As said in #3510 we should probably reduce the number of tests for normal builds and only have them run for special builds (but then really all of them). Ideally these builds can also be executed from PRs (e.g. if someone modifies a storage plugin). Maybe these special builds also include fuzzing methods #185, which is also only needed from time to time (or when relevant changes are done). |
Just an idea: maybe |
Another idea: we could use ramdisk for parts or all of the tests, at least on slow machines. |
Yes, this is might be the easier solution! |
This is a result of #2595. Now we finally test all storage plugins. Maybe Currently the tests are executed with these pluigns:
(based on the testdata in The The script also runs everything twice. Once for the On my machine the test takes only ~63s (Ryzen 3700X, 32GB RAM, NVMe SSD), but the distribution looks like this:
The fact that
PS. the Maybe disabling |
Thanks for the hint, this is actually useless, it does not test the storage plugins. Furthermore, it would be a very easy fix.
This is on purpose to test such KeySets. Further tests with bigger KeySets can be added (contributions welcome!) but this should not replace the simple tests.
Reusing KDB might make the tests faster but then they would be different tests. Tests that test the behavior of sequences of get/set are in Btw. using tmpfs might be positive for all tests, testscr_check_kdb_internal_suite is not the only test that takes some time because of I/O load. Maybe Robert will look at these things. |
What exactly is the difference between testing a I can't think of a reason why a plugin would be able to store What exactly is the goal of In particular, I don't see the point of running the
Yes, reusing the
That is certainly true. But definitely needs further investigation. There are a few Another thing I just noticed, |
To test round-trip and other storage properties of a given mountpoint.
The difference is for storage plugin developers: you get a very small example of when your plugin fails and not a huge test case where you need to find out yourself what actually the problem is. It is possible that there is some overlap between the tests but I would not waste my time (nor I would ask anyone to waste their time) trying to eliminate this.
As long as we do not know for sure that this is a bottleneck (and I doubt it), such optimizations would be waste of development time. |
Ok, but that should not be the goal of The tests we run in our CI are there to ensure PRs don't introduce regressions and adhere to some baseline requirements. The CI is not a debugging tool for developers. We can provide a separate test suite that plugin developers can run themselves to get details about what is going wrong. |
Exactly, thus the idea to extract the tests which can always run and other tests only to be run for releases (or development). This was my first comment on this issue. |
The tests are running much faster on
I'll change this to low priority until somebody can fix the real underlying problem as @kodebach pointed out. |
The |
Thank you very much! Can we close this issue or is separation of some tests still useful? |
I mark this issue stale as it did not have any activity for one year. I'll close it in two weeks if no further activity occurs. If you want it to be alive again, ping the issue by writing a message here or create a new issue with the remainder of this issue. |
I closed this issue now because it has been inactive for more than one year. If I closed it by mistake, please do not hesitate to reopen it or create a new issue with the remainder of this issue. |
The
testscr_check_kdb_internal_suite
takes more than 30 minutes ona7
.We need to investigate why this takes so long and if things can be improved.
The text was updated successfully, but these errors were encountered: