basename_natural: Improve performance #2943
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
ISSUE TYPE
RUNTIME ENVIRONMENT
CHECKLIST
CONTRIBUTING
document has been read [REQUIRED]DESCRIPTION
Various optimizations to
basename_natural
functions.Breaking change: The regex was from
\D
to\D+
so that all non-digits are grouped together and less elements are generated inbasename_list
, this results in the sorting order being changed so that numbers come before non-numbers after non-number prefixes. For example:Tests have been updated to reflect this. If this change is undesirable then I do have a version that keeps the original behavior, however it's slightly slower and much more complex.
MOTIVATION AND CONTEXT
Helps with #1173, but doesn't completely address it.
In certain scenarios with file sorting method left as the default natural sort, a large chunk of time used in viewing large directories is spent on sorting. Running
ranger --profile
on a directory with 76k files on an NVMe drive shows 16 seconds out of the 24 seconds startup time is spent generating the natural sort keys.TESTING
Ran tests with
make test_py
on Python 3.11.Results of profiling with
ranger --profile
on my/nix/store
directory, which has 76k files:Testing only
basename_natural
withtimeit
shows an improvement from 15.0s to 1.3s on the same directory. (Alternate version that keeps the sorting behavior runs in 2.2s).IMAGES / VIDEOS