Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Explore prefetch-compatible trie layout #117

Open
eiennohito opened this issue Oct 20, 2021 · 0 comments
Open

Explore prefetch-compatible trie layout #117

eiennohito opened this issue Oct 20, 2021 · 0 comments
Labels
enhancement New feature or request performance Something is slow
Milestone

Comments

@eiennohito
Copy link
Collaborator

eiennohito commented Oct 20, 2021

When building a dictionary, we need to build Trie index.

Probably, we shouldn't move away from darts-clone-compatible structure, but there can be variations on the construction. It would be interesting to explore whether it is possible to make prefetch-compatible trie that could load the trie entries in L1 cache for the next symbol speculatively, even with not 100% accuracy (e.g. getting hits for high-frequency entries).

Rust Implementations to Note: https://github.com/takuyaa/yada (it seems to follow darts-clone internal structure with original darts construction procedure)

@eiennohito eiennohito added this to the 0.7 milestone Oct 20, 2021
@eiennohito eiennohito changed the title Building Trie Building Trie Index Oct 20, 2021
@eiennohito eiennohito changed the title Building Trie Index Explore prefetch-compatible trie layout Oct 25, 2021
@eiennohito eiennohito added enhancement New feature or request performance Something is slow labels Oct 25, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request performance Something is slow
Projects
None yet
Development

No branches or pull requests

1 participant