Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[server][da-vinci] Bumped RocksDB dep and adopt multiget async io by default #950

Merged
merged 4 commits into from
May 9, 2024
Merged

[server][da-vinci] Bumped RocksDB dep and adopt multiget async io by default #950

merged 4 commits into from
May 9, 2024

Commits on May 7, 2024

  1. [server][da-vinci] Bumped RocksDB dep and adopt multiget async io by … 

    …default

This PR bumps up the RocksDB dep and expose a config to enable async io
for multi-get and the default value is true.
rocksdb.read.async.io.enabled: default true
In theoy, with this config and posix filesystem, RocksDB multiget API
will be speeded up quite a bit based on the benchmarking:
https://rocksdb.org/blog/2022/10/07/asynchronous-io-in-rocksdb.html

So far, such optimization only applies to the chunk lookup for large
value/rmd, and if it is proved to be more performant by checking
the lookup latency for large value in the read path, we can apply
such optimization in more areas:
1. DaVinci with DISK mode.
2. Ingestion code path by looking up entries in batch for AA/WC use cases.
3. Regular read path.
So far, multi-get API can only be used against the same RocksDB database,
so we might need some re-org of RocksDB databases if this API is truly
helpful.
    @gaojieliu
    gaojieliu committed May 7, 2024
    Configuration menu
    Copy the full SHA
    801001e View commit details
    Browse the repository at this point in the history

  2. Removed unused code

    @gaojieliu
    gaojieliu committed May 7, 2024
    Configuration menu
    Copy the full SHA
    8d982e5 View commit details
    Browse the repository at this point in the history

  3. Addressed the spotbug issue

    @gaojieliu
    gaojieliu committed May 7, 2024
    Configuration menu
    Copy the full SHA
    c5fb9e8 View commit details
    Browse the repository at this point in the history

  4. Addressed comment

    @gaojieliu
    gaojieliu committed May 7, 2024
    Configuration menu
    Copy the full SHA
    e5e22b5 View commit details
    Browse the repository at this point in the history