Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[feat]Rocksdb backend #121

Open
wants to merge 67 commits into
base: master
Choose a base branch
from
Open

[feat]Rocksdb backend #121

wants to merge 67 commits into from

Commits on Jul 17, 2021

  1. RocksDB hashtable backend for Tensorflow Recommenders Addons.

    @bashimao
    bashimao committed Jul 17, 2021
    Configuration menu
    Copy the full SHA
    d6a137c View commit details
    Browse the repository at this point in the history

Commits on Jul 18, 2021

  1. Fix major bugs and establish proper compiler toolchain.

    @bashimao
    bashimao committed Jul 18, 2021
    Configuration menu
    Copy the full SHA
    0b48160 View commit details
    Browse the repository at this point in the history

  2. Only create if not in read_only mode.

    @bashimao
    bashimao committed Jul 18, 2021
    Configuration menu
    Copy the full SHA
    6eb84c4 View commit details
    Browse the repository at this point in the history

  3. Decouple cleaning from creating a column family.

    @bashimao
    bashimao committed Jul 18, 2021
    Configuration menu
    Copy the full SHA
    e8cbf37 View commit details
    Browse the repository at this point in the history

Commits on Jul 20, 2021

  1. Correct pending linker issues.

    @bashimao
    bashimao committed Jul 20, 2021
    Configuration menu
    Copy the full SHA
    9cdaec7 View commit details
    Browse the repository at this point in the history

  2. Add op definitions for RocksDB.

    @bashimao
    bashimao committed Jul 20, 2021
    Configuration menu
    Copy the full SHA
    42aff13 View commit details
    Browse the repository at this point in the history

  3. Need to include op-definitions in compiler directives.

    @bashimao
    bashimao committed Jul 20, 2021
    Configuration menu
    Copy the full SHA
    222fe26 View commit details
    Browse the repository at this point in the history

  4. Automatically import RocksDB table object upon module load.

    @bashimao
    bashimao committed Jul 20, 2021
    Configuration menu
    Copy the full SHA
    052c2e2 View commit details
    Browse the repository at this point in the history

  5. Add interface code to properly transfer arguments to the C++ implemen… 

    …tation.
    @bashimao
    bashimao committed Jul 20, 2021
    Configuration menu
    Copy the full SHA
    96856d7 View commit details
    Browse the repository at this point in the history

  6. Typo...

    @bashimao
    bashimao committed Jul 20, 2021
    Configuration menu
    Copy the full SHA
    06738d2 View commit details
    Browse the repository at this point in the history

  7. Fix opening non-existent database bug.

    @bashimao
    bashimao committed Jul 20, 2021
    Configuration menu
    Copy the full SHA
    23a0b0e View commit details
    Browse the repository at this point in the history

  8. Merge branch 'tensorflow:master' into rocksdb-backend

    @bashimao
    bashimao committed Jul 20, 2021
    Configuration menu
    Copy the full SHA
    b55ac03 View commit details
    Browse the repository at this point in the history

  9. Fix typo.

    @bashimao
    bashimao committed Jul 20, 2021
    Configuration menu
    Copy the full SHA
    c2ae3c3 View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2021

  1. Add connection pool functionality to allow access multiple tables in … 

    …the same database simultaneously.
    @bashimao
    bashimao committed Jul 22, 2021
    Configuration menu
    Copy the full SHA
    159c4c3 View commit details
    Browse the repository at this point in the history

  2. Fix insert bug. Add configurable size function, two modes: 1) through… 

    … RocksDB estimation; 2) through iteration method.
    @bashimao
    bashimao committed Jul 22, 2021
    Configuration menu
    Copy the full SHA
    51e8389 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'rocksdb-backend' of github.com:bashimao/recommenders-ad… 

    …dons into rocksdb-backend
    @bashimao
    bashimao committed Jul 22, 2021
    Configuration menu
    Copy the full SHA
    e8558e1 View commit details
    Browse the repository at this point in the history

Commits on Jul 23, 2021

  1. Fix up all known issues, add registry functionalities.

    @bashimao
    bashimao committed Jul 23, 2021
    Configuration menu
    Copy the full SHA
    4c0d09f View commit details
    Browse the repository at this point in the history

  2. Make DBWrapper thread-safe.

    @bashimao
    bashimao committed Jul 23, 2021
    Configuration menu
    Copy the full SHA
    6287344 View commit details
    Browse the repository at this point in the history

  3. Switch to reader/writer lock for performance.

    @bashimao
    bashimao committed Jul 23, 2021
    Configuration menu
    Copy the full SHA
    101275c View commit details
    Browse the repository at this point in the history

  4. Allow settign export_path from Python.

    @bashimao
    bashimao committed Jul 23, 2021
    Configuration menu
    Copy the full SHA
    ccc7b58 View commit details
    Browse the repository at this point in the history

  5. Extract the default value's shape.

    @bashimao
    bashimao committed Jul 23, 2021
    Configuration menu
    Copy the full SHA
    d9e8680 View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2021

  1. Improved support for multi-dimensional tensors.

    @bashimao
    bashimao committed Jul 24, 2021
    Configuration menu
    Copy the full SHA
    ed8f58a View commit details
    Browse the repository at this point in the history

  2. Improve multi-threading stability. Added limit for export to pass tes… 

    …ts. (THIS MIGHT BE WRONG!)
    @bashimao
    bashimao committed Jul 24, 2021
    Configuration menu
    Copy the full SHA
    15c7953 View commit details
    Browse the repository at this point in the history

  3. Fix typo.

    @bashimao
    bashimao committed Jul 24, 2021
    Configuration menu
    Copy the full SHA
    2fb2dec View commit details
    Browse the repository at this point in the history

  4. Now passing all but 2 tests.

    @bashimao
    bashimao committed Jul 24, 2021
    Configuration menu
    Copy the full SHA
    2dff086 View commit details
    Browse the repository at this point in the history

  5. Loop condition was wrong.

    @bashimao
    bashimao committed Jul 24, 2021
    Configuration menu
    Copy the full SHA
    c7828ee View commit details
    Browse the repository at this point in the history

  6. Pointer value instead of pointed to value written to file.

    @bashimao
    bashimao committed Jul 24, 2021
    Configuration menu
    Copy the full SHA
    43fe98e View commit details
    Browse the repository at this point in the history

  7. Protect against divide by zero.

    @bashimao
    bashimao committed Jul 24, 2021
    Configuration menu
    Copy the full SHA
    1de6a0a View commit details
    Browse the repository at this point in the history

  8. Add Import/Export lock to avoid overlapping transactions - despite re… 

    …ader-writer lock.
    @bashimao
    bashimao committed Jul 24, 2021
    Configuration menu
    Copy the full SHA
    626ce3a View commit details
    Browse the repository at this point in the history

  9. Remove export lock idea. Instead make the mutex absolute. We have a p… 

    …otential race-condition otherwise.
    @bashimao
    bashimao committed Jul 24, 2021
    Configuration menu
    Copy the full SHA
    972c4f9 View commit details
    Browse the repository at this point in the history

  10. Passing all relevant checks.

    @bashimao
    bashimao committed Jul 24, 2021
    Configuration menu
    Copy the full SHA
    e6d52d9 View commit details
    Browse the repository at this point in the history

Commits on Aug 10, 2021

  1. Reformatted cc-/h-files to conform to Google-style.

    @bashimao
    bashimao committed Aug 10, 2021
    Configuration menu
    Copy the full SHA
    c770dc0 View commit details
    Browse the repository at this point in the history

  2. Tick up copyright disclaimer version number.

    @bashimao
    bashimao committed Aug 10, 2021
    Configuration menu
    Copy the full SHA
    62600bd View commit details
    Browse the repository at this point in the history

Commits on Aug 20, 2021

  1. Reformat with yapf.

    @bashimao
    bashimao committed Aug 20, 2021
    Configuration menu
    Copy the full SHA
    c8ce053 View commit details
    Browse the repository at this point in the history

Commits on Aug 22, 2021

  1. Tick down rules_foreign_cc package version according to Heka's recomm… 

    …endation.
    @bashimao
    bashimao committed Aug 22, 2021
    Configuration menu
    Copy the full SHA
    ff725b2 View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2021

  1. Merge branch 'master' into rocksdb-backend 

    # Conflicts:
#	tensorflow_recommenders_addons/dynamic_embedding/python/ops/dynamic_embedding_variable.py
    @bashimao
    bashimao committed Aug 30, 2021
    Configuration menu
    Copy the full SHA
    eff7a39 View commit details
    Browse the repository at this point in the history

  2. Reformat python code.

    @bashimao
    bashimao committed Aug 30, 2021
    Configuration menu
    Copy the full SHA
    de7fb53 View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2021

  1. Merge branch 'master' into rocksdb-backend

    @bashimao
    bashimao committed Oct 17, 2021
    Configuration menu
    Copy the full SHA
    82816b0 View commit details
    Browse the repository at this point in the history

Commits on Oct 19, 2021

  1. Code cosmetics

    @bashimao
    bashimao committed Oct 19, 2021
    Configuration menu
    Copy the full SHA
    804efb9 View commit details
    Browse the repository at this point in the history

Commits on Oct 24, 2021

  1. Fix Python format according to proposed coding standards.

    @bashimao
    bashimao committed Oct 24, 2021
    Configuration menu
    Copy the full SHA
    e116424 View commit details
    Browse the repository at this point in the history

  2. Manually ran clang format. Not sure why it did not work before.

    @bashimao
    bashimao committed Oct 24, 2021
    Configuration menu
    Copy the full SHA
    36bd30a View commit details
    Browse the repository at this point in the history

Commits on Jan 5, 2022

  1. Merge branch 'master' into rocksdb-backend

    @rhdong
    rhdong committed Jan 5, 2022
    Configuration menu
    Copy the full SHA
    bbeca4a View commit details
    Browse the repository at this point in the history

Commits on Jan 23, 2022

  1. Merge branch 'master' into rocksdb-backend

    @bashimao
    bashimao committed Jan 23, 2022
    Configuration menu
    Copy the full SHA
    05d6d2a View commit details
    Browse the repository at this point in the history

  2. Fix up Bazel rules for 0.6 foreign_cc.

    @bashimao
    bashimao committed Jan 23, 2022
    Configuration menu
    Copy the full SHA
    45a748b View commit details
    Browse the repository at this point in the history

Commits on Jan 26, 2022

  1. Somehow these libs are no longer needed?!

    @bashimao
    bashimao committed Jan 26, 2022
    Configuration menu
    Copy the full SHA
    567e92d View commit details
    Browse the repository at this point in the history

  2. Resolve most wiggles that prevent compiling ant testing.

    @bashimao
    bashimao committed Jan 26, 2022
    Configuration menu
    Copy the full SHA
    dbcd36b View commit details
    Browse the repository at this point in the history

  3. Temporary fix for library search path.

    @bashimao
    bashimao committed Jan 26, 2022
    Configuration menu
    Copy the full SHA
    ddbb499 View commit details
    Browse the repository at this point in the history

Commits on Jan 27, 2022

  1. Apply yapf.

    @bashimao
    bashimao committed Jan 27, 2022
    Configuration menu
    Copy the full SHA
    bff04bb View commit details
    Browse the repository at this point in the history

  2. Apply Clang Format.

    @bashimao
    bashimao committed Jan 27, 2022
    Configuration menu
    Copy the full SHA
    08c83e4 View commit details
    Browse the repository at this point in the history

  3. Change to a more commonly used path that has symlinks.

    @bashimao
    bashimao committed Jan 27, 2022
    Configuration menu
    Copy the full SHA
    fc7ac16 View commit details
    Browse the repository at this point in the history

  4. Add dependencies to ci images.

    @bashimao
    bashimao committed Jan 27, 2022
    Configuration menu
    Copy the full SHA
    f6bd10d View commit details
    Browse the repository at this point in the history

  5. Fix syntax error. ;-)

    @bashimao
    bashimao committed Jan 27, 2022
    Configuration menu
    Copy the full SHA
    46be6bd View commit details
    Browse the repository at this point in the history

Commits on Mar 27, 2022

  1. Merge branch 'master' into rocksdb-backend

    @bashimao
    bashimao committed Mar 27, 2022
    Configuration menu
    Copy the full SHA
    ecdc998 View commit details
    Browse the repository at this point in the history

  2. Now included in base image. tensorflow#121 (comment)

    @bashimao
    bashimao committed Mar 27, 2022
    Configuration menu
    Copy the full SHA
    0011199 View commit details
    Browse the repository at this point in the history

  3. Simplify a couple of things.

    @bashimao
    bashimao committed Mar 27, 2022
    Configuration menu
    Copy the full SHA
    fbd18bc View commit details
    Browse the repository at this point in the history

  4. Can process true/false. So why is it 0/1?

    @bashimao
    bashimao committed Mar 27, 2022
    Configuration menu
    Copy the full SHA
    b32139d View commit details
    Browse the repository at this point in the history

  5. Update C++ portion of RocksDB implementation to accommodate interface… 

    … adjustments.
    @bashimao
    bashimao committed Mar 27, 2022
    Configuration menu
    Copy the full SHA
    15714b3 View commit details
    Browse the repository at this point in the history

Commits on May 26, 2022

  1. Merge branch 'tensorflow:master' into rocksdb-backend

    @bashimao
    bashimao committed May 26, 2022
    Configuration menu
    Copy the full SHA
    9811e82 View commit details
    Browse the repository at this point in the history

Commits on Jun 12, 2022

  1. Merge branch 'master' into rocksdb-backend

    @bashimao
    bashimao committed Jun 12, 2022
    Configuration menu
    Copy the full SHA
    db166a3 View commit details
    Browse the repository at this point in the history

  2. Allow using pre-built rocksdb version.

    @bashimao
    bashimao committed Jun 12, 2022
    Configuration menu
    Copy the full SHA
    366638d View commit details
    Browse the repository at this point in the history

Commits on Aug 15, 2022

  1. Merge branch 'master' into rocksdb-backend

    @bashimao
    bashimao committed Aug 15, 2022
    Configuration menu
    Copy the full SHA
    a1bb85f View commit details
    Browse the repository at this point in the history

  2. Fix up bazel environment to use pre-compiled rocksdb.

    @bashimao
    bashimao committed Aug 15, 2022
    Configuration menu
    Copy the full SHA
    778f244 View commit details
    Browse the repository at this point in the history

  3. Implement MemoryUsed API.

    @bashimao
    bashimao committed Aug 15, 2022
    Configuration menu
    Copy the full SHA
    f86ec70 View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2022

  1. Bugfixes and implement FindWithExists API.

    @bashimao
    bashimao committed Aug 16, 2022
    Configuration menu
    Copy the full SHA
    eaa6219 View commit details
    Browse the repository at this point in the history

  2. Minor bugfix.

    @bashimao
    bashimao committed Aug 16, 2022
    Configuration menu
    Copy the full SHA
    4ebd8ac View commit details
    Browse the repository at this point in the history

  3. Add minor error check.

    @bashimao
    bashimao committed Aug 16, 2022
    Configuration menu
    Copy the full SHA
    4a86ac2 View commit details
    Browse the repository at this point in the history

  4. Accumulation API. Dummy implementation.

    @bashimao
    bashimao committed Aug 16, 2022
    Configuration menu
    Copy the full SHA
    53b732d View commit details
    Browse the repository at this point in the history