-
Notifications
You must be signed in to change notification settings - Fork 388
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
OAK-10778 - Support downloading from Mongo in parallel. #1435
Commits on Apr 25, 2024
-
Support downloading from Mongo in parallel. Adds new boolean system p…
…roperty: oak.indexer.pipelined.mongoParallelDump.
Configuration menu - View commit details
-
Copy full SHA for e903c7a - Browse repository at this point
Copy the full SHA e903c7aView commit details -
Configuration menu - View commit details
-
Copy full SHA for a074ea0 - Browse repository at this point
Copy the full SHA a074ea0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5f58aa9 - Browse repository at this point
Copy the full SHA 5f58aa9View commit details -
Retrieve from system properties the version of the Mongo docker image…
… to be used for tests.
Configuration menu - View commit details
-
Copy full SHA for 65387ce - Browse repository at this point
Copy the full SHA 65387ceView commit details
Commits on Apr 26, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 45623cf - Browse repository at this point
Copy the full SHA 45623cfView commit details -
Improve documentation and replace use of var keyword by explicit type…
… declaration where this makes code more clear.
Configuration menu - View commit details
-
Copy full SHA for 934bea6 - Browse repository at this point
Copy the full SHA 934bea6View commit details
Commits on Apr 29, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 3440733 - Browse repository at this point
Copy the full SHA 3440733View commit details -
Configuration menu - View commit details
-
Copy full SHA for 55b1157 - Browse repository at this point
Copy the full SHA 55b1157View commit details -
Reuse the name defined in MongoDockerRule for the Docker image for in…
…tegration tests.
Configuration menu - View commit details
-
Copy full SHA for c412f2f - Browse repository at this point
Copy the full SHA c412f2fView commit details
Commits on Apr 30, 2024
-
Configuration menu - View commit details
-
Copy full SHA for b2684bd - Browse repository at this point
Copy the full SHA b2684bdView commit details -
Reduce frequency of logging progress messages in the downloader from …
…every 10k to every 20k.
Configuration menu - View commit details
-
Copy full SHA for 051cd51 - Browse repository at this point
Copy the full SHA 051cd51View commit details
Commits on May 2, 2024
-
When downloading the full range of values in _modified, use $gte(0) a…
…nd $lte(Long.MAX_VALUE) instead $exits(_modified). $exists also checks for the property being equals to null, which cannot be verified just by looking at an index, because indexes in MongoDB do not contain null values. Using $exists requires retrieving the full document from the column store, which dramatically slows down the traversal.
Configuration menu - View commit details
-
Copy full SHA for f5fd28c - Browse repository at this point
Copy the full SHA f5fd28cView commit details
Commits on May 6, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 3fd5c3c - Browse repository at this point
Copy the full SHA 3fd5c3cView commit details
Commits on May 7, 2024
-
Improve error handling when parallel download fails with some excepti…
…on. Ensures that both download threads are shutdown gracefully. Small refactoring.
Configuration menu - View commit details
-
Copy full SHA for 63254c2 - Browse repository at this point
Copy the full SHA 63254c2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 70f431e - Browse repository at this point
Copy the full SHA 70f431eView commit details
Commits on May 8, 2024
-
Configuration menu - View commit details
-
Copy full SHA for a1bc963 - Browse repository at this point
Copy the full SHA a1bc963View commit details
Commits on May 10, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 8aceb22 - Browse repository at this point
Copy the full SHA 8aceb22View commit details -
Add a test for Pipelined strategy where the mongo filter does not mat…
…ch any documents. Add additional comments.
Configuration menu - View commit details
-
Copy full SHA for 9e1a1a3 - Browse repository at this point
Copy the full SHA 9e1a1a3View commit details -
Make NodeDocumentCodec thread safe and simplify logic of switch state…
…ment to reduce its size.
Configuration menu - View commit details
-
Copy full SHA for d76d5fe - Browse repository at this point
Copy the full SHA d76d5feView commit details -
Configuration menu - View commit details
-
Copy full SHA for 0d6f607 - Browse repository at this point
Copy the full SHA 0d6f607View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7dad006 - Browse repository at this point
Copy the full SHA 7dad006View commit details -
Configuration menu - View commit details
-
Copy full SHA for d7d2f12 - Browse repository at this point
Copy the full SHA d7d2f12View commit details -
Fix: Download only documents with _modified also when doing a column …
…traversal (retry on connection errors false). Documents without _modified field are not needed to build the FFS and like this there is no need to do a null check when calling getModified on the documents.
Configuration menu - View commit details
-
Copy full SHA for 1dcb944 - Browse repository at this point
Copy the full SHA 1dcb944View commit details
Commits on May 13, 2024
-
Configuration menu - View commit details
-
Copy full SHA for d3f2377 - Browse repository at this point
Copy the full SHA d3f2377View commit details