{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":6934395,"defaultBranch":"main","name":"rocksdb","ownerLogin":"facebook","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2012-11-30T06:16:18.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/69631?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1714595653.0","currentOid":""},"activityList":{"items":[{"before":"4eaf6281202c87fa0c09c07c0ca197a70004fd66","after":"2eb404de13499e2c7611ef2cc82e2feca742debf","ref":"refs/heads/main","pushedAt":"2024-05-16T04:17:53.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Print non-ok status for multi_ops_txns_stress test (#12660)\n\nSummary:\nCurrently `assert(s.ok())` does not offer much information to debug.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12660\n\nTest Plan:\nrevert https://github.com/facebook/rocksdb/issues/12639 and run `python3 ./tools/db_crashtest.py blackbox --write_policy write_prepared --recycle_log_file_num=1 --threads=1 --test_multiops_txn`\n\nbefore this PR:\n```\nExit Before Killing\nstdout:\n\nstderr:\n db_stress: db_stress_tool/multi_ops_txns_stress.cc:1529: void rocksdb::MultiOpsTxnsStressTest::PreloadDb(rocksdb::SharedState*, int, uint32_t, uint32_t, uint32_t, uint32_t): Assertion `s.ok()' failed.\n```\n\nafter this PR:\n```\nExit Before Killing\nstdout:\n\nstderr:\n Verification failed: PreloadDB failed: Invalid argument: WriteOptions::disableWAL option is not supported if DBOptions::recycle_log_file_num > 0\ndb_stress: db_stress_tool/db_stress_test_base.cc:517: void rocksdb::StressTest::ProcessStatus(rocksdb::SharedState*, std::string, rocksdb::Status, bool) const: Assertion `false' failed.\nReceived signal 6 (Aborted)\n```\n\nReviewed By: ajkr\n\nDifferential Revision: D57410819\n\nPulled By: cbi42\n\nfbshipit-source-id: a03c2202c3fd666eb2f58bae24e0c9e3e6ed4265","shortMessageHtmlLink":"Print non-ok status for multi_ops_txns_stress test (#12660)"}},{"before":"3ed46e0668f840bea490e29beeac7777c50ae8fb","after":"4eaf6281202c87fa0c09c07c0ca197a70004fd66","ref":"refs/heads/main","pushedAt":"2024-05-16T02:16:41.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Add `Iterator` property \"rocksdb.iterator.is-value-pinned\" (#12659)\n\nSummary:\n`ReadOptions::pin_data` already has the effect of pinning the `Slice` returned by `Iterator::value()` when the value is stored inline (e.g., `kTypeValue`). This PR adds a bit of visibility into that via a new `Iterator` property, \"rocksdb.iterator.is-value-pinned\", as well as some documentation and tests.\n\nSee also: https://github.com/facebook/rocksdb/issues/12658\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12659\n\nReviewed By: cbi42\n\nDifferential Revision: D57391200\n\nPulled By: ajkr\n\nfbshipit-source-id: 0caa8db27ca1aba86ee2addc3dfd6f0e003d32e2","shortMessageHtmlLink":"Add Iterator property \"rocksdb.iterator.is-value-pinned\" (#12659)"}},{"before":"b4c6956a5933792267fa2b9ac7ba7c95117abdfc","after":"3ed46e0668f840bea490e29beeac7777c50ae8fb","ref":"refs/heads/main","pushedAt":"2024-05-14T23:48:20.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Handle early exit in DBErrorHandlingFSTests (#12655)\n\nSummary:\nTo avoid use-after-free on custom env on ASSERT_WHATEVER failure.\n\nThis is motivated by a rare crash seen in DBErrorHandlingFSTest.WALWriteError (VersionSet::GetObsoleteFiles in a SstFileManagerImpl::ClearError thread) and wanting to rule out this being related to that.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12655\n\nTest Plan: manually seeing ASSERT_WHATEVER failures, especially under ASAN\n\nReviewed By: cbi42\n\nDifferential Revision: D57358202\n\nPulled By: pdillinger\n\nfbshipit-source-id: 4da2a0d73a54380b257e5cc1ab6c666e26b83973","shortMessageHtmlLink":"Handle early exit in DBErrorHandlingFSTests (#12655)"}},{"before":"b9fc13db69ebddc746a0a0ebad7f6cf37144a0cb","after":"b4c6956a5933792267fa2b9ac7ba7c95117abdfc","ref":"refs/heads/main","pushedAt":"2024-05-14T23:37:50.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Add MultiGetEntity AttributeGroup API to stress test (#12640)\n\nSummary:\nContinuing from https://github.com/facebook/rocksdb/pull/12605, adding AttributeGroup `MultiGetEntity` API to stress tests.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12640\n\nTest Plan:\n**AttributeGroup Tests**\n\nNonBatchOps\n```\npython3 tools/db_crashtest.py blackbox --simple --max_key=25000000 --write_buffer_size=4194304 --use_attribute_group=1 --use_put_entity_one_in=1 --use_multi_get=1\n```\n\nBatchOps\n```\npython3 tools/db_crashtest.py blackbox --test_batches_snapshots=1 --max_key=25000000 --write_buffer_size=4194304 --use_attribute_group=1 --use_put_entity_one_in=1 --use_multi_get=1\n```\n\nCfConsistency Test\n```\npython3 tools/db_crashtest.py blackbox --cf_consistency --max_key=25000000 --write_buffer_size=4194304 --use_attribute_group=1 --use_put_entity_one_in=1 --use_multi_get=1\n```\n\n**Non-AttributeGroup Tests**\n\nNonBatchOps\n```\npython3 tools/db_crashtest.py blackbox --simple --max_key=25000000 --write_buffer_size=4194304 --use_attribute_group=0 --use_put_entity_one_in=1 --use_multi_get=1\n```\n\nBatchOps\n```\npython3 tools/db_crashtest.py blackbox --test_batches_snapshots=1 --max_key=25000000 --write_buffer_size=4194304 --use_attribute_group=0 --use_put_entity_one_in=1 --use_multi_get=1\n```\n\nCfConsistency Test\n```\npython3 tools/db_crashtest.py blackbox --cf_consistency --max_key=25000000 --write_buffer_size=4194304 --use_attribute_group=0 --use_put_entity_one_in=1 --use_multi_get=1\n```\n\nReviewed By: ltamasi\n\nDifferential Revision: D57233931\n\nPulled By: jaykorean\n\nfbshipit-source-id: 8cea771ac2e5749050bf5319360c6c5aa476d7d5","shortMessageHtmlLink":"Add MultiGetEntity AttributeGroup API to stress test (#12640)"}},{"before":"1567d50a2754753cd80b1c2e56b862673ad09bbd","after":"b9fc13db69ebddc746a0a0ebad7f6cf37144a0cb","ref":"refs/heads/main","pushedAt":"2024-05-14T22:57:15.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Add padding before timestamp size record if it doesn't fit into a WAL block. (#12614)\n\nSummary:\nIf timestamp size record doesn't fit into a block, without padding `Writer::EmitPhysicalRecord` fails on assert (either `assert(block_offset_ + kHeaderSize + n <= kBlockSize);` or `assert(block_offset_ + kRecyclableHeaderSize + n <= kBlockSize)`, depending on whether recycling log files is enabled) in debug build. In release, current block grows beyond 32K, `block_offset_` gets reset on next `AddRecord` and all the subsequent blocks are no longer aligned by block size.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12614\n\nReviewed By: ltamasi\n\nDifferential Revision: D57302140\n\nPulled By: jowlyzhang\n\nfbshipit-source-id: cacb5cefb7586885e52a8137ae23a95e1aefca2d","shortMessageHtmlLink":"Add padding before timestamp size record if it doesn't fit into a WAL…"}},{"before":"c110091d368b8a01b5be36a14198769e60786c05","after":"1567d50a2754753cd80b1c2e56b862673ad09bbd","ref":"refs/heads/main","pushedAt":"2024-05-14T16:32:03.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Temporarily disable file ingestion if LockWAL is tested (#12642)\n\nSummary:\nAs titled. A proper fix should probably be failing file ingestion if the DB is in a lock wal state as it promises to \"Freezes the logical state of the DB\".\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12642\n\nReviewed By: pdillinger\n\nDifferential Revision: D57235869\n\nPulled By: jowlyzhang\n\nfbshipit-source-id: c70031463842220f865621eb6f53424df27d29e9","shortMessageHtmlLink":"Temporarily disable file ingestion if LockWAL is tested (#12642)"}},{"before":"20213d01a333aca20182e5aab6e4513537d9758c","after":"c110091d368b8a01b5be36a14198769e60786c05","ref":"refs/heads/main","pushedAt":"2024-05-13T22:46:36.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Support read timestamp in ldb (#12641)\n\nSummary:\nAs titled. Also updated sst_dump to print out user-defined timestamp separately and in human readable format.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12641\n\nTest Plan:\nmanually tested:\nExample success run:\n./ldb --db=$TEST_DB multi_get_entity 0x00000000000000950000000000000120 --key_hex --column_family=7 --read_timestamp=1115613683797942 --value_hex\n0x00000000000000950000000000000120 ==> :0x0E0000000A0B080906070405020300011E1F1C1D1A1B181916171415121310112E2F2C2D2A2B282926272425222320213E3F3C3D3A3B383936373435323330314E4F4C4D4A4B484946474445424340415E5F5C5D5A5B58595657545552535051\nExample failed run:\nFailed: GetEntity failed: Invalid argument: column family enables user-defined timestamp while --read_timestamp is not a valid uint64 value.\n\nsst_dump print out:\n'000000000000015D000000000000012B000000000000013B|timestamp:1113554493256671' seq:2330405, type:1 => 010000000504070609080B0A0D0C0F0E111013121514171619181B1A1D1C1F1E212023222524272629282B2A2D2C2F2E313033323534373639383B3A3D3C3F3E\n\nReviewed By: ltamasi\n\nDifferential Revision: D57297006\n\nPulled By: jowlyzhang\n\nfbshipit-source-id: 8486d91468e4f6c0d42dca3c9629f1c45a92bf5a","shortMessageHtmlLink":"Support read timestamp in ldb (#12641)"}},{"before":"7747abdc151be4227a3885379403bfd7a268208b","after":"20213d01a333aca20182e5aab6e4513537d9758c","ref":"refs/heads/main","pushedAt":"2024-05-13T20:16:31.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Fix crash in CompactFiles() of conflict range under `preclude_last_level_data_seconds > 0` (#12628)\n\nSummary:\n**Context/Summary:**\n\nPreviously `CompactFiles()` used `RangeOverlapWithCompaction()` to check for conflict when sanitizing input files while later used `FilesRangeOverlapWithCompaction()` to assert for no conflict. The latter function checks for more conflict scenarios than the former does, particularly the ones arising from `preclude_last_level_data_seconds > 0` (i.e, compaction can output to second-to-the-last level). So we ran into assertion violation in `CompactFiles()` like below\n```\n Assertion `output_level == 0 || !FilesRangeOverlapWithCompaction( input_files, output_level, Compaction::EvaluatePenultimateLevel(vstorage, ioptions_, start_level, output_level))' failed.\n```\n\nThis PR make `CompactFiles()` used `FilesRangeOverlapWithCompaction()` and return Aborted status upon range conflict instead of crashing (during debug build) or proceed incorrectly (during non-debug build). To do so cleanly, I included a refactoring to make `FilesRangeOverlapWithCompaction()` part of `SanitizeAndConvertCompactionInputFiles()`, replacing `RangeOverlapWithCompaction()`.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12628\n\nTest Plan: New UT crashed before the fix and return correct status after the fix.\n\nReviewed By: cbi42\n\nDifferential Revision: D57123536\n\nPulled By: hx235\n\nfbshipit-source-id: f963a2c9e7ba1a9927a67fcc87f0dce126d3a430","shortMessageHtmlLink":"Fix crash in CompactFiles() of conflict range under `preclude_last_le…"}},{"before":"b75438f9860e3cff5e713917ed22e0ac394a758c","after":"7747abdc151be4227a3885379403bfd7a268208b","ref":"refs/heads/main","pushedAt":"2024-05-13T19:46:28.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Disable PromoteL0 in crash test (#12651)\n\nSummary:\nSeeing way too many errors likely related to PromoteL0 from https://github.com/facebook/rocksdb/issues/12617, containing\n```\nCannot delete table file #N from level 0 since it is on level X\n```\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12651\n\nTest Plan: watch crash test results\n\nReviewed By: hx235\n\nDifferential Revision: D57286208\n\nPulled By: pdillinger\n\nfbshipit-source-id: f7f0560cc0804ca297373c8d20ebc34986cc19d0","shortMessageHtmlLink":"Disable PromoteL0 in crash test (#12651)"}},{"before":"7d9642d8761725cbc02a6bd6e8011a2865ec4d00","after":"b75438f9860e3cff5e713917ed22e0ac394a758c","ref":"refs/heads/main","pushedAt":"2024-05-11T00:59:40.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Allow disableWAL+recycle with WritePreparedTxnDB internals (#12639)\n\nSummary:\nFollow-up from https://github.com/facebook/rocksdb/issues/12403\n\nThe crash test was periodically failing with the\n\"disableWAL option is not supported if recycle_log_file_num > 0\" failure, despite not setting the disableWAL from the user side.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12639\n\nTest Plan: db_stress reproducer now passes. Added WAL recycling to txn DB unit tests, which is generally more difficult for correctness. Many tests now cover this change and pass.\n\nReviewed By: anand1976\n\nDifferential Revision: D57227617\n\nPulled By: pdillinger\n\nfbshipit-source-id: db9abefeb505bce624b45bc64009694d2a5baed9","shortMessageHtmlLink":"Allow disableWAL+recycle with WritePreparedTxnDB internals (#12639)"}},{"before":"b92d874c8b5a71ed611c2ca5912ed2b194e28c46","after":"7d9642d8761725cbc02a6bd6e8011a2865ec4d00","ref":"refs/heads/main","pushedAt":"2024-05-10T19:39:02.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Add logging for read timestamp during VerifyDB (#12638)\n\nSummary:\nAs titled. To help debug some verification failures.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12638\n\nTest Plan: manually tested\n\nReviewed By: ajkr\n\nDifferential Revision: D57219549\n\nPulled By: jowlyzhang\n\nfbshipit-source-id: 59c05ac85fb1c24449e7394ea04172c855d86420","shortMessageHtmlLink":"Add logging for read timestamp during VerifyDB (#12638)"}},{"before":"1f2715d1d2ff7573576887c5a4ed25493ed393e3","after":"b92d874c8b5a71ed611c2ca5912ed2b194e28c46","ref":"refs/heads/main","pushedAt":"2024-05-09T23:53:24.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Support MultiGetEntity in optimistic and WriteCommitted pessimistic transactions (#12634)\n\nSummary:\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12634\n\nThe patch implements support for the `MultiGetEntity` API in optimistic transactions and pessimistic transactions with the WriteCommitted policy. Similarly to the other wide-column transaction APIs, the implementation leverages the `WriteBatchWithIndex` layer.\n\nReviewed By: jaykorean\n\nDifferential Revision: D57177638\n\nfbshipit-source-id: 2d9f9f287fc97e7c126830b48d21457c7c35db3f","shortMessageHtmlLink":"Support MultiGetEntity in optimistic and WriteCommitted pessimistic t…"}},{"before":"9bddac0dcfb6ea7fc55e64f41a2670cbc23c4a2d","after":"1f2715d1d2ff7573576887c5a4ed25493ed393e3","ref":"refs/heads/main","pushedAt":"2024-05-09T23:44:50.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"AttributeGroup APIs in stress test - PutEntity and GetEntity (#12605)\n\nSummary:\nAdding AttributeGroup APIs in stress test. This contains the following changes only. More PRs to follow.\n\n- Introduce `use_attribute_group` flag\n- AttributeGroup `PutEntity()` and `GetEntity()` are now used per `use_attribute_group` flag in BatchOps, NonBatchOps and CfConsistency tests\n\nIn the next PRs I plan to add\n- AttributeGroup `MultiGetEntity()` in Stress Test\n- AttributeGroupIterator in Stress Test (along with CoalescingIterator)\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12605\n\nTest Plan:\nNonBatchOps\n```\npython3 tools/db_crashtest.py blackbox --simple --max_key=25000000 --write_buffer_size=4194304 --use_attribute_group=1 --use_put_entity_one_in=1\n```\n\nBatchOps\n```\npython3 tools/db_crashtest.py blackbox --test_batches_snapshots=1 --max_key=25000000 --write_buffer_size=4194304 --use_attribute_group=1 --use_put_entity_one_in=1\n```\n\nCfConsistency Test\n```\npython3 tools/db_crashtest.py blackbox --cf_consistency --max_key=25000000 --write_buffer_size=4194304 --use_attribute_group=1 --use_put_entity_one_in=1\n```\n\nReviewed By: ltamasi\n\nDifferential Revision: D56916768\n\nPulled By: jaykorean\n\nfbshipit-source-id: 8555d9e0d05927740a10e4e8301e44beec59a6f5","shortMessageHtmlLink":"AttributeGroup APIs in stress test - PutEntity and GetEntity (#12605)"}},{"before":"97e70906fab2bdc2878f5e05864165526dca79fe","after":"9bddac0dcfb6ea7fc55e64f41a2670cbc23c4a2d","ref":"refs/heads/main","pushedAt":"2024-05-09T22:41:02.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Add more public APIs to crash test (#12617)\n\nSummary:\n**Context/Summary:**\nAs titled. Bonus: found that PromoteL0 called with other concurrent PromoteL0 will return non-okay error so clarify the API.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12617\n\nTest Plan: CI\n\nReviewed By: ajkr\n\nDifferential Revision: D56954428\n\nPulled By: hx235\n\nfbshipit-source-id: 0e056153c515003fd241ffec59b0d8a27529db4c","shortMessageHtmlLink":"Add more public APIs to crash test (#12617)"}},{"before":"1a3357648fa2357097330e2abede82be1a749909","after":"97e70906fab2bdc2878f5e05864165526dca79fe","ref":"refs/heads/main","pushedAt":"2024-05-09T19:30:40.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Improve the sanity checks in (Multi)GetEntity and friends (#12630)\n\nSummary:\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12630\n\nThe patch cleans up, improves, and brings into sync (to the extent possible without API signature changes) the sanity checks around the `GetEntity` / `MultiGetEntity` family of APIs, including the read-your-own-writes (`WriteBatchWithIndex`) and transaction layers. The checks are centralized in two main sets of entry points, namely in `DB(Impl)` and the \"main\" `GetEntityFromBatchAndDB` / `MultiGetEntityFromBatchAndDB` overloads in `WriteBatchWithIndex`. This eliminates the need to duplicate the checks in the transaction classes.\n\nReviewed By: jaykorean\n\nDifferential Revision: D57125741\n\nfbshipit-source-id: 4dd059ef644a9b173fbba767538943397e4cc6cd","shortMessageHtmlLink":"Improve the sanity checks in (Multi)GetEntity and friends (#12630)"}},{"before":"9dc171e3bb8443f004d54ebac832dbf6ba4511a7","after":"1a3357648fa2357097330e2abede82be1a749909","ref":"refs/heads/main","pushedAt":"2024-05-09T18:40:59.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Error log update to db_impl_compaction_flush.cc (#12608)\n\nSummary:\nMake the error looks better.\n\nSome inconsistency here and [here](https://github.com/WweiL/rocksdb/blob/e46ab9d4f0a0e63bfc668421e2994efa918d6570/db/db_impl/db_impl_compaction_flush.cc#L2701-L2702)\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12608\n\nReviewed By: ajkr\n\nDifferential Revision: D57134933\n\nPulled By: cbi42\n\nfbshipit-source-id: 2f19f077f388d196652a4e3afd2526f18bf75b2d","shortMessageHtmlLink":"Error log update to db_impl_compaction_flush.cc (#12608)"}},{"before":"a8e954024fb086cddb7c3262eff461ba32dc5ee7","after":"08f93221f50700f19f11555fb46abfe708a716d1","ref":"refs/heads/9.2.fb","pushedAt":"2024-05-08T23:03:34.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"jowlyzhang","name":"Yu Zhang","path":"/jowlyzhang","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/5846404?s=80&v=4"},"commit":{"message":"Fix issue that cause false alarm corruption report (#12626)\n\nSummary:\nThe state of `saved_seq_for_penul_check_` is not correctly maintained with the current flow. It's supposed to store the original sequence number for a `kTypeValuePreferredSeqno` entry for use in the `DecideOutputLevel` function. However, it's not always properly cleared.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12626\n\nTest Plan:\nAdded unit test that would fail before the fix\n./tiered_compaction_test --gtest_filter=\"*InterleavedTimedPutAndPut*\"\n\nReviewed By: pdillinger\n\nDifferential Revision: D57123469\n\nPulled By: jowlyzhang\n\nfbshipit-source-id: 8d73214b3b6dc152daf19b6bd6ee9063581dc277","shortMessageHtmlLink":"Fix issue that cause false alarm corruption report (#12626)"}},{"before":"eeda54fe63c0c05a4aa3938de32874269a26e2c9","after":"9dc171e3bb8443f004d54ebac832dbf6ba4511a7","ref":"refs/heads/main","pushedAt":"2024-05-08T22:55:43.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Fix issue that cause false alarm corruption report (#12626)\n\nSummary:\nThe state of `saved_seq_for_penul_check_` is not correctly maintained with the current flow. It's supposed to store the original sequence number for a `kTypeValuePreferredSeqno` entry for use in the `DecideOutputLevel` function. However, it's not always properly cleared.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12626\n\nTest Plan:\nAdded unit test that would fail before the fix\n./tiered_compaction_test --gtest_filter=\"*InterleavedTimedPutAndPut*\"\n\nReviewed By: pdillinger\n\nDifferential Revision: D57123469\n\nPulled By: jowlyzhang\n\nfbshipit-source-id: 8d73214b3b6dc152daf19b6bd6ee9063581dc277","shortMessageHtmlLink":"Fix issue that cause false alarm corruption report (#12626)"}},{"before":"933ac0e05ce8c14847d0c67160c3c5869eae9656","after":"eeda54fe63c0c05a4aa3938de32874269a26e2c9","ref":"refs/heads/main","pushedAt":"2024-05-08T20:19:55.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Fix db_crashtest.py for prefixpercent and enable_compaction_filter (#12627)\n\nSummary:\nAfter https://github.com/facebook/rocksdb/issues/12624 seeing db_stress failures due to db_crashtest.py calling it with --prefixpercent=5 --enable_compaction_filter=1\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12627\n\nTest Plan: watch crash test\n\nReviewed By: ajkr\n\nDifferential Revision: D57121592\n\nPulled By: pdillinger\n\nfbshipit-source-id: 55727355a7662e67efcd22d7e353153e78e24f59","shortMessageHtmlLink":"Fix db_crashtest.py for prefixpercent and enable_compaction_filter (#…"}},{"before":"b8400c9fafc1efdb8b75a8459fc725225f953f95","after":"933ac0e05ce8c14847d0c67160c3c5869eae9656","ref":"refs/heads/main","pushedAt":"2024-05-08T15:35:31.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Fix locking for `ColumnFamilyOptions::inplace_update_support` (#12624)\n\nSummary:\nIn `SaveValue()`, the read lock needs to be obtained before `VerifyEntryChecksum()` because the KV checksum verification reads the entire value metadata+data, which is all mutable when `ColumnFamilyOptions::inplace_update_support == true`.\n\nIn `MemTable::Update()`, the write lock needs to be obtained before mutating the value metadata (changing the value size) because it can be read concurrently.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12624\n\nTest Plan:\n```\n$ make COMPILE_WITH_TSAN=1 -j56 db_stress\n...\n$ python3 tools/db_crashtest.py blackbox --simple --max_key=10 --inplace_update_support=1 --interval=10 --allow_concurrent_memtable_write=0\n```\n\nReviewed By: cbi42\n\nDifferential Revision: D57034571\n\nPulled By: ajkr\n\nfbshipit-source-id: 3dddf881ad87923143acdf6bfec12ce47bb13a48","shortMessageHtmlLink":"Fix locking for ColumnFamilyOptions::inplace_update_support (#12624)"}},{"before":"5bf2c00a35debbd59840814090ce4e3b7f007798","after":"b8400c9fafc1efdb8b75a8459fc725225f953f95","ref":"refs/heads/main","pushedAt":"2024-05-08T00:58:28.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Make linux file write life time hinting work (#12595)\n\nSummary:\nThe life time hint fcntl takes a 64-bit unsigned int, so make sure to pass a uint64_t when doing the syscall.\n\nSee:\n\nhttps://man7.org/linux/man-pages/man2/fcntl.2.html\nhttps://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c75b1d9421f80f4143e389d2d50ddfc8a28c8c35\n\nThis is one of those \"How did this ever work?\", as Env::WriteLifeTimeHint hint is definitely not the same as an 64-bit unsigned int.\nWhat's surprising is that SetWriteLifeTimeHint does pass a valid hint from time to time.\n\nThanks,\nHans\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12595\n\nReviewed By: cbi42\n\nDifferential Revision: D56901280\n\nPulled By: ajkr\n\nfbshipit-source-id: f276348863cbc29a537bed9450b16b0cc513ea78","shortMessageHtmlLink":"Make linux file write life time hinting work (#12595)"}},{"before":"83d051a8d921a038df6e256ea0fa0ed0055b7c83","after":"5bf2c00a35debbd59840814090ce4e3b7f007798","ref":"refs/heads/main","pushedAt":"2024-05-07T19:06:20.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Clarify manual compaction and file ingestion behavior with FIFO compaction (#12618)\n\nSummary:\nFor manual compaction, FIFO compaction will always skip key range overlapping checking with SST files. If CompactRange() is called with CompactionRangeOptions::change_level=true, a CF with FIFO compaction will now return Status::NotSupported.\n\nFor file ingestion, we will always ingest into L0. Previously, it's possible to ingest files into non-L0 levels with FIFO compaction.\n\nThese changes also help to fix [this](https://github.com/facebook/rocksdb/blob/a178d15bafae1c9ea51f19691b2d1fb9dd3b6a3f/db/db_impl/db_impl_compaction_flush.cc#L1269) assertion failure in crash tests.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12618\n\nTest Plan: added unit tests to verify the new behavior.\n\nReviewed By: hx235\n\nDifferential Revision: D56962401\n\nPulled By: cbi42\n\nfbshipit-source-id: 19812a1509650b4162b379ca5bee02f2e9d9569d","shortMessageHtmlLink":"Clarify manual compaction and file ingestion behavior with FIFO compa…"}},{"before":"eaa3226ef769e9c412b5885f9cdc45a85526e024","after":"83d051a8d921a038df6e256ea0fa0ed0055b7c83","ref":"refs/heads/main","pushedAt":"2024-05-07T18:42:36.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Add release note for GetEntity transaction support (#12625)\n\nSummary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/12625\n\nReviewed By: jaykorean\n\nDifferential Revision: D57059775\n\nfbshipit-source-id: 80b3ddb51d538c6c21b69cd589f4ee8dd13596c9","shortMessageHtmlLink":"Add release note for GetEntity transaction support (#12625)"}},{"before":"36ab251c07f9feaafaecf62de854283e0c580619","after":"eaa3226ef769e9c412b5885f9cdc45a85526e024","ref":"refs/heads/main","pushedAt":"2024-05-07T17:24:53.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Add support for GetEntity in optimistic and WriteCommitted pessimistic transactions (#12623)\n\nSummary:\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12623\n\nThe PR adds support for the `GetEntity` API to optimistic and WriteCommitted pessimistic transactions. `MultiGetEntity` support and the `ForUpdate` variants of these read APIs will be implemented in subsequent PRs.\n\nReviewed By: jaykorean\n\nDifferential Revision: D57030879\n\nfbshipit-source-id: 1f0aed6418782975fe537b6b3d437fad31fcbd43","shortMessageHtmlLink":"Add support for GetEntity in optimistic and WriteCommitted pessimisti…"}},{"before":"45c290660a2de85260d404a2c71e07fda0b33ce6","after":"36ab251c07f9feaafaecf62de854283e0c580619","ref":"refs/heads/main","pushedAt":"2024-05-06T23:53:00.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Expose block based metadata cache options via C API (#12611)\n\nSummary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/12611\n\nReviewed By: jaykorean\n\nDifferential Revision: D56961823\n\nPulled By: ajkr\n\nfbshipit-source-id: aa062cdb49a0bb2c1148a81d4c882a4733c7790e","shortMessageHtmlLink":"Expose block based metadata cache options via C API (#12611)"}},{"before":"0fef690bd50ecb8a92dad5e9865376e73b246570","after":"45c290660a2de85260d404a2c71e07fda0b33ce6","ref":"refs/heads/main","pushedAt":"2024-05-06T21:45:21.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Add PutEntity support for optimistic and WritePrepared pessimistic transactions (#12606)\n\nSummary:\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12606\n\nThe patch extends optimistic transactions and WriteCommitted pessimistic transactions with support for the `PutEntity` API. Similarly to the other APIs, `PutEntity` is available via both the `Transaction` and `TransactionDB` interfaces, where using the latter executes the write in a single-operation transaction as usual. Support for read APIs and other write policies (WritePrepared, WriteUnprepared) will be added in separate PRs.\n\nReviewed By: jaykorean\n\nDifferential Revision: D56911242\n\nfbshipit-source-id: 57cf8bb6c6b1b40ba4a8a652831c13a617644289","shortMessageHtmlLink":"Add PutEntity support for optimistic and WritePrepared pessimistic tr…"}},{"before":"6fdc4c52823d0b32bb18321b0d4b14ab70d09e92","after":"0fef690bd50ecb8a92dad5e9865376e73b246570","ref":"refs/heads/main","pushedAt":"2024-05-06T18:59:41.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Sync non-latest WALs during flush for 2PC, single-CF DBs (#12622)\n\nSummary:\nPreviously we skipped syncing the non-latest WALs during memtable flush when the DB had only one column family. Normally that is fine because those non-latest WALs would not be read by recovery. However, in case of `DBOptions::allow_2pc == true`, there could be unmatched prepare records in those WALs making them needed by recovery. As a result, the missing sync could have resulted in the recovered WAL state falling behind the recovered SST state. When we detect that case, we return a `Status::Corruption` saying \"SST file is ahead of WALs\".\n\nThis PR proposes syncing the WAL in case of `DBOptions::allow_2pc`. This introduces the sync in some scenarios where it isn't needed (e.g., non-recent WALs contain no prepares) but I suspect the simplicity is worth it.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12622\n\nReviewed By: cbi42\n\nDifferential Revision: D56987303\n\nPulled By: ajkr\n\nfbshipit-source-id: 7fe9395458018a18d77e907a3b5429065c0e2e48","shortMessageHtmlLink":"Sync non-latest WALs during flush for 2PC, single-CF DBs (#12622)"}},{"before":"3fdc7243f3c72ac191a56cfc16b4262b86c3fcb7","after":"6fdc4c52823d0b32bb18321b0d4b14ab70d09e92","ref":"refs/heads/main","pushedAt":"2024-05-06T18:07:03.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Fix a corruption bug in `CreateColumnFamilyWithImport()` (#12602)\n\nSummary:\nwhen importing files from multiple CFs into a new CF, we were reusing the epoch numbers assigned by the original CFs. This means L0 files in the new CF can have the same epoch number (assigned originally by different CFs). While CreateColumnFamilyWithImport() requires each original CF to have disjoint key range, after an intra-l0 compaction, we still can end up with L0 files with the same epoch number but overlapping key range. This PR attempt to fix this by reassigning epoch numbers when importing multiple CFs.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12602\n\nTest Plan:\na new repro unit test. Before this PR, it fails with\n```\n[ RUN ] ImportColumnFamilyTest.AssignEpochNumberToMultipleCF\ndb/import_column_family_test.cc:1048: Failure\ndb_->WaitForCompact(o)\nCorruption: force_consistency_checks(DEBUG): VersionBuilder: L0 files of same epoch number but overlapping range https://github.com/facebook/rocksdb/issues/44 , smallest key: '6B6579303030303030' seq:511, type:1 , largest key: '6B6579303031303239' seq:510, type:1 , epoch number: 3 vs. file https://github.com/facebook/rocksdb/issues/36 , smallest key: '6B6579303030313030' seq:401, type:1 , largest key: '6B6579303030313939' seq:500, type:1 , epoch number: 3\n```\n\nReviewed By: hx235\n\nDifferential Revision: D56851808\n\nPulled By: cbi42\n\nfbshipit-source-id: 01b8c790c9f1f2a168047ead670e73633f705b84","shortMessageHtmlLink":"Fix a corruption bug in CreateColumnFamilyWithImport() (#12602)"}},{"before":"7bf6d4c9d50a5760deaf2b264175d8eaf5ecfb09","after":"3fdc7243f3c72ac191a56cfc16b4262b86c3fcb7","ref":"refs/heads/main","pushedAt":"2024-05-06T15:57:44.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Fix truncating last character in the StderrLogger (#12620)\n\nSummary:\nThis PR fixes a bug in the StderrLogger that truncated the last character in the logline. The problem was that we provided an incorrect max size parameter into the vsnprintf function. The size didn't take into account the null byte that the function automatically adds.\n\nBefore fix\n```\n** File Read Latency Histogram By Level [default] **\n2024/05/04-18:50:24.209304 4788 [/db_impl/db_impl.cc:498] Shutdown: canceling all background wor\n2024/05/04-18:50:24.209598 4788 [/db_impl/db_impl.cc:692] Shutdown complet\n```\n\nAfter fix\n```\n** File Read Latency Histogram By Level [default] **\n\n2024/05/04-18:51:19.814584 4d4d [/db_impl/db_impl.cc:498] Shutdown: canceling all background work\n2024/05/04-18:51:19.815528 4d4d [/db_impl/db_impl.cc:692] Shutdown complete\n```\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12620\n\nTest Plan:\ntested on examples/simple_example.cc with StderrLogger\nFixes: https://github.com/facebook/rocksdb/issues/12576\n\nReviewed By: jaykorean\n\nDifferential Revision: D56972332\n\nPulled By: ajkr\n\nfbshipit-source-id: 70405e8231ae6e90d24fe0b351bc8e749176bd15","shortMessageHtmlLink":"Fix truncating last character in the StderrLogger (#12620)"}},{"before":"a178d15bafae1c9ea51f19691b2d1fb9dd3b6a3f","after":"7bf6d4c9d50a5760deaf2b264175d8eaf5ecfb09","ref":"refs/heads/main","pushedAt":"2024-05-04T00:24:44.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Lazily construct `BlockBasedTableIterator::block_handles_` (#12616)\n\nSummary:\nOur external benchmark attributed a CPU regression to https://github.com/facebook/rocksdb/issues/11860. Based on the CPU profile the new overhead is from `std::deque`. The deque is always empty for these scans so we do not need to construct it. This PR lazily constructs it only when it is needed.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12616\n\nTest Plan:\n- Command: `TEST_TMPDIR=/dev/shm ./db_bench -benchmarks=filluniquerandom,seekrandom[-X10] -compression_type=none -disable_auto_compactions=true -write_buffer_size=524288 -value_size=1024 -num=10000 -reads=100000`\n- Results\n - Before this PR: `seekrandom [AVG 10 runs] : 47811 (± 431) ops/sec`\n - After this PR: `seekrandom [AVG 10 runs] : 51013 (± 632) ops/sec`\n\nReviewed By: jaykorean\n\nDifferential Revision: D56954136\n\nPulled By: ajkr\n\nfbshipit-source-id: b4d34c9b6c6c2e83d4fff06deacb9f0df2ad042f","shortMessageHtmlLink":"Lazily construct BlockBasedTableIterator::block_handles_ (#12616)"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAESzE1VQA","startCursor":null,"endCursor":null}},"title":"Activity · facebook/rocksdb"}