{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":156018,"defaultBranch":"unstable","name":"redis","ownerLogin":"redis","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2009-03-21T22:32:25.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/1529926?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1713194357.0","currentOid":""},"activityList":{"items":[{"before":"c33c91dbcea21d895bcda7ca5756c24f86664b22","after":"03cd525ffaa301822ebdf96e95e0ffe19474f07d","ref":"refs/heads/hash-field-expiry-integ","pushedAt":"2024-05-03T03:11:42.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"sundb","name":"debing.sun","path":"/sundb","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/965798?s=80&v=4"},"commit":{"message":"Fix reply schema for hfe related commands (#13238)","shortMessageHtmlLink":"Fix reply schema for hfe related commands (#13238)"}},{"before":"c18ff05665cb195190fbbd37235e15b2b86ebb63","after":"c33c91dbcea21d895bcda7ca5756c24f86664b22","ref":"refs/heads/hash-field-expiry-integ","pushedAt":"2024-04-25T15:29:03.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"moticless","name":"Moti Cohen","path":"/moticless","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/24944278?s=80&v=4"},"commit":{"message":"Support HSET+expire in one command, at infra level (#13230)\n\nUnify infra of `HSETF`, `HEXPIRE`, `HSET` and provide API for RDB load\r\nas well. Whereas setting plain fields is rather straightforward, setting\r\nexpiration time to fields might be time-consuming and complex since each \r\nupdate of expiration time, not only updates `ebuckets` of corresponding hash, \r\nbut also might update `ebuckets` of global HFE DS. It is required to opt \r\nsequence of field updates with expirartion for a given hash, such that only once\r\ndone, the global HFE DS will get updated.\r\n\r\nTo do so, follow the scheme:\r\n1. Call `hashTypeSetExInit()` to initialize the HashTypeSetEx struct.\r\n2. Call `hashTypeSetEx()` one time or more, for each field/expiration update.\r\n3. Call `hashTypeSetExDone()` for notification and update of global HFE.\r\n\r\nIf expiration is not required, then avoid this API and use instead hashTypeSet().","shortMessageHtmlLink":"Support HSET+expire in one command, at infra level (#13230)"}},{"before":"772564fc9e7a4415c79ef47486db1aacf9cb8915","after":"f95031c4733078788063de775c968b6dc85522c0","ref":"refs/heads/unstable","pushedAt":"2024-04-25T06:11:45.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"sundb","name":"debing.sun","path":"/sundb","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/965798?s=80&v=4"},"commit":{"message":"Fix CI failure caused by PR #13231 (#13233)\n\nFor my mistake, in the last revert commit in #13231, I originally wanted\r\nto revert the last one, but reverted the penultimate fix.\r\nNow that we have fix another potential memory read issue in [`743f1dd`\r\n(#13231)](https://github.com/redis/redis/pull/13231/commits/743f1dde79b433fdb8ea13de4fd73457d4fe25eb),\r\nnow it just seems to avoid confusion, i will verify in the future\r\nwhether it will have any impact, if so we will add this PR to backport.\r\n\r\nFailed CI: https://github.com/sundb/redis/actions/runs/8826731960","shortMessageHtmlLink":"Fix CI failure caused by PR #13231 (#13233)"}},{"before":"804110a487f048669aa9d9412e5789ec43f4fe39","after":"772564fc9e7a4415c79ef47486db1aacf9cb8915","ref":"refs/heads/unstable","pushedAt":"2024-04-24T08:15:42.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"sundb","name":"debing.sun","path":"/sundb","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/965798?s=80&v=4"},"commit":{"message":"Fix forget to update the dict's node in the kvstore's rehashing list after defragment (#13231)\n\nIntroducted by #13013\r\n\r\nAfter defragmenting the dictionary in the kvstore, if the dict is\r\nreallocated, the value of its node in the kvstore rehashing list must be\r\nupdated.","shortMessageHtmlLink":"Fix forget to update the dict's node in the kvstore's rehashing list …"}},{"before":"4581d43230fab23600b90f731edb0472dbea1c4d","after":"c18ff05665cb195190fbbd37235e15b2b86ebb63","ref":"refs/heads/hash-field-expiry-integ","pushedAt":"2024-04-18T13:06:30.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"moticless","name":"Moti Cohen","path":"/moticless","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/24944278?s=80&v=4"},"commit":{"message":"Hash Field Expiration - Basic support\n\n- Add ebuckets & mstr data structures\r\n- Integrate active & lazy expiration\r\n- Add most of the commands \r\n- Add support for dict (listpack is missing)\r\nTODOs: RDB, notification, listpack, HSET, HGETF, defrag, aof","shortMessageHtmlLink":"Hash Field Expiration - Basic support"}},{"before":"e3550f01dde29d5d1eaa37dbb4533692c5680f06","after":"804110a487f048669aa9d9412e5789ec43f4fe39","ref":"refs/heads/unstable","pushedAt":"2024-04-16T09:43:34.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"oranagra","name":"Oran Agra","path":"/oranagra","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/7045099?s=80&v=4"},"commit":{"message":"Allocate Lua VM code with jemalloc instead of libc, and count it used memory (#13133)\n\n## Background\r\n1. Currently Lua memory control does not pass through Redis's zmalloc.c.\r\nRedis maxmemory cannot limit memory problems caused by users abusing lua\r\nsince these lua VM memory is not part of used_memory.\r\n\r\n2. Since jemalloc is much better (fragmentation and speed), and also we\r\nknow it and trust it. we are\r\ngoing to use jemalloc instead of libc to allocate the Lua VM code and\r\ncount it used memory.\r\n\r\n## Process:\r\nIn this PR, we will use jemalloc in lua. \r\n1. Create an arena for all lua vm (script and function), which is\r\nshared, in order to avoid blocking defragger.\r\n2. Create a bound tcache for the lua VM, since the lua VM and the main\r\nthread are by default in the same tcache, and if there is no isolated\r\ntcache, lua may request memory from the tcache which has just been freed\r\nby main thread, and vice versa\r\nOn the other hand, since lua vm might be release in bio thread, but\r\ntcache is not thread-safe, we need to recreate\r\n the tcache every time we recreate the lua vm.\r\n3. Remove lua memory statistics from memory fragmentation statistics to\r\navoid the effects of lua memory fragmentation\r\n\r\n## Other\r\nAdd the following new fields to `INFO DEBUG` (we may promote them to\r\nINFO MEMORY some day)\r\n1. allocator_allocated_lua: total number of bytes allocated of lua arena\r\n2. allocator_active_lua: total number of bytes in active pages allocated\r\nin lua arena\r\n3. allocator_resident_lua: maximum number of bytes in physically\r\nresident data pages mapped in lua arena\r\n4. allocator_frag_bytes_lua: fragment bytes in lua arena\r\n\r\nThis is oranagra's idea, and i got some help from sundb.\r\n\r\nThis solves the third point in #13102.\r\n\r\n---------\r\n\r\nCo-authored-by: debing.sun \r\nCo-authored-by: Oran Agra ","shortMessageHtmlLink":"Allocate Lua VM code with jemalloc instead of libc, and count it used…"}},{"before":"aada46a0c8e94c1716c308562ae53484a2e06e02","after":null,"ref":"refs/heads/dependabot/github_actions/cross-platform-actions/action-0.23.0","pushedAt":"2024-04-15T15:19:17.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"dependabot[bot]","name":null,"path":"/apps/dependabot","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/29110?s=80&v=4"}},{"before":null,"after":"6b0245b50c9d18c92cf5676ff396eef0dcf2946b","ref":"refs/heads/dependabot/github_actions/cross-platform-actions/action-0.24.0","pushedAt":"2024-04-15T15:19:10.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"dependabot[bot]","name":null,"path":"/apps/dependabot","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/29110?s=80&v=4"},"commit":{"message":"Bump cross-platform-actions/action from 0.22.0 to 0.24.0\n\nBumps [cross-platform-actions/action](https://github.com/cross-platform-actions/action) from 0.22.0 to 0.24.0.\n- [Release notes](https://github.com/cross-platform-actions/action/releases)\n- [Changelog](https://github.com/cross-platform-actions/action/blob/master/changelog.md)\n- [Commits](https://github.com/cross-platform-actions/action/compare/v0.22.0...v0.24.0)\n\n---\nupdated-dependencies:\n- dependency-name: cross-platform-actions/action\n dependency-type: direct:production\n update-type: version-update:semver-minor\n...\n\nSigned-off-by: dependabot[bot] ","shortMessageHtmlLink":"Bump cross-platform-actions/action from 0.22.0 to 0.24.0"}},{"before":"f4481e657f905074fa515701af3f695757817d88","after":"e3550f01dde29d5d1eaa37dbb4533692c5680f06","ref":"refs/heads/unstable","pushedAt":"2024-04-08T08:12:57.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"oranagra","name":"Oran Agra","path":"/oranagra","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/7045099?s=80&v=4"},"commit":{"message":"redis-cli - sendReadOnly() to work with Redis Cloud (#13195)\n\nWhen using Redis Cloud, sendReadOnly() exit with `Error: ERR unknown\r\ncommand 'READONLY'`.\r\nIt is impacting `--memkeys`, `--bigkeys`, `--hotkeys`, and will impact\r\n`--keystats`.\r\nAdded one line to ignore this error.\r\n\r\nissue introduced in #12735 (not yet released).","shortMessageHtmlLink":"redis-cli - sendReadOnly() to work with Redis Cloud (#13195)"}},{"before":"4581d43230fab23600b90f731edb0472dbea1c4d","after":"f4481e657f905074fa515701af3f695757817d88","ref":"refs/heads/unstable","pushedAt":"2024-04-07T12:59:36.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"sundb","name":"debing.sun","path":"/sundb","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/965798?s=80&v=4"},"commit":{"message":"Use usleep() instead of sched_yield() to yield cpu (#13183)\n\nwhen the main thread and the module thread are in the same thread,\r\nsched_yield() can work well.\r\nwhen they are both bind to different cpus, sched_yield() will look for\r\nthe thread with the highest priority, and if the module thread is always\r\nthe highest priority on a cpu, it will take a long time to let the main\r\nthread to reacquire the GIL.\r\n\r\nref https://man7.org/linux/man-pages/man2/sched_yield.2.html\r\n```\r\nIf the calling thread is the only thread in the highest priority\r\nlist at that time, it will continue to run after a call to\r\nsched_yield().\r\n```","shortMessageHtmlLink":"Use usleep() instead of sched_yield() to yield cpu (#13183)"}},{"before":null,"after":"4581d43230fab23600b90f731edb0472dbea1c4d","ref":"refs/heads/hash-field-expiry-integ","pushedAt":"2024-04-07T11:40:36.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"moticless","name":"Moti Cohen","path":"/moticless","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/24944278?s=80&v=4"},"commit":{"message":"Fix daylight race condition and some thread leaks (#13191)\n\nfix some issues that come from sanitizer thread report.\r\n\r\n1. when the main thread is updating daylight_active, other threads (bio,\r\nmodule thread) may be writing logs at the same time.\r\n```\r\nWARNING: ThreadSanitizer: data race (pid=661064)\r\n Read of size 4 at 0x55c9a4d11c70 by thread T2:\r\n #0 serverLogRaw /home/sundb/data/redis_fork/src/server.c:116 (redis-server+0x8d797) (BuildId: dca0b1945ba30010e36129bdb296e488dd2b32d0)\r\n #1 _serverLog.constprop.2 /home/sundb/data/redis_fork/src/server.c:146 (redis-server+0x2a3b14) (BuildId: dca0b1945ba30010e36129bdb296e488dd2b32d0)\r\n #2 bioProcessBackgroundJobs /home/sundb/data/redis_fork/src/bio.c:329 (redis-server+0x1c24ca) (BuildId: dca0b1945ba30010e36129bdb296e488dd2b32d0)\r\n\r\n Previous write of size 4 at 0x55c9a4d11c70 by main thread (mutexes: write M0, write M1, write M2, write M3):\r\n #0 updateCachedTimeWithUs /home/sundb/data/redis_fork/src/server.c:1102 (redis-server+0x925e7) (BuildId: dca0b1945ba30010e36129bdb296e488dd2b32d0)\r\n #1 updateCachedTimeWithUs /home/sundb/data/redis_fork/src/server.c:1087 (redis-server+0x925e7)\r\n #2 updateCachedTime /home/sundb/data/redis_fork/src/server.c:1118 (redis-server+0x925e7)\r\n #3 afterSleep /home/sundb/data/redis_fork/src/server.c:1811 (redis-server+0x925e7)\r\n #4 aeProcessEvents /home/sundb/data/redis_fork/src/ae.c:389 (redis-server+0x85ae0) (BuildId: dca0b1945ba30010e36129bdb296e488dd2b32d0)\r\n #5 aeProcessEvents /home/sundb/data/redis_fork/src/ae.c:342 (redis-server+0x85ae0)\r\n #6 aeMain /home/sundb/data/redis_fork/src/ae.c:477 (redis-server+0x85ae0)\r\n #7 main /home/sundb/data/redis_fork/src/server.c:7211 (redis-server+0x7168c) (BuildId: dca0b1945ba30010e36129bdb296e488dd2b32d0)\r\n```\r\n\r\n2. thread leaks in module tests\r\n```\r\nWARNING: ThreadSanitizer: thread leak (pid=668683)\r\n Thread T13 (tid=670041, finished) created by main thread at:\r\n #0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:1036 (libtsan.so.2+0x3d179) (BuildId: 28a9f70061dbb2dfa2cef661d3b23aff4ea13536)\r\n #1 HelloBlockNoTracking_RedisCommand /home/sundb/data/redis_fork/tests/modules/blockonbackground.c:200 (blockonbackground.so+0x97fd) (BuildId: 9cd187906c57e88cdf896d121d1d96448b37a136)\r\n #2 HelloBlockNoTracking_RedisCommand /home/sundb/data/redis_fork/tests/modules/blockonbackground.c:169 (blockonbackground.so+0x97fd)\r\n #3 call /home/sundb/data/redis_fork/src/server.c:3546 (redis-server+0x9b7fb) (BuildId: dca0b1945ba30010e36129bdb296e488dd2b32d0)\r\n #4 processCommand /home/sundb/data/redis_fork/src/server.c:4176 (redis-server+0xa091c) (BuildId: dca0b1945ba30010e36129bdb296e488dd2b32d0)\r\n #5 processCommandAndResetClient /home/sundb/data/redis_fork/src/networking.c:2468 (redis-server+0xd2b8e) (BuildId: dca0b1945ba30010e36129bdb296e488dd2b32d0)\r\n #6 processInputBuffer /home/sundb/data/redis_fork/src/networking.c:2576 (redis-server+0xd2b8e)\r\n #7 readQueryFromClient /home/sundb/data/redis_fork/src/networking.c:2722 (redis-server+0xd358f) (BuildId: dca0b1945ba30010e36129bdb296e488dd2b32d0)\r\n #8 callHandler /home/sundb/data/redis_fork/src/connhelpers.h:58 (redis-server+0x288a7b) (BuildId: dca0b1945ba30010e36129bdb296e488dd2b32d0)\r\n #9 connSocketEventHandler /home/sundb/data/redis_fork/src/socket.c:277 (redis-server+0x288a7b)\r\n #10 aeProcessEvents /home/sundb/data/redis_fork/src/ae.c:417 (redis-server+0x85b45) (BuildId: dca0b1945ba30010e36129bdb296e488dd2b32d0)\r\n #11 aeProcessEvents /home/sundb/data/redis_fork/src/ae.c:342 (redis-server+0x85b45)\r\n #12 aeMain /home/sundb/data/redis_fork/src/ae.c:477 (redis-server+0x85b45)\r\n #13 main /home/sundb/data/redis_fork/src/server.c:7211 (redis-server+0x7168c) (BuildId: dca0b1945ba30010e36129bdb296e488dd2b32d0)\r\n```","shortMessageHtmlLink":"Fix daylight race condition and some thread leaks (#13191)"}},{"before":"4df037962dd446a4a686e2b6d56d5367b6c9f0db","after":"4581d43230fab23600b90f731edb0472dbea1c4d","ref":"refs/heads/unstable","pushedAt":"2024-04-04T10:49:51.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"oranagra","name":"Oran Agra","path":"/oranagra","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/7045099?s=80&v=4"},"commit":{"message":"Fix daylight race condition and some thread leaks (#13191)\n\nfix some issues that come from sanitizer thread report.\r\n\r\n1. when the main thread is updating daylight_active, other threads (bio,\r\nmodule thread) may be writing logs at the same time.\r\n```\r\nWARNING: ThreadSanitizer: data race (pid=661064)\r\n Read of size 4 at 0x55c9a4d11c70 by thread T2:\r\n #0 serverLogRaw /home/sundb/data/redis_fork/src/server.c:116 (redis-server+0x8d797) (BuildId: dca0b1945ba30010e36129bdb296e488dd2b32d0)\r\n #1 _serverLog.constprop.2 /home/sundb/data/redis_fork/src/server.c:146 (redis-server+0x2a3b14) (BuildId: dca0b1945ba30010e36129bdb296e488dd2b32d0)\r\n #2 bioProcessBackgroundJobs /home/sundb/data/redis_fork/src/bio.c:329 (redis-server+0x1c24ca) (BuildId: dca0b1945ba30010e36129bdb296e488dd2b32d0)\r\n\r\n Previous write of size 4 at 0x55c9a4d11c70 by main thread (mutexes: write M0, write M1, write M2, write M3):\r\n #0 updateCachedTimeWithUs /home/sundb/data/redis_fork/src/server.c:1102 (redis-server+0x925e7) (BuildId: dca0b1945ba30010e36129bdb296e488dd2b32d0)\r\n #1 updateCachedTimeWithUs /home/sundb/data/redis_fork/src/server.c:1087 (redis-server+0x925e7)\r\n #2 updateCachedTime /home/sundb/data/redis_fork/src/server.c:1118 (redis-server+0x925e7)\r\n #3 afterSleep /home/sundb/data/redis_fork/src/server.c:1811 (redis-server+0x925e7)\r\n #4 aeProcessEvents /home/sundb/data/redis_fork/src/ae.c:389 (redis-server+0x85ae0) (BuildId: dca0b1945ba30010e36129bdb296e488dd2b32d0)\r\n #5 aeProcessEvents /home/sundb/data/redis_fork/src/ae.c:342 (redis-server+0x85ae0)\r\n #6 aeMain /home/sundb/data/redis_fork/src/ae.c:477 (redis-server+0x85ae0)\r\n #7 main /home/sundb/data/redis_fork/src/server.c:7211 (redis-server+0x7168c) (BuildId: dca0b1945ba30010e36129bdb296e488dd2b32d0)\r\n```\r\n\r\n2. thread leaks in module tests\r\n```\r\nWARNING: ThreadSanitizer: thread leak (pid=668683)\r\n Thread T13 (tid=670041, finished) created by main thread at:\r\n #0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:1036 (libtsan.so.2+0x3d179) (BuildId: 28a9f70061dbb2dfa2cef661d3b23aff4ea13536)\r\n #1 HelloBlockNoTracking_RedisCommand /home/sundb/data/redis_fork/tests/modules/blockonbackground.c:200 (blockonbackground.so+0x97fd) (BuildId: 9cd187906c57e88cdf896d121d1d96448b37a136)\r\n #2 HelloBlockNoTracking_RedisCommand /home/sundb/data/redis_fork/tests/modules/blockonbackground.c:169 (blockonbackground.so+0x97fd)\r\n #3 call /home/sundb/data/redis_fork/src/server.c:3546 (redis-server+0x9b7fb) (BuildId: dca0b1945ba30010e36129bdb296e488dd2b32d0)\r\n #4 processCommand /home/sundb/data/redis_fork/src/server.c:4176 (redis-server+0xa091c) (BuildId: dca0b1945ba30010e36129bdb296e488dd2b32d0)\r\n #5 processCommandAndResetClient /home/sundb/data/redis_fork/src/networking.c:2468 (redis-server+0xd2b8e) (BuildId: dca0b1945ba30010e36129bdb296e488dd2b32d0)\r\n #6 processInputBuffer /home/sundb/data/redis_fork/src/networking.c:2576 (redis-server+0xd2b8e)\r\n #7 readQueryFromClient /home/sundb/data/redis_fork/src/networking.c:2722 (redis-server+0xd358f) (BuildId: dca0b1945ba30010e36129bdb296e488dd2b32d0)\r\n #8 callHandler /home/sundb/data/redis_fork/src/connhelpers.h:58 (redis-server+0x288a7b) (BuildId: dca0b1945ba30010e36129bdb296e488dd2b32d0)\r\n #9 connSocketEventHandler /home/sundb/data/redis_fork/src/socket.c:277 (redis-server+0x288a7b)\r\n #10 aeProcessEvents /home/sundb/data/redis_fork/src/ae.c:417 (redis-server+0x85b45) (BuildId: dca0b1945ba30010e36129bdb296e488dd2b32d0)\r\n #11 aeProcessEvents /home/sundb/data/redis_fork/src/ae.c:342 (redis-server+0x85b45)\r\n #12 aeMain /home/sundb/data/redis_fork/src/ae.c:477 (redis-server+0x85b45)\r\n #13 main /home/sundb/data/redis_fork/src/server.c:7211 (redis-server+0x7168c) (BuildId: dca0b1945ba30010e36129bdb296e488dd2b32d0)\r\n```","shortMessageHtmlLink":"Fix daylight race condition and some thread leaks (#13191)"}},{"before":"ce47834309ec8fd74cbeaf676313005ee440faa5","after":"4df037962dd446a4a686e2b6d56d5367b6c9f0db","ref":"refs/heads/unstable","pushedAt":"2024-04-02T12:09:52.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"moticless","name":"Moti Cohen","path":"/moticless","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/24944278?s=80&v=4"},"commit":{"message":"Change FLUSHALL/FLUSHDB SYNC to run as blocking ASYNC (#13167)\n\n# Overview\r\nUsers utilize the `FLUSHDB SYNC` and `FLUSHALL SYNC` commands for a variety of \r\nreasons. The main issue with this command is that if the database becomes \r\nsubstantial in size, the server will be unresponsive for an extended period. \r\nOther than freezing application traffic, this may also lead some clients making \r\nincorrect judgments about the server's availability. For instance, a watchdog may \r\nerroneously decide to terminate the process, resulting in potential adverse \r\noutcomes. While a `FLUSH* ASYNC` can address these issues, it might not be used \r\nfor two reasons: firstly, it's not the default, and secondly, in some cases, the \r\nclient issuing the flush wants to wait for its completion before repopulating the \r\ndatabase.\r\n\r\nBetween the option of triggering FLUSH* asynchronously in the background without \r\nindication for completion versus running it synchronously in the foreground by \r\nthe main thread, there is another more appealing option. We can block the\r\nclient that requested the flush, execute the flush command in the background, and \r\nonce done, unblock the client and return notification for completion. This approach \r\nensures the server remains responsive to other clients, and the blocked client \r\nreceives the expected response only after the flush operation has been successfully \r\ncarried out.\r\n\r\n# Implementation details\r\nInstead of defining yet another flavor to the flush command, we can modify\r\n`FLUSHALL SYNC` and `FLUSHDB SYNC` always run in this new mode.\r\n\r\n## Extending BIO Threads capabilities\r\nToday jobs that are carried out by BIO threads don't have the capability to \r\nindicate completion to the main thread. We can add this infrastructure by having\r\nan additional dummy job, coined as completion-job, that eventually will be written \r\nby BIO threads to a response-queue. The main thread will take care to consume items\r\nfrom the response-queue and call the provided callback function of each \r\ncompletion-job.\r\n\r\n## FLUSH* SYNC to run as blocking ASYNC\r\nCommand `FLUSH* SYNC` will be modified to create one or more async jobs to flush\r\nDB(s) and afterward will push additional completion-job request. By sending the\r\ncompletion job request only at the end, the main thread will be called back only\r\nafter all the preceding jobs completed their task in the background. During that\r\ntime, the client of the command is suspended and marked as `BLOCKED_LAZYFREE`\r\nwhereas any other client will be able to communicate with the server without any\r\nissue.","shortMessageHtmlLink":"Change FLUSHALL/FLUSHDB SYNC to run as blocking ASYNC (#13167)"}},{"before":"0b34396924eca4edc524469886dc5be6c77ec4ed","after":"ce47834309ec8fd74cbeaf676313005ee440faa5","ref":"refs/heads/unstable","pushedAt":"2024-04-01T15:08:55.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"moticless","name":"Moti Cohen","path":"/moticless","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/24944278?s=80&v=4"},"commit":{"message":"kvstoreIteratorNext() wrongly reset iterator twice (#13178)\n\nIt calls kvstoreIteratorNextDict() which eventually calls dictResumeRehashing()\r\nAnd then, on return, it calls dictResetIterator(iter) which calls dictResumeRehashing().\r\nWe end up with pauserehash value decremented twice instead of once.","shortMessageHtmlLink":"kvstoreIteratorNext() wrongly reset iterator twice (#13178)"}},{"before":"d6d38dc9285a01261b93b85277086d53c7b7714a","after":null,"ref":"refs/heads/revert-13157-license_change","pushedAt":"2024-03-26T18:01:49.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"LiorKogan","name":"Lior Kogan","path":"/LiorKogan","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/7252991?s=80&v=4"}},{"before":null,"after":"d6d38dc9285a01261b93b85277086d53c7b7714a","ref":"refs/heads/revert-13157-license_change","pushedAt":"2024-03-21T03:50:13.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"enjoy-binbin","name":"Binbin","path":"/enjoy-binbin","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22811481?s=80&v=4"},"commit":{"message":"Revert \"Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157)\"\n\nThis reverts commit 0b34396924eca4edc524469886dc5be6c77ec4ed.","shortMessageHtmlLink":"Revert \"Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157)\""}},{"before":"e64d91c37105bc2e23816b6f81b9ffc5e5d99801","after":"0b34396924eca4edc524469886dc5be6c77ec4ed","ref":"refs/heads/unstable","pushedAt":"2024-03-20T22:38:24.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"K-Jo","name":"Pieter Cailliau","path":"/K-Jo","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4069725?s=80&v=4"},"commit":{"message":"Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157)\n\n[Read more about the license change\r\nhere](https://redis.com/blog/redis-adopts-dual-source-available-licensing/)\r\nLive long and prosper 🖖","shortMessageHtmlLink":"Change license from BSD-3 to dual RSALv2+SSPLv1 (#13157)"}},{"before":"bad33f8738b4be5f58c4439a0c78312e4afbe432","after":"e64d91c37105bc2e23816b6f81b9ffc5e5d99801","ref":"refs/heads/unstable","pushedAt":"2024-03-20T20:44:29.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"oranagra","name":"Oran Agra","path":"/oranagra","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/7045099?s=80&v=4"},"commit":{"message":"Fix dict use-after-free problem in kvs->rehashing (#13154)\n\nIn ASAN CI, we find server may crash because of NULL ptr in `kvstoreIncrementallyRehash`.\r\nthe reason is that we use two phase unlink in `dbGenericDelete`. After `kvstoreDictTwoPhaseUnlinkFind`,\r\nthe dict may be in rehashing and only have one element in ht[0] of `db->keys`.\r\n\r\nWhen we delete the last element in `db->keys` meanwhile `db->keys` is in rehashing, we may free the\r\ndict in `kvstoreDictTwoPhaseUnlinkFree` without deleting the node in `kvs->rehashing`. Then we may\r\nuse this freed ptr in `kvstoreIncrementallyRehash` in the `serverCron` and cause the crash.\r\nThis is indeed a use-after-free problem.\r\n\r\nThe fix is to call rehashingCompleted in dictRelease and dictEmpty, so that every call for\r\nrehashingStarted is always matched with a rehashingCompleted.\r\n\r\nAdding a test in the unit test to catch it consistently\r\n\r\n---------\r\n\r\nCo-authored-by: Oran Agra \r\nCo-authored-by: debing.sun ","shortMessageHtmlLink":"Fix dict use-after-free problem in kvs->rehashing (#13154)"}},{"before":"e04d41d78dcdc4f2b332cdcf168521257ab0024e","after":"bad33f8738b4be5f58c4439a0c78312e4afbe432","ref":"refs/heads/unstable","pushedAt":"2024-03-19T06:52:55.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"oranagra","name":"Oran Agra","path":"/oranagra","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/7045099?s=80&v=4"},"commit":{"message":"fix wrong data type conversion in zrangeResultBeginStore (#13148)\n\nIn `beginResultEmission`, -1 means the result length is not known in\r\nadvance. But after #12185, if we pass -1 to `zrangeResultBeginStore`, it\r\nwill convert to SIZE_MAX in `zsetTypeCreate` and try to `dictExpand`.\r\nAlthough `dictExpand` won't succeed because the size overflows, I think\r\nwe'd better to avoid this wrong conversion.\r\n\r\nThis bug can be triggered when the source of `zrangestore` doesn't exist\r\nor we use `zrangestore` command with `byscore` or `bylex`.\r\nThe impact is that dst keys will be converted to use skiplist instead of\r\nlistpack.","shortMessageHtmlLink":"fix wrong data type conversion in zrangeResultBeginStore (#13148)"}},{"before":"aeada20140570643cda084d374056469a5c27bf2","after":"e04d41d78dcdc4f2b332cdcf168521257ab0024e","ref":"refs/heads/unstable","pushedAt":"2024-03-19T06:18:23.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"oranagra","name":"Oran Agra","path":"/oranagra","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/7045099?s=80&v=4"},"commit":{"message":"Prevent lua error_reply abuse from causing errorstats to become larger (#13141)\n\nUsers who abuse lua error_reply will generate a new error object on each\r\nerror call, which can make server.errors get bigger and bigger. This\r\nwill\r\ncause the server to block when calling INFO (we also return errorstats\r\nby\r\ndefault).\r\n\r\nTo prevent the damage it can cause, when a misuse is detected, we will\r\nprint a warning log and disable the errorstats to avoid adding more new\r\nerrors. It can be re-enabled via CONFIG RESETSTAT.\r\n\r\nBecause server.errors may be very large (it may be better now since we\r\nhave the limit), config resetstat may block for a while. So in\r\nresetErrorTableStats, we will try to lazyfree server.errors.\r\n\r\nSee the related discussion at the end of #8217.","shortMessageHtmlLink":"Prevent lua error_reply abuse from causing errorstats to become larger ("}},{"before":"7b070423b8e20cf14a9ab80dfa7bea5ba88c54a3","after":"aeada20140570643cda084d374056469a5c27bf2","ref":"refs/heads/unstable","pushedAt":"2024-03-19T02:14:19.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"soloestoy","name":"zhaozhao.zz","path":"/soloestoy","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/24804835?s=80&v=4"},"commit":{"message":"Avoid unnecessary dict shrink in zremrangeGenericCommand (#13143)\n\nIf the skiplist is emptied, there is no need to shrink the dict in\r\nskiplist, it can be deleted directly.","shortMessageHtmlLink":"Avoid unnecessary dict shrink in zremrangeGenericCommand (#13143)"}},{"before":"98a6e55d4e7eed5bb68480672a3358ee21b9dce8","after":"7b070423b8e20cf14a9ab80dfa7bea5ba88c54a3","ref":"refs/heads/unstable","pushedAt":"2024-03-18T15:41:54.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"oranagra","name":"Oran Agra","path":"/oranagra","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/7045099?s=80&v=4"},"commit":{"message":"Fix dictionary use-after-free in active expire and make kvstore iter to respect EMPTY flag (#13135)\n\nAfter #13072, there is an use-after-free error. In expireScanCallback, we\r\nwill delete the dict, and then in dictScan we will continue to use the dict,\r\nlike we will doing `dictResumeRehashing(d)` in the end, this casued an error.\r\n\r\nIn this PR, in freeDictIfNeeded, if the dict's pauserehash is set, don't\r\ndelete the dict yet, and then when scan returns try to delete it again.\r\n\r\nAt the same time, we noticed that there will be similar problems in iterator.\r\nWe may also delete elements during the iteration process, causing the dict\r\nto be deleted, so the part related to iter in the PR has also been modified.\r\ndictResetIterator was also missing from the previous kvstoreIteratorNextDict,\r\nwe currently have no scenario that elements will be deleted in kvstoreIterator\r\nprocess, deal with it together to avoid future problems. Added some simple\r\ntests to verify the changes.\r\n\r\nIn addition, the modification in #13072 omitted initTempDb and emptyDbAsync,\r\nand they were also added. This PR also remove the slow flag from the expire\r\ntest (consumes 1.3s) so that problems can be found in CI in the future.","shortMessageHtmlLink":"Fix dictionary use-after-free in active expire and make kvstore iter …"}},{"before":"3f725b8619ad9ae9eb129e760a47e635972be3ee","after":"98a6e55d4e7eed5bb68480672a3358ee21b9dce8","ref":"refs/heads/unstable","pushedAt":"2024-03-18T06:22:19.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"oranagra","name":"Oran Agra","path":"/oranagra","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/7045099?s=80&v=4"},"commit":{"message":"Add missing REDIS_STATIC in quicklist (#13147)\n\nCompiler complained when I tried to compile only quicklist.c.\r\nSince static keyword is needed when a static function declaration is\r\nplaced before its implementation.\r\n\r\n```\r\n#ifndef REDIS_STATIC\r\n#define REDIS_STATIC static\r\n#endif\r\n```\r\n\r\n[How to solve static declaration follows non-static declaration in GCC C\r\ncode?](https://stackoverflow.com/questions/3148244/how-to-solve-static-declaration-follows-non-static-declaration-in-gcc-c-code)","shortMessageHtmlLink":"Add missing REDIS_STATIC in quicklist (#13147)"}},{"before":"1d77a8e2c5e2322cc119af34a6685876af90effd","after":"3f725b8619ad9ae9eb129e760a47e635972be3ee","ref":"refs/heads/unstable","pushedAt":"2024-03-17T07:06:51.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"oranagra","name":"Oran Agra","path":"/oranagra","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/7045099?s=80&v=4"},"commit":{"message":"Change mmap rand bits as a temporary mitigation to resolve asan bug (#13150)\n\nThere is a new change in linux kernel 6.6.6 that uses randomization of\r\naddress space to harden security, but it interferes with the way ASAN\r\nworks. Folks are working on a fix, but this is a temporary mitigation\r\nfor us to get our CI to be green again.\r\n\r\nSee https://github.com/google/sanitizers/issues/1716 for more\r\ninformation\r\n\r\nSee\r\nhttps://github.com/redis/redis/actions/runs/8305126288/job/22731614306?pr=13149\r\nfor a recent failure","shortMessageHtmlLink":"Change mmap rand bits as a temporary mitigation to resolve asan bug (#…"}},{"before":"3b3d16f7488063f0d2ee7d5cfaeba736169046de","after":"1d77a8e2c5e2322cc119af34a6685876af90effd","ref":"refs/heads/unstable","pushedAt":"2024-03-13T15:02:00.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"oranagra","name":"Oran Agra","path":"/oranagra","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/7045099?s=80&v=4"},"commit":{"message":"Makefile respect user's REDIS_CFLAGS and OPT (#13073)\n\nThis change to the Makefile makes it possible to opt out of\r\n`-fno-omit-frame-pointer` added in #12973 and `-flto` (#11350). Those\r\nfeatures were implemented by conditionally modifying the `REDIS_CFLAGS`\r\nand `REDIS_LDFLAGS` variables. Historically, those variables provided a\r\nway for users to pass options to the compiler and linker unchanged.\r\n\r\nInstead of conditionally appending optimization flags to REDIS_CFLAGS\r\nand REDIS_LDFLAGS, I want to append them to the OPTIMIZATION variable.\r\n\r\nLater in the Makefile, we have `OPT=$(OPTIMIZATION)` (meaning\r\nOPTIMIZATION is only a default for OPT, but OPT can be overridden by the\r\nuser), and later the flags are combined like this:\r\n\r\nFINAL_CFLAGS=$(STD) $(WARN) $(OPT) $(DEBUG) $(CFLAGS) $(REDIS_CFLAGS)\r\n FINAL_LDFLAGS=$(LDFLAGS) $(OPT) $(REDIS_LDFLAGS) $(DEBUG)\r\n\r\nThis makes it possible for the the user to override all optimization\r\nflags with e.g. `make OPT=-O1` or just `make OPT=`.\r\n\r\nFor some reason `-O3` was also already added to REDIS_LDFLAGS by default\r\nin #12339, so I added OPT to FINAL_LDFLAGS to avoid more complex logic\r\n(such as introducing a separate LD_OPT variable).","shortMessageHtmlLink":"Makefile respect user's REDIS_CFLAGS and OPT (#13073)"}},{"before":"ad28d222edcef9d4496fd7a94656013f07dd08e5","after":"3b3d16f7488063f0d2ee7d5cfaeba736169046de","ref":"refs/heads/unstable","pushedAt":"2024-03-13T06:30:20.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"oranagra","name":"Oran Agra","path":"/oranagra","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/7045099?s=80&v=4"},"commit":{"message":"Add KVSTORE_FREE_EMPTY_DICTS to cluster mode keys / expires kvstore (#13072)\n\nCurrently (following #11695, and #12822), keys kvstore and expires\r\nkvstore both flag with ON_DEMAND, it means that a cluster node will\r\nonly allocate a dict when the slot is assigned to it and populated,\r\nbut on the other hand, when the slot is unassigned, the dict will\r\nremain allocated.\r\n\r\nWe considered releasing the dict when the slot is unassigned, but it\r\ncauses complications on replicas. On the other hand, from benchmarks\r\nwe conducted, it looks like the performance impact of releasing the\r\ndict when it becomes empty and re-allocate it when a key is added\r\nagain, isn't huge.\r\n\r\nThis PR add KVSTORE_FREE_EMPTY_DICTS to cluster mode keys / expires\r\nkvstore.\r\n\r\nThe impact is about about 2% performance drop, for this hopefully\r\nuncommon scenario.\r\n\r\n---------\r\n\r\nCo-authored-by: Oran Agra ","shortMessageHtmlLink":"Add KVSTORE_FREE_EMPTY_DICTS to cluster mode keys / expires kvstore (#…"}},{"before":"a8e745117f3648cc813d18f5a8cf1218ec83cb5d","after":"ad28d222edcef9d4496fd7a94656013f07dd08e5","ref":"refs/heads/unstable","pushedAt":"2024-03-13T06:27:41.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"oranagra","name":"Oran Agra","path":"/oranagra","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/7045099?s=80&v=4"},"commit":{"message":"Lua eval scripts first in first out LRU eviction (#13108)\n\nIn some cases, users will abuse lua eval. Each EVAL call generates\r\na new lua script, which is added to the lua interpreter and cached\r\nto redis-server, consuming a large amount of memory over time.\r\n\r\nSince EVAL is mostly the one that abuses the lua cache, and these\r\nwon't have pipeline issues (i.e. the script won't disappear\r\nunexpectedly,\r\nand cause errors like it would with SCRIPT LOAD and EVALSHA),\r\nwe implement a plain FIFO LRU eviction only for these (not for\r\nscripts loaded with SCRIPT LOAD).\r\n\r\n### Implementation notes:\r\nWhen not abused we'll probably have less than 100 scripts, and when\r\nabused we'll have many thousands. So we use a hard coded value of 500\r\nscripts. And considering that we don't have many scripts, then unlike\r\nkeys, we don't need to worry about the memory usage of keeping a true\r\nsorted LRU linked list. We compute the SHA of each script anyway,\r\nand put the script in a dict, we can store a listNode there, and use\r\nit for quick removal and re-insertion into an LRU list each time the\r\nscript is used.\r\n\r\n### New interfaces:\r\nAt the same time, a new `evicted_scripts` field is added to\r\nINFO, which represents the number of evicted eval scripts. Users\r\ncan check it to see if they are abusing EVAL.\r\n\r\n### benchmark:\r\n`./src/redis-benchmark -P 10 -n 1000000 -r 10000000000 eval \"return\r\n__rand_int__\" 0`\r\n\r\nThe simple abuse of eval benchmark test that will create 1 million EVAL\r\nscripts. The performance has been improved by 50%, and the max latency\r\nhas dropped from 500ms to 13ms (this may be caused by table expansion\r\ninside Lua when the number of scripts is large). And in the INFO memory,\r\nit used to consume 120MB (server cache) + 310MB (lua engine), but now\r\nit only consumes 70KB (server cache) + 210KB (lua_engine) because of\r\nthe scripts eviction.\r\n\r\nFor non-abusive case of about 100 EVAL scripts, there's no noticeable\r\nchange in performance or memory usage.\r\n\r\n### unlikely potentially breaking change:\r\nin theory, a user can maybe load a\r\nscript with EVAL and then use EVALSHA to call it (by calculating the\r\nSHA1 value on the client side), it could be that if we read the docs\r\ncarefully we'll realized it's a valid scenario, but we suppose it's\r\nextremely rare. So it may happen that EVALSHA acts on a script created\r\nby EVAL, and the script is evicted and EVALSHA returns a NOSCRIPT error.\r\nthat is if you have more than 500 scripts being used in the same\r\ntransaction / pipeline.\r\n\r\nThis solves the second point in #13102.","shortMessageHtmlLink":"Lua eval scripts first in first out LRU eviction (#13108)"}},{"before":"9efc6ad6a63198e6f6af3c5e9fabc3a7ffc6fe4b","after":"a8e745117f3648cc813d18f5a8cf1218ec83cb5d","ref":"refs/heads/unstable","pushedAt":"2024-03-13T06:23:32.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"oranagra","name":"Oran Agra","path":"/oranagra","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/7045099?s=80&v=4"},"commit":{"message":"Xread last entry in stream (#7388) (#13117)\n\nAllow using `+` as a special ID for last item in stream on XREAD\r\ncommand.\r\n\r\nThis would allow to iterate on a stream with XREAD starting with the\r\nlast available message instead of the next one which `$` is used for.\r\nI.e. the caller can use `BLOCK` and `+` on the first call, and change to\r\n`$` on the next call.\r\n\r\nCloses #7388\r\n\r\n---------\r\n\r\nCo-authored-by: Felipe Machado <462154+felipou@users.noreply.github.com>","shortMessageHtmlLink":"Xread last entry in stream (#7388) (#13117)"}},{"before":"9c065c417d5cd7a1b35310d8c7760eeb119be1c0","after":"9efc6ad6a63198e6f6af3c5e9fabc3a7ffc6fe4b","ref":"refs/heads/unstable","pushedAt":"2024-03-12T16:26:13.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"madolson","name":"Madelyn Olson","path":"/madolson","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34459052?s=80&v=4"},"commit":{"message":"Add API RedisModule_ClusterKeySlot and RedisModule_ClusterCanonicalKeyNameInSlot (#13069)\n\nSometimes it's useful to compute a key's cluster slot in a module.\r\n\r\nThis API function is just like the command CLUSTER KEYSLOT (but faster).\r\n\r\nA \"reverse\" API is also added:\r\n`RedisModule_ClusterCanonicalKeyNameInSlot`. Given a slot, it returns a\r\nshort string that we can call a canonical key for the slot.","shortMessageHtmlLink":"Add API RedisModule_ClusterKeySlot and RedisModule_ClusterCanonicalKe…"}},{"before":"da727ad445a640950baa097124070468a0316cc9","after":"9c065c417d5cd7a1b35310d8c7760eeb119be1c0","ref":"refs/heads/unstable","pushedAt":"2024-03-12T14:35:52.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"oranagra","name":"Oran Agra","path":"/oranagra","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/7045099?s=80&v=4"},"commit":{"message":"Enable accept4() on *BSD (#13104)\n\nRedis enabled `accept4` on Linux after #9177, reducing extra system\r\ncalls for sockets.\r\n\r\n`accept4` system call is also widely supported on *BSD and Solaris in\r\naddition to Linux. This PR enables `accept4` on all platforms that\r\nsupport it.\r\n\r\n### References\r\n- [accept4 on\r\nFreeBSD](https://man.freebsd.org/cgi/man.cgi?query=accept4&sektion=2&n=1)\r\n- [accept4 on\r\nDragonFly](https://man.dragonflybsd.org/?command=accept§ion=2)\r\n- [accept4 on NetBSD](https://man.netbsd.org/accept.2)\r\n- [accept4 on OpenBSD](https://man.openbsd.org/accept4.2)\r\n- [accept4 on\r\nSolaris](https://docs.oracle.com/cd/E88353_01/html/E37843/accept4-3c.html)","shortMessageHtmlLink":"Enable accept4() on *BSD (#13104)"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAEQGGhxQA","startCursor":null,"endCursor":null}},"title":"Activity · redis/redis"}