Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[crash] Redis Stack Cluster 7.2.4 with RedisJSON 2.6.7 #1597

Open
cletustboone opened this issue Apr 18, 2024 · 1 comment
Open

[crash] Redis Stack Cluster 7.2.4 with RedisJSON 2.6.7 #1597

cletustboone opened this issue Apr 18, 2024 · 1 comment

Comments

@cletustboone
Copy link

Hi there. I'm running a 4-shard cluster on EC2 Amazon Linux 2023 instances. About once a day, we are seeing redis crash with this error from the RedisJSON module. This always happens on a master, but we have not been able to trace this to any calling code yet, so I am not sure which command causing this. At any rate, figured you'd want the bug report.

redis-server[4139]: === REDIS BUG REPORT START: Cut & paste starting from here ===
redis-server[4139]: 4139:M 26 Jan 2024 17:34:26.256 # Redis 7.2.4 crashed by signal: 6, si_code: -6
redis-server[4139]: 4139:M 26 Jan 2024 17:34:26.256 # Crashed running the instruction at: 0x7f38c28a153c
redis-server[4139]: ------ STACK TRACE ------
redis-server[4139]: EIP:
redis-server[4139]: /lib64/libc.so.6(+0xa153c)[0x7f38c28a153c]
redis-server[4139]: Backtrace:
redis-server[4139]: /lib64/libc.so.6(+0x54dd0)[0x7f38c2854dd0]
redis-server[4139]: /lib64/libc.so.6(+0xa153c)[0x7f38c28a153c]
redis-server[4139]: /lib64/libc.so.6(raise+0x16)[0x7f38c2854d26]
redis-server[4139]: /lib64/libc.so.6(abort+0xd3)[0x7f38c28287f3]
redis-server[4139]: /lib64/libc.so.6(+0x2871b)[0x7f38c282871b]
redis-server[4139]: /lib64/libc.so.6(+0x4dcc6)[0x7f38c284dcc6]
redis-server[4139]: /opt/redis-stack/bin/redis-server *:6379 [cluster][0x58c8fb]
redis-server[4139]: /opt/redis-stack/bin/redis-server *:6379 [cluster](createStringObject+0x75)[0x48f5b5]
redis-server[4139]: /opt/redis-stack/bin/redis-server *:6379 [cluster](RM_CreateString+0x15)[0x533375]
redis-server[4139]: /opt/redis-stack/lib/redistimeseries.so(SeriesRecord_Deserialize+0xe8)[0x7f38c30d6be8]
redis-server[4139]: /opt/redis-stack/lib/redistimeseries.so(MR_RecordDeSerialize+0x2f)[0x7f38c30f029f]
redis-server[4139]: /opt/redis-stack/lib/redistimeseries.so(+0x23367)[0x7f38c30d7367]
redis-server[4139]: /opt/redis-stack/lib/redistimeseries.so(MR_RecordDeSerialize+0x2f)[0x7f38c30f029f]
redis-server[4139]: /opt/redis-stack/lib/redistimeseries.so(+0x394a7)[0x7f38c30ed4a7]
redis-server[4139]: /opt/redis-stack/lib/redistimeseries.so(+0x3784e)[0x7f38c30eb84e]
redis-server[4139]: /opt/redis-stack/lib/redistimeseries.so(+0x3ee76)[0x7f38c30f2e76]
redis-server[4139]: /lib64/libc.so.6(+0x9f7f2)[0x7f38c289f7f2]
redis-server[4139]: /lib64/libc.so.6(+0x3f450)[0x7f38c283f450]
redis-server[4139]: ------ REGISTERS ------
redis-server[4139]: 4139:M 26 Jan 2024 17:34:26.258 #
redis-server[4139]: RAX:0000000000000000 RBX:00007f38121fc640
redis-server[4139]: RCX:00007f38c28a153c RDX:0000000000000006
redis-server[4139]: RDI:000000000000102b RSI:000000000000df4f
redis-server[4139]: RBP:000000000000df4f RSP:00007f38121fad60
redis-server[4139]: R8 :00007f38121fae30 R9 :0000000000000000
redis-server[4139]: R10:0000000000000008 R11:0000000000000246
redis-server[4139]: R12:0000000000000006 R13:0000000000000073
redis-server[4139]: R14:0000000000653a78 R15:00007f38176577e8
redis-server[4139]: RIP:00007f38c28a153c EFL:0000000000000246
redis-server[4139]: CSGSFS:002b000000000033
redis-server[4139]: 4139:M 26 Jan 2024 17:34:26.258 # (00007f38121fad6f) -> 00007f38c29fa680
redis-server[4139]: 4139:M 26 Jan 2024 17:34:26.258 # (00007f38121fad6e) -> 0000000000681db0
redis-server[4139]: 4139:M 26 Jan 2024 17:34:26.258 # (00007f38121fad6d) -> 00007f38121fae40
redis-server[4139]: 4139:M 26 Jan 2024 17:34:26.258 # (00007f38121fad6c) -> 00007f38c29fa680
redis-server[4139]: 4139:M 26 Jan 2024 17:34:26.258 # (00007f38121fad6b) -> 788c50d1da49c000
redis-server[4139]: 4139:M 26 Jan 2024 17:34:26.258 # (00007f38121fad6a) -> 0000000000000000
redis-server[4139]: 4139:M 26 Jan 2024 17:34:26.258 # (00007f38121fad69) -> 788c50d1da49c000
redis-server[4139]: 4139:M 26 Jan 2024 17:34:26.258 # (00007f38121fad68) -> 0000008000000000
redis-server[4139]: 4139:M 26 Jan 2024 17:34:26.258 # (00007f38121fad67) -> 0000000000000000
redis-server[4139]: 4139:M 26 Jan 2024 17:34:26.258 # (00007f38121fad66) -> 0000000000000000
redis-server[4139]: 4139:M 26 Jan 2024 17:34:26.258 # (00007f38121fad65) -> 0000000000000000
redis-server[4139]: 4139:M 26 Jan 2024 17:34:26.258 # (00007f38121fad64) -> 0000000000000000
redis-server[4139]: 4139:M 26 Jan 2024 17:34:26.258 # (00007f38121fad63) -> 0000000000000000
redis-server[4139]: 4139:M 26 Jan 2024 17:34:26.258 # (00007f38121fad62) -> 00007f37fc000bc4
redis-server[4139]: 4139:M 26 Jan 2024 17:34:26.258 # (00007f38121fad61) -> 00007f37fc000b60
redis-server[4139]: 4139:M 26 Jan 2024 17:34:26.258 # (00007f38121fad60) -> 00007f37fc000bc4
redis-server[4139]: ------ INFO OUTPUT ------
redis-server[4139]: # Server
redis-server[4139]: redis_version:7.2.4
redis-server[4139]: redis_git_sha1:d2c8a4b9
redis-server[4139]: redis_git_dirty:0
redis-server[4139]: redis_build_id:38239669611a33ed
redis-server[4139]: redis_mode:cluster
redis-server[4139]: os:Linux 6.1.72-96.166.amzn2023.x86_64 x86_64
redis-server[4139]: arch_bits:64
redis-server[4139]: monotonic_clock:POSIX clock_gettime
redis-server[4139]: multiplexing_api:epoll
redis-server[4139]: atomicvar_api:c11-builtin
redis-server[4139]: gcc_version:11.4.1
redis-server[4139]: process_id:4139
redis-server[4139]: process_supervised:no
redis-server[4139]: run_id:fab608d235e67dec29a300f774adc731dfb917f8
redis-server[4139]: tcp_port:6379
redis-server[4139]: server_time_usec:1706290466258294
redis-server[4139]: uptime_in_seconds:72663
redis-server[4139]: uptime_in_days:0
redis-server[4139]: hz:10
redis-server[4139]: configured_hz:10
redis-server[4139]: lru_clock:11791650
redis-server[4139]: executable:/opt/redis-stack/bin/redis-server
redis-server[4139]: config_file:/etc/redis-stack.conf
redis-server[4139]: io_threads_active:0
redis-server[4139]: listener0:name=tcp,bind=*,bind=-::*,port=6379
redis-server[4139]: 
redis-server[4139]: # Clients
redis-server[4139]: connected_clients:111
redis-server[4139]: cluster_connections:14
redis-server[4139]: maxclients:10000
redis-server[4139]: client_recent_max_input_buffer:327680
redis-server[4139]: client_recent_max_output_buffer:20504
redis-server[4139]: blocked_clients:5
redis-server[4139]: tracking_clients:0
redis-server[4139]: clients_in_timeout_table:0
redis-server[4139]: total_blocking_keys:0
redis-server[4139]: total_blocking_keys_on_nokey:0
redis-server[4139]: 
redis-server[4139]: # Memory
redis-server[4139]: used_memory:2129418712
redis-server[4139]: used_memory_human:1.98G
redis-server[4139]: used_memory_rss:2207027200
redis-server[4139]: used_memory_rss_human:2.06G
redis-server[4139]: used_memory_peak:2138500328
redis-server[4139]: used_memory_peak_human:1.99G
redis-server[4139]: used_memory_peak_perc:99.58%
redis-server[4139]: used_memory_overhead:38270576
redis-server[4139]: used_memory_startup:1800360
redis-server[4139]: used_memory_dataset:2091148136
redis-server[4139]: used_memory_dataset_perc:98.29%
redis-server[4139]: allocator_allocated:2133449816
redis-server[4139]: allocator_active:2166620160
redis-server[4139]: allocator_resident:2209480704
redis-server[4139]: total_system_memory:16546299904
redis-server[4139]: total_system_memory_human:15.41G
redis-server[4139]: used_memory_lua:31744
redis-server[4139]: used_memory_vm_eval:31744
redis-server[4139]: used_memory_lua_human:31.00K
redis-server[4139]: used_memory_scripts_eval:0
redis-server[4139]: number_of_cached_scripts:0
redis-server[4139]: number_of_functions:0
redis-server[4139]: number_of_libraries:0
redis-server[4139]: used_memory_vm_functions:32768
redis-server[4139]: used_memory_vm_total:64512
redis-server[4139]: used_memory_vm_total_human:63.00K
redis-server[4139]: used_memory_functions:184
redis-server[4139]: used_memory_scripts:184
redis-server[4139]: used_memory_scripts_human:184B
redis-server[4139]: maxmemory:0
redis-server[4139]: maxmemory_human:0B
redis-server[4139]: maxmemory_policy:noeviction
redis-server[4139]: allocator_frag_ratio:1.02
redis-server[4139]: allocator_frag_bytes:33170344
redis-server[4139]: allocator_rss_ratio:1.02
redis-server[4139]: allocator_rss_bytes:42860544
redis-server[4139]: rss_overhead_ratio:1.00
redis-server[4139]: rss_overhead_bytes:-2453504
redis-server[4139]: mem_fragmentation_ratio:1.04
redis-server[4139]: mem_fragmentation_bytes:78678912
redis-server[4139]: mem_not_counted_for_evict:13472
redis-server[4139]: mem_replication_backlog:1048592
redis-server[4139]: mem_total_replication_buffers:1066208
redis-server[4139]: mem_clients_slaves:17632
redis-server[4139]: mem_clients_normal:1508160
redis-server[4139]: mem_cluster_links:12864
redis-server[4139]: mem_aof_buffer:0
redis-server[4139]: mem_allocator:jemalloc-5.3.0
redis-server[4139]: active_defrag_running:0
redis-server[4139]: lazyfree_pending_objects:0
redis-server[4139]: lazyfreed_objects:0
redis-server[4139]: 
redis-server[4139]: # Persistence
redis-server[4139]: loading:0
redis-server[4139]: async_loading:0
redis-server[4139]: current_cow_peak:0
redis-server[4139]: current_cow_size:0
redis-server[4139]: current_cow_size_age:0
redis-server[4139]: current_fork_perc:0.00
redis-server[4139]: current_save_keys_processed:0
redis-server[4139]: current_save_keys_total:0
redis-server[4139]: rdb_changes_since_last_save:316
redis-server[4139]: rdb_bgsave_in_progress:0
redis-server[4139]: rdb_last_save_time:1706290231
redis-server[4139]: rdb_last_bgsave_status:ok
redis-server[4139]: rdb_last_bgsave_time_sec:5
redis-server[4139]: rdb_current_bgsave_time_sec:-1
redis-server[4139]: rdb_saves:80
redis-server[4139]: rdb_last_cow_size:6471680
redis-server[4139]: rdb_last_load_keys_expired:0
redis-server[4139]: rdb_last_load_keys_loaded:315089
redis-server[4139]: aof_enabled:0
redis-server[4139]: aof_rewrite_in_progress:0
redis-server[4139]: aof_rewrite_scheduled:0
redis-server[4139]: aof_last_rewrite_time_sec:-1
redis-server[4139]: aof_current_rewrite_time_sec:-1
redis-server[4139]: aof_last_bgrewrite_status:ok
redis-server[4139]: aof_rewrites:0
redis-server[4139]: aof_rewrites_consecutive_failures:0
redis-server[4139]: aof_last_write_status:ok
redis-server[4139]: aof_last_cow_size:0
redis-server[4139]: module_fork_in_progress:0
redis-server[4139]: module_fork_last_cow_size:0
redis-server[4139]: 
redis-server[4139]: # Stats
redis-server[4139]: total_connections_received:2435
redis-server[4139]: total_commands_processed:407988
redis-server[4139]: instantaneous_ops_per_sec:392
redis-server[4139]: total_net_input_bytes:681164979
redis-server[4139]: total_net_output_bytes:263033942
redis-server[4139]: total_net_repl_input_bytes:191829877
redis-server[4139]: total_net_repl_output_bytes:225413557
redis-server[4139]: instantaneous_input_kbps:1335.26
redis-server[4139]: instantaneous_output_kbps:124.22
redis-server[4139]: instantaneous_input_repl_kbps:0.00
redis-server[4139]: instantaneous_output_repl_kbps:0.00
redis-server[4139]: rejected_connections:0
redis-server[4139]: sync_full:1
redis-server[4139]: sync_partial_ok:1
redis-server[4139]: sync_partial_err:1
redis-server[4139]: expired_keys:2041
redis-server[4139]: expired_stale_perc:0.00
redis-server[4139]: expired_time_cap_reached_count:0
redis-server[4139]: expire_cycle_cpu_milliseconds:6649
redis-server[4139]: evicted_keys:0
redis-server[4139]: evicted_clients:0
redis-server[4139]: total_eviction_exceeded_time:0
redis-server[4139]: current_eviction_exceeded_time:0
redis-server[4139]: keyspace_hits:773394
redis-server[4139]: keyspace_misses:0
redis-server[4139]: pubsub_channels:0
redis-server[4139]: pubsub_patterns:0
redis-server[4139]: pubsubshard_channels:0
redis-server[4139]: latest_fork_usec:24466
redis-server[4139]: total_forks:81
redis-server[4139]: migrate_cached_sockets:0
redis-server[4139]: slave_expires_tracked_keys:0
redis-server[4139]: active_defrag_hits:0
redis-server[4139]: active_defrag_misses:0
redis-server[4139]: active_defrag_key_hits:0
redis-server[4139]: active_defrag_key_misses:0
redis-server[4139]: total_active_defrag_time:0
redis-server[4139]: current_active_defrag_time:0
redis-server[4139]: tracking_total_keys:0
redis-server[4139]: tracking_total_items:0
redis-server[4139]: tracking_total_prefixes:0
redis-server[4139]: unexpected_error_replies:0
redis-server[4139]: total_error_replies:3812
redis-server[4139]: dump_payload_sanitizations:0
redis-server[4139]: total_reads_processed:234994
redis-server[4139]: total_writes_processed:273857
redis-server[4139]: io_threaded_reads_processed:0
redis-server[4139]: io_threaded_writes_processed:0
redis-server[4139]: reply_buffer_shrinks:2365
redis-server[4139]: reply_buffer_expands:243
redis-server[4139]: eventloop_cycles:1770496
redis-server[4139]: eventloop_duration_sum:130803516
redis-server[4139]: eventloop_duration_cmd_sum:3084352
redis-server[4139]: instantaneous_eventloop_cycles_per_sec:414
redis-server[4139]: instantaneous_eventloop_duration_usec:111
redis-server[4139]: acl_access_denied_auth:0
redis-server[4139]: acl_access_denied_cmd:0
redis-server[4139]: acl_access_denied_key:0
redis-server[4139]: acl_access_denied_channel:0
redis-server[4139]: 
redis-server[4139]: # Replication
redis-server[4139]: role:master
redis-server[4139]: connected_slaves:1
redis-server[4139]: slave0:ip=10.100.0.197,port=6379,state=online,offset=1964540014,lag=0
redis-server[4139]: master_failover_state:no-failover
redis-server[4139]: master_replid:5a8af0d4f621395190c2c3b77e5ca6845d495a7d
redis-server[4139]: master_replid2:4f5b7426318178f373a9ce29693fac239c554777
redis-server[4139]: master_repl_offset:1964540014
redis-server[4139]: second_repl_offset:1930570814
redis-server[4139]: repl_backlog_active:1
redis-server[4139]: repl_backlog_size:1048576
redis-server[4139]: repl_backlog_first_byte_offset:1963489742
redis-server[4139]: repl_backlog_histlen:1050273
redis-server[4139]: 
redis-server[4139]: # CPU
redis-server[4139]: used_cpu_sys:48.088235
redis-server[4139]: used_cpu_user:112.556871
redis-server[4139]: used_cpu_sys_children:20.861858
redis-server[4139]: used_cpu_user_children:292.125779
redis-server[4139]: used_cpu_sys_main_thread:0.708745
redis-server[4139]: used_cpu_user_main_thread:2.350830
redis-server[4139]: 
redis-server[4139]: # Modules
redis-server[4139]: module:name=timeseries,ver=11011,api=1,filters=0,usedby=[],using=[],options=[]
redis-server[4139]: module:name=ReJSON,ver=20607,api=1,filters=0,usedby=[search],using=[],options=[handle-io-errors]
redis-server[4139]: module:name=search,ver=20811,api=1,filters=0,usedby=[],using=[ReJSON],options=[handle-io-errors]
redis-server[4139]: 
redis-server[4139]: # Commandstats
redis-server[4139]: cmdstat_del:calls=1062,usec=31820,usec_per_call=29.96,rejected_calls=0,failed_calls=0
redis-server[4139]: cmdstat_config|set:calls=3,usec=10,usec_per_call=3.33,rejected_calls=0,failed_calls=0
redis-server[4139]: cmdstat_config|get:calls=12,usec=89,usec_per_call=7.42,rejected_calls=0,failed_calls=0
redis-server[4139]: cmdstat_replconf:calls=70768,usec=85464,usec_per_call=1.21,rejected_calls=0,failed_calls=0
redis-server[4139]: cmdstat_slowlog|get:calls=65,usec=481,usec_per_call=7.40,rejected_calls=0,failed_calls=0
redis-server[4139]: cmdstat_pexpireat:calls=737,usec=2409,usec_per_call=3.27,rejected_calls=0,failed_calls=0
redis-server[4139]: cmdstat_psync:calls=2,usec=74,usec_per_call=37.00,rejected_calls=0,failed_calls=0
redis-server[4139]: cmdstat_cluster|failover:calls=1,usec=31,usec_per_call=31.00,rejected_calls=0,failed_calls=0
redis-server[4139]: cmdstat_cluster|info:calls=68,usec=3554,usec_per_call=52.26,rejected_calls=0,failed_calls=0
redis-server[4139]: cmdstat_cluster|replicate:calls=1,usec=162,usec_per_call=162.00,rejected_calls=0,failed_calls=0
redis-server[4139]: cmdstat_cluster|slots:calls=6267,usec=217002,usec_per_call=34.63,rejected_calls=0,failed_calls=0
redis-server[4139]: cmdstat_cluster|forget:calls=4,usec=144,usec_per_call=36.00,rejected_calls=0,failed_calls=0
redis-server[4139]: cmdstat_cluster|nodes:calls=103,usec=8741,usec_per_call=84.86,rejected_calls=0,failed_calls=0
redis-server[4139]: cmdstat_cluster|meet:calls=1,usec=27,usec_per_call=27.00,rejected_calls=0,failed_calls=0
redis-server[4139]: cmdstat_acl|setuser:calls=7,usec=124,usec_per_call=17.71,rejected_calls=0,failed_calls=0
redis-server[4139]: cmdstat_acl|getuser:calls=1,usec=2,usec_per_call=2.00,rejected_calls=0,failed_calls=0
redis-server[4139]: cmdstat_expire:calls=46517,usec=179906,usec_per_call=3.87,rejected_calls=0,failed_calls=0
redis-server[4139]: cmdstat_auth:calls=36,usec=223,usec_per_call=6.19,rejected_calls=0,failed_calls=4
redis-server[4139]: cmdstat_json.set:calls=1811,usec=75052,usec_per_call=41.44,rejected_calls=0,failed_calls=0
redis-server[4139]: cmdstat_dbsize:calls=6,usec=11,usec_per_call=1.83,rejected_calls=0,failed_calls=0
redis-server[4139]: cmdstat_timeseries.REFRESHCLUSTER:calls=2,usec=112,usec_per_call=56.00,rejected_calls=0,failed_calls=0
redis-server[4139]: cmdstat_client|list:calls=89,usec=8287,usec_per_call=93.11,rejected_calls=0,failed_calls=0
redis-server[4139]: cmdstat_ts.add:calls=47270,usec=1284466,usec_per_call=27.17,rejected_calls=0,failed_calls=16
redis-server[4139]: cmdstat_ping:calls=75,usec=51,usec_per_call=0.68,rejected_calls=0,failed_calls=0
redis-server[4139]: cmdstat_select:calls=1,usec=1,usec_per_call=1.00,rejected_calls=0,failed_calls=0
redis-server[4139]: cmdstat_ts.mrange:calls=13005,usec=18052033475,usec_per_call=1388084.00,rejected_calls=0,failed_calls=3791
redis-server[4139]: cmdstat_timeseries.INNERCOMMUNICATION:calls=208638,usec=982268,usec_per_call=4.71,rejected_calls=0,failed_calls=1
redis-server[4139]: cmdstat_multi:calls=1,usec=1,usec_per_call=1.00,rejected_calls=0,failed_calls=0
redis-server[4139]: cmdstat_hset:calls=11262,usec=18138,usec_per_call=1.61,rejected_calls=0,failed_calls=0
redis-server[4139]: cmdstat_exec:calls=1,usec=277,usec_per_call=277.00,rejected_calls=0,failed_calls=0
redis-server[4139]: cmdstat_info:calls=165,usec=8398,usec_per_call=50.90,rejected_calls=0,failed_calls=0
redis-server[4139]: cmdstat_timeseries.HELLO:calls=7,usec=28,usec_per_call=4.00,rejected_calls=0,failed_calls=0
redis-server[4139]: 
redis-server[4139]: # Errorstats
redis-server[4139]: errorstat_CLUSTERDOWN:count=190
redis-server[4139]: errorstat_ERR:count=20
redis-server[4139]: errorstat_Multi-shard:count=3602
redis-server[4139]: 
redis-server[4139]: # Latencystats
redis-server[4139]: latency_percentiles_usec_del:p50=3.007,p99=192.511,p99.9=261.119
redis-server[4139]: latency_percentiles_usec_config|set:p50=4.015,p99=4.015,p99.9=4.015
redis-server[4139]: latency_percentiles_usec_config|get:p50=3.007,p99=54.015,p99.9=54.015
redis-server[4139]: latency_percentiles_usec_replconf:p50=1.003,p99=2.007,p99.9=7.007
redis-server[4139]: latency_percentiles_usec_slowlog|get:p50=1.003,p99=36.095,p99.9=44.031
redis-server[4139]: latency_percentiles_usec_pexpireat:p50=2.007,p99=21.119,p99.9=28.031
redis-server[4139]: latency_percentiles_usec_psync:p50=33.023,p99=41.215,p99.9=41.215
redis-server[4139]: latency_percentiles_usec_cluster|failover:p50=31.103,p99=31.103,p99.9=31.103
redis-server[4139]: latency_percentiles_usec_cluster|info:p50=51.199,p99=93.183,p99.9=96.255
redis-server[4139]: latency_percentiles_usec_cluster|replicate:p50=162.815,p99=162.815,p99.9=162.815
redis-server[4139]: latency_percentiles_usec_cluster|slots:p50=34.047,p99=57.087,p99.9=66.047
redis-server[4139]: latency_percentiles_usec_cluster|forget:p50=8.031,p99=114.175,p99.9=114.175
redis-server[4139]: latency_percentiles_usec_cluster|nodes:p50=81.407,p99=112.127,p99.9=115.199
redis-server[4139]: latency_percentiles_usec_cluster|meet:p50=27.007,p99=27.007,p99.9=27.007
redis-server[4139]: latency_percentiles_usec_acl|setuser:p50=8.031,p99=73.215,p99.9=73.215
redis-server[4139]: latency_percentiles_usec_acl|getuser:p50=2.007,p99=2.007,p99.9=2.007
redis-server[4139]: latency_percentiles_usec_expire:p50=3.007,p99=18.047,p99.9=55.039
redis-server[4139]: latency_percentiles_usec_auth:p50=5.023,p99=16.063,p99.9=16.063
redis-server[4139]: latency_percentiles_usec_json.set:p50=35.071,p99=171.007,p99.9=317.439
redis-server[4139]: latency_percentiles_usec_dbsize:p50=2.007,p99=2.007,p99.9=2.007
redis-server[4139]: latency_percentiles_usec_timeseries.REFRESHCLUSTER:p50=17.023,p99=95.231,p99.9=95.231
redis-server[4139]: latency_percentiles_usec_client|list:p50=84.479,p99=195.583,p99.9=230.399
redis-server[4139]: latency_percentiles_usec_ts.add:p50=18.047,p99=139.263,p99.9=226.303
redis-server[4139]: latency_percentiles_usec_ping:p50=1.003,p99=2.007,p99.9=3.007
redis-server[4139]: latency_percentiles_usec_select:p50=1.003,p99=1.003,p99.9=1.003
redis-server[4139]: latency_percentiles_usec_ts.mrange:p50=4849.663,p99=1002438.655,p99.9=1002438.655
redis-server[4139]: latency_percentiles_usec_timeseries.INNERCOMMUNICATION:p50=4.015,p99=26.111,p99.9=67.071
redis-server[4139]: latency_percentiles_usec_multi:p50=1.003,p99=1.003,p99.9=1.003
redis-server[4139]: latency_percentiles_usec_hset:p50=1.003,p99=9.023,p99.9=27.007
redis-server[4139]: latency_percentiles_usec_exec:p50=278.527,p99=278.527,p99.9=278.527
redis-server[4139]: latency_percentiles_usec_info:p50=28.031,p99=107.007,p99.9=138.239
redis-server[4139]: latency_percentiles_usec_timeseries.HELLO:p50=4.015,p99=8.031,p99.9=8.031
redis-server[4139]: 
redis-server[4139]: # Cluster
redis-server[4139]: cluster_enabled:1
redis-server[4139]: 
redis-server[4139]: # Keyspace
redis-server[4139]: db0:keys=324171,expires=305858,avg_ttl=1333546842
redis-server[4139]: 
redis-server[4139]: # Cluster info
redis-server[4139]: cluster_state:ok
redis-server[4139]: cluster_slots_assigned:16384
redis-server[4139]: cluster_slots_ok:16384
redis-server[4139]: cluster_slots_pfail:0
redis-server[4139]: cluster_slots_fail:0
redis-server[4139]: cluster_known_nodes:8
redis-server[4139]: cluster_size:4
redis-server[4139]: cluster_current_epoch:13
redis-server[4139]: cluster_my_epoch:12
redis-server[4139]: cluster_stats_messages_ping_sent:173035
redis-server[4139]: cluster_stats_messages_pong_sent:169821
redis-server[4139]: cluster_stats_messages_meet_sent:1
redis-server[4139]: cluster_stats_messages_fail_sent:7
redis-server[4139]: cluster_stats_messages_auth-req_sent:7
redis-server[4139]: cluster_stats_messages_auth-ack_sent:1
redis-server[4139]: cluster_stats_messages_mfstart_sent:1
redis-server[4139]: cluster_stats_messages_sent:342873
redis-server[4139]: cluster_stats_messages_ping_received:169814
redis-server[4139]: cluster_stats_messages_pong_received:172981
redis-server[4139]: cluster_stats_messages_auth-req_received:4
redis-server[4139]: cluster_stats_messages_auth-ack_received:4
redis-server[4139]: cluster_stats_messages_received:342803
redis-server[4139]: total_cluster_links_buffer_limit_exceeded:0
redis-server[4139]: ------ CLUSTER NODES OUTPUT ------
redis-server[4139]: 2c87be10c83a189a4dad0e2ea4348960f1326c47 10.100.2.66:6379@16379,,tls-port=0,shard-id=e0f97eca498185f0b7266de6033309455bfe71f8 master,fail - 1706287007020 1706287004503 11 disconnected
redis-server[4139]: 97cc25bce4dbc27f48f011b57fde090c1bad29ed 10.100.1.131:6379@16379,,tls-port=0,shard-id=1af5a690e30f4b31b3aa91cf453b1613a611d63c master - 0 1706290466185 9 connected 12288-16383
redis-server[4139]: a6f5899319752ae65867241fa725be7687242f6b 10.100.1.253:6379@16379,,tls-port=0,shard-id=de84cafefedfdf088c47fc69af77fb5e2e52b432 myself,master - 0 1706290464000 12 connected 0-4095
redis-server[4139]: 00ed0ef5357c4a0fe0f88d769d999c39f0813088 10.100.1.191:6379@16379,,tls-port=0,shard-id=1af5a690e30f4b31b3aa91cf453b1613a611d63c slave 97cc25bce4dbc27f48f011b57fde090c1bad29ed 0 1706290464000 9 connected
redis-server[4139]: bced7a5f993b14088621fbacc0e7c1f61b5179d3 10.100.0.128:6379@16379,,tls-port=0,shard-id=e0f97eca498185f0b7266de6033309455bfe71f8 master - 0 1706290464067 13 connected 8192-12287
redis-server[4139]: 3b851779ba28d923b66a514ada3ff24f1e43f551 10.100.2.75:6379@16379,,tls-port=0,shard-id=cd903426f7bc17b72f02638a2664da13b7c870bd master - 0 1706290465000 10 connected 4096-8191
redis-server[4139]: 476061950c6809413166c94e9d93044e23bd9c49 10.100.0.197:6379@16379,,tls-port=0,shard-id=de84cafefedfdf088c47fc69af77fb5e2e52b432 slave a6f5899319752ae65867241fa725be7687242f6b 0 1706290465580 12 connected
redis-server[4139]: 1d030f4d1730252e95c17a430d09850ad12071cc 10.100.0.191:6379@16379,,tls-port=0,shard-id=cd903426f7bc17b72f02638a2664da13b7c870bd slave 3b851779ba28d923b66a514ada3ff24f1e43f551 0 1706290465176 10 connected
redis-server[4139]: ------ CLIENT LIST OUTPUT ------
redis-server[4139]: id=10132 addr=10.250.0.155:37616 laddr=10.100.1.253:6379 fd=50 name= age=221 idle=221 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=4 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=NULL user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10029 addr=10.250.1.202:7936 laddr=10.100.1.253:6379 fd=54 name= age=2625 idle=222 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=expire user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10230 addr=10.250.2.230:26150 laddr=10.100.1.253:6379 fd=123 name= age=176 idle=0 flags=b db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=174 multi-mem=0 rbs=1024 rbp=178 obl=0 oll=0 omem=0 tot-mem=22678 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10231 addr=10.250.2.230:33953 laddr=10.100.1.253:6379 fd=133 name= age=176 idle=1 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=178 obl=0 oll=0 omem=0 tot-mem=22400 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10232 addr=10.250.0.14:7452 laddr=10.100.1.253:6379 fd=134 name= age=176 idle=1 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=170 obl=0 oll=0 omem=0 tot-mem=22400 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10233 addr=10.250.2.230:10611 laddr=10.100.1.253:6379 fd=135 name= age=176 idle=1 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=176 obl=0 oll=0 omem=0 tot-mem=22400 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10234 addr=10.250.1.109:11208 laddr=10.100.1.253:6379 fd=127 name= age=176 idle=1 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=178 obl=0 oll=0 omem=0 tot-mem=22400 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10235 addr=10.250.1.109:27504 laddr=10.100.1.253:6379 fd=136 name= age=176 idle=1 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=171 obl=0 oll=0 omem=0 tot-mem=22400 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10236 addr=10.250.0.14:19212 laddr=10.100.1.253:6379 fd=132 name= age=176 idle=1 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=176 obl=0 oll=0 omem=0 tot-mem=22400 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10237 addr=10.250.2.230:13529 laddr=10.100.1.253:6379 fd=137 name= age=176 idle=1 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=180 obl=0 oll=0 omem=0 tot-mem=22400 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10238 addr=10.250.0.14:53997 laddr=10.100.1.253:6379 fd=138 name= age=176 idle=1 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=184 obl=0 oll=0 omem=0 tot-mem=22400 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10147 addr=10.250.0.14:8160 laddr=10.100.1.253:6379 fd=57 name= age=178 idle=178 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10148 addr=10.250.1.109:27002 laddr=10.100.1.253:6379 fd=68 name= age=178 idle=178 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10149 addr=10.250.0.14:7165 laddr=10.100.1.253:6379 fd=69 name= age=178 idle=178 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10150 addr=10.250.0.14:10997 laddr=10.100.1.253:6379 fd=70 name= age=178 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10151 addr=10.250.0.14:41776 laddr=10.100.1.253:6379 fd=60 name= age=178 idle=178 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10152 addr=10.250.0.14:12127 laddr=10.100.1.253:6379 fd=71 name= age=178 idle=178 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10153 addr=10.250.0.14:22003 laddr=10.100.1.253:6379 fd=72 name= age=178 idle=178 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10154 addr=10.250.2.230:21409 laddr=10.100.1.253:6379 fd=73 name= age=178 idle=178 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10155 addr=10.100.1.131:37418 laddr=10.100.1.253:6379 fd=77 name= age=178 idle=0 flags=b db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=5257 qbuf-free=191341 argv-mem=163 multi-mem=0 rbs=1024 rbp=5 obl=0 oll=0 omem=0 tot-mem=198739 events=r cmd=timeseries.INNERCOMMUNICATION user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10156 addr=10.100.2.75:56570 laddr=10.100.1.253:6379 fd=78 name= age=178 idle=0 flags=b db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=196598 argv-mem=163 multi-mem=0 rbs=1024 rbp=5 obl=0 oll=0 omem=0 tot-mem=198739 events=r cmd=timeseries.INNERCOMMUNICATION user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10157 addr=10.100.0.128:57566 laddr=10.100.1.253:6379 fd=79 name= age=178 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=229366 argv-mem=0 multi-mem=0 rbs=1024 rbp=5 obl=0 oll=0 omem=0 tot-mem=231296 events=r cmd=timeseries.INNERCOMMUNICATION user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10239 addr=10.250.1.109:40288 laddr=10.100.1.253:6379 fd=139 name= age=176 idle=1 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=172 obl=0 oll=0 omem=0 tot-mem=22400 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=12892 addr=10.250.1.109:23763 laddr=10.100.1.253:6379 fd=36 name= age=12 idle=1 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=22400 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10240 addr=10.250.2.230:34121 laddr=10.100.1.253:6379 fd=140 name= age=176 idle=1 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=175 obl=0 oll=0 omem=0 tot-mem=22400 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10241 addr=10.250.0.14:17201 laddr=10.100.1.253:6379 fd=141 name= age=176 idle=12 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10158 addr=10.250.0.14:45862 laddr=10.100.1.253:6379 fd=80 name= age=178 idle=178 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10159 addr=10.250.2.230:19107 laddr=10.100.1.253:6379 fd=61 name= age=178 idle=178 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10134 addr=10.250.0.155:38559 laddr=10.100.1.253:6379 fd=55 name= age=221 idle=220 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=expire user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10119 addr=10.250.0.14:3845 laddr=10.100.1.253:6379 fd=37 name= age=242 idle=237 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=expire user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=257 addr=10.100.2.35:48138 laddr=10.100.1.253:6379 fd=31 name= age=67159 idle=39 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=cluster|slots user=admin redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10053 addr=10.250.2.229:15664 laddr=10.100.1.253:6379 fd=48 name= age=2023 idle=222 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=expire user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10099 addr=10.250.2.230:29336 laddr=10.100.1.253:6379 fd=21 name= age=844 idle=240 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=expire user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10164 addr=10.250.0.14:42898 laddr=10.100.1.253:6379 fd=59 name= age=178 idle=178 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10165 addr=10.250.0.14:43416 laddr=10.100.1.253:6379 fd=81 name= age=178 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=258 addr=10.100.2.35:48146 laddr=10.100.1.253:6379 fd=32 name= age=67159 idle=14 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=cluster|slots user=admin redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10135 addr=10.250.2.229:32859 laddr=10.100.1.253:6379 fd=56 name= age=220 idle=220 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=expire user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10199 addr=10.250.1.109:16925 laddr=10.100.1.253:6379 fd=106 name= age=177 idle=176 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10200 addr=10.250.1.109:33709 laddr=10.100.1.253:6379 fd=107 name= age=177 idle=176 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10201 addr=10.250.2.230:8189 laddr=10.100.1.253:6379 fd=108 name= age=177 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10202 addr=10.250.1.109:41451 laddr=10.100.1.253:6379 fd=104 name= age=177 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10203 addr=10.250.0.14:7166 laddr=10.100.1.253:6379 fd=109 name= age=177 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10204 addr=10.250.1.109:10481 laddr=10.100.1.253:6379 fd=110 name= age=177 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=4 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=NULL user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10205 addr=10.250.2.230:37491 laddr=10.100.1.253:6379 fd=111 name= age=177 idle=176 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10127 addr=10.250.1.109:32849 laddr=10.100.1.253:6379 fd=46 name= age=240 idle=240 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=4 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=NULL user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10128 addr=10.250.0.14:44698 laddr=10.100.1.253:6379 fd=47 name= age=240 idle=240 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=4 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=NULL user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10129 addr=10.250.2.230:42200 laddr=10.100.1.253:6379 fd=51 name= age=240 idle=240 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=4 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=NULL user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10166 addr=10.250.1.109:45152 laddr=10.100.1.253:6379 fd=82 name= age=178 idle=178 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10167 addr=10.250.2.230:64668 laddr=10.100.1.253:6379 fd=83 name= age=178 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10168 addr=10.250.0.14:24798 laddr=10.100.1.253:6379 fd=84 name= age=178 idle=178 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=4 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=NULL user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=259 addr=10.100.2.35:48152 laddr=10.100.1.253:6379 fd=33 name= age=67159 idle=9 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=cluster|slots user=admin redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10032 addr=10.250.2.229:49691 laddr=10.100.1.253:6379 fd=58 name= age=2623 idle=221 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=expire user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10206 addr=10.250.0.14:20769 laddr=10.100.1.253:6379 fd=112 name= age=177 idle=176 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10207 addr=10.250.0.14:32814 laddr=10.100.1.253:6379 fd=105 name= age=177 idle=176 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10208 addr=10.250.1.109:58468 laddr=10.100.1.253:6379 fd=113 name= age=177 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10209 addr=10.250.2.230:19631 laddr=10.100.1.253:6379 fd=114 name= age=177 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10210 addr=10.250.1.109:18699 laddr=10.100.1.253:6379 fd=115 name= age=177 idle=176 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10123 addr=10.250.0.14:18189 laddr=10.100.1.253:6379 fd=42 name= age=241 idle=241 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=4 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=NULL user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10124 addr=10.250.0.14:18185 laddr=10.100.1.253:6379 fd=43 name= age=241 idle=241 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=4 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=NULL user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=260 addr=10.100.2.35:48160 laddr=10.100.1.253:6379 fd=34 name= age=67159 idle=24 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=cluster|slots user=admin redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10170 addr=10.250.0.14:36648 laddr=10.100.1.253:6379 fd=86 name= age=178 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=85 addr=10.100.0.197:39066 laddr=10.100.1.253:6379 fd=28 name= age=70924 idle=0 flags=S db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=1 omem=20504 tot-mem=42904 events=r cmd=replconf user=admin redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10118 addr=10.250.2.230:59998 laddr=10.100.1.253:6379 fd=29 name= age=242 idle=242 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=4 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=NULL user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10077 addr=10.250.2.230:6648 laddr=10.100.1.253:6379 fd=40 name= age=1440 idle=241 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=expire user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10211 addr=10.250.0.14:27674 laddr=10.100.1.253:6379 fd=116 name= age=177 idle=176 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10212 addr=10.250.2.230:15846 laddr=10.100.1.253:6379 fd=117 name= age=177 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10213 addr=10.250.1.109:63252 laddr=10.100.1.253:6379 fd=118 name= age=177 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=4 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=NULL user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10215 addr=10.250.1.109:54042 laddr=10.100.1.253:6379 fd=120 name= age=177 idle=176 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10216 addr=10.250.2.230:50948 laddr=10.100.1.253:6379 fd=121 name= age=177 idle=1 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=22400 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10125 addr=10.250.0.14:40893 laddr=10.100.1.253:6379 fd=44 name= age=241 idle=240 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=expire user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10171 addr=10.250.0.14:11780 laddr=10.100.1.253:6379 fd=87 name= age=178 idle=178 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10173 addr=10.250.2.230:22962 laddr=10.100.1.253:6379 fd=88 name= age=178 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10174 addr=10.250.0.14:32743 laddr=10.100.1.253:6379 fd=89 name= age=178 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10130 addr=10.250.2.230:37482 laddr=10.100.1.253:6379 fd=52 name= age=239 idle=239 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=4 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=NULL user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=9986 addr=10.250.1.109:9991 laddr=10.100.1.253:6379 fd=227 name= age=3229 idle=241 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=expire user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=256 addr=10.100.2.35:48126 laddr=10.100.1.253:6379 fd=15 name= age=67159 idle=19 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=cluster|slots user=admin redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10121 addr=10.250.0.14:47009 laddr=10.100.1.253:6379 fd=39 name= age=241 idle=239 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=expire user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10131 addr=10.250.1.202:1203 laddr=10.100.1.253:6379 fd=49 name= age=222 idle=222 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=4 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=NULL user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10217 addr=10.250.1.109:1484 laddr=10.100.1.253:6379 fd=122 name= age=177 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10122 addr=10.250.0.14:27987 laddr=10.100.1.253:6379 fd=41 name= age=241 idle=241 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=4 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=NULL user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10176 addr=10.250.1.109:7473 laddr=10.100.1.253:6379 fd=90 name= age=177 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10177 addr=10.250.0.14:64213 laddr=10.100.1.253:6379 fd=91 name= age=177 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10178 addr=10.250.0.14:2337 laddr=10.100.1.253:6379 fd=92 name= age=177 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10179 addr=10.250.1.109:14558 laddr=10.100.1.253:6379 fd=93 name= age=177 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10180 addr=10.250.0.14:47788 laddr=10.100.1.253:6379 fd=85 name= age=177 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10181 addr=10.250.0.14:50418 laddr=10.100.1.253:6379 fd=94 name= age=177 idle=176 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10182 addr=10.250.2.230:46624 laddr=10.100.1.253:6379 fd=95 name= age=177 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10183 addr=10.250.0.14:53335 laddr=10.100.1.253:6379 fd=96 name= age=177 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10219 addr=10.250.2.230:14096 laddr=10.100.1.253:6379 fd=124 name= age=177 idle=1 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=22400 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10220 addr=10.250.2.230:53469 laddr=10.100.1.253:6379 fd=125 name= age=176 idle=0 flags=b db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=174 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=22678 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10221 addr=10.250.2.230:60487 laddr=10.100.1.253:6379 fd=126 name= age=176 idle=1 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=22400 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10184 addr=10.250.0.14:14370 laddr=10.100.1.253:6379 fd=97 name= age=177 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10133 addr=10.250.2.229:17700 laddr=10.100.1.253:6379 fd=53 name= age=221 idle=219 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=expire user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10141 addr=10.250.2.230:5283 laddr=10.100.1.253:6379 fd=62 name= age=178 idle=178 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10142 addr=10.250.2.230:28654 laddr=10.100.1.253:6379 fd=63 name= age=178 idle=178 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10143 addr=10.250.1.109:64614 laddr=10.100.1.253:6379 fd=64 name= age=178 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10144 addr=10.250.2.230:9940 laddr=10.100.1.253:6379 fd=65 name= age=178 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10145 addr=10.250.1.109:27966 laddr=10.100.1.253:6379 fd=66 name= age=178 idle=178 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10185 addr=10.250.1.109:55694 laddr=10.100.1.253:6379 fd=98 name= age=177 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10186 addr=10.250.1.109:63602 laddr=10.100.1.253:6379 fd=99 name= age=177 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10223 addr=10.250.0.14:15751 laddr=10.100.1.253:6379 fd=128 name= age=176 idle=0 flags=b db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=174 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=22678 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10224 addr=10.250.2.230:12418 laddr=10.100.1.253:6379 fd=129 name= age=176 idle=1 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=22400 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10225 addr=10.250.0.14:43900 laddr=10.100.1.253:6379 fd=130 name= age=176 idle=1 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=22400 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10228 addr=10.250.1.109:50810 laddr=10.100.1.253:6379 fd=131 name= age=176 idle=1 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=22400 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10187 addr=10.250.0.14:5049 laddr=10.100.1.253:6379 fd=100 name= age=177 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10188 addr=10.250.0.14:12538 laddr=10.100.1.253:6379 fd=101 name= age=177 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10189 addr=10.250.2.230:43423 laddr=10.100.1.253:6379 fd=102 name= age=177 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10190 addr=10.250.1.109:25350 laddr=10.100.1.253:6379 fd=103 name= age=177 idle=177 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10146 addr=10.250.2.230:26617 laddr=10.100.1.253:6379 fd=67 name= age=178 idle=178 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=ts.mrange user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10117 addr=10.250.1.109:43366 laddr=10.100.1.253:6379 fd=20 name= age=242 idle=240 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=expire user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10126 addr=10.250.1.109:47022 laddr=10.100.1.253:6379 fd=45 name= age=241 idle=241 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=4 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=NULL user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: id=10120 addr=10.250.0.14:17099 laddr=10.100.1.253:6379 fd=38 name= age=241 idle=239 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1928 events=r cmd=expire user=default redir=-1 resp=2 lib-name= lib-ver=
redis-server[4139]: ------ MODULES INFO OUTPUT ------
redis-server[4139]: # ReJSON_trace
redis-server[4139]: ReJSON_backtrace:   0: redis_module::basic_info_command_handler
redis-server[4139]:    1: rejson::__info_func
redis-server[4139]:    2: modulesCollectInfo
redis-server[4139]:              at /__w/redis-stack/redis-stack/redis/src/module.c:10293:9
redis-server[4139]:    3: logModulesInfo
redis-server[4139]:              at /__w/redis-stack/redis-stack/redis/src/debug.c:1900:22
redis-server[4139]:       printCrashReport
redis-server[4139]:              at /__w/redis-stack/redis-stack/redis/src/debug.c:2183:5
redis-server[4139]:    4: sigsegvHandler
redis-server[4139]:              at /__w/redis-stack/redis-stack/redis/src/debug.c:2164:5
redis-server[4139]:    5: <unknown>
redis-server[4139]:    6: __pthread_kill_implementation
redis-server[4139]:    7: raise
redis-server[4139]:    8: abort
redis-server[4139]:    9: __assert_fail_base.cold
redis-server[4139]:   10: __assert_fail
redis-server[4139]:   11: _sdsnewlen.constprop.0
redis-server[4139]:              at /__w/redis-stack/redis-stack/redis/src/sds.c:115:5
redis-server[4139]:   12: sdsnewlen
redis-server[4139]:              at /__w/redis-stack/redis-stack/redis/src/sds.c:170:12
redis-server[4139]:       createRawStringObject
redis-server[4139]:              at /__w/redis-stack/redis-stack/redis/src/object.c:86:37
redis-server[4139]:       createStringObject
redis-server[4139]:              at /__w/redis-stack/redis-stack/redis/src/object.c:127:16
redis-server[4139]:   13: RM_CreateString
redis-server[4139]:              at /__w/redis-stack/redis-stack/redis/src/module.c:2531:28
redis-server[4139]:   14: SerializationCtxReadeRedisString
redis-server[4139]:              at /build/src/libmr_integration.c:161:12
redis-server[4139]:       SeriesRecord_Deserialize
redis-server[4139]:              at /build/src/libmr_integration.c:947:33
redis-server[4139]:   15: MR_RecordDeSerialize
redis-server[4139]:              at /build/deps/LibMR/src/record.c:110:17
redis-server[4139]:   16: ListRecord_Deserialize
redis-server[4139]:              at /build/src/libmr_integration.c:843:9
redis-server[4139]:   17: MR_RecordDeSerialize
redis-server[4139]:              at /build/deps/LibMR/src/record.c:110:17
redis-server[4139]:   18: MR_SetRecord
redis-server[4139]:              at /build/deps/LibMR/src/mr.c:600:17
redis-server[4139]:   19: MR_ExecutionMain
redis-server[4139]:              at /build/deps/LibMR/src/mr.c:1213:5
redis-server[4139]:   20: thread_do
redis-server[4139]:              at /build/deps/LibMR/src/utils/thpool.c:363:9
redis-server[4139]:   21: start_thread
redis-server[4139]:   22: __clone3
redis-server[4139]: 
redis-server[4139]: 
redis-server[4139]: # search_version
redis-server[4139]: search_version:2.8.11
redis-server[4139]: search_redis_version:7.2.4 - oss
redis-server[4139]: 
redis-server[4139]: # search_index
redis-server[4139]: search_number_of_indexes:1
redis-server[4139]: 
redis-server[4139]: # search_fields_statistics
redis-server[4139]: search_fields_numeric:Numeric=3,Sortable=2
redis-server[4139]: search_fields_tag:Tag=5
redis-server[4139]: search_fields_vector:Vector=1,HNSW=1
redis-server[4139]: 
redis-server[4139]: # search_dialect_statistics
redis-server[4139]: search_dialect_1:0
redis-server[4139]: search_dialect_2:0
redis-server[4139]: search_dialect_3:0
redis-server[4139]: search_dialect_4:0
redis-server[4139]: 
redis-server[4139]: # search_runtime_configurations
redis-server[4139]: search_concurrent_mode:OFF
redis-server[4139]: search_enableGC:ON
redis-server[4139]: search_minimal_term_prefix:2
redis-server[4139]: search_maximal_prefix_expansions:200
redis-server[4139]: search_query_timeout_ms:500
redis-server[4139]: search_timeout_policy:return
redis-server[4139]: search_cursor_read_size:1000
redis-server[4139]: search_cursor_max_idle_time:300000
redis-server[4139]: search_max_doc_table_size:1000000
redis-server[4139]: search_max_search_results:1000000
redis-server[4139]: search_max_aggregate_results:-1
redis-server[4139]: search_search_pool_size:20
redis-server[4139]: search_index_pool_size:8
redis-server[4139]: search_gc_scan_size:100
redis-server[4139]: search_min_phonetic_term_length:3
redis-server[4139]: ------ CONFIG DEBUG OUTPUT ------
redis-server[4139]: lazyfree-lazy-eviction no
redis-server[4139]: activedefrag no
redis-server[4139]: lazyfree-lazy-expire no
redis-server[4139]: repl-diskless-load disabled
redis-server[4139]: lazyfree-lazy-user-flush no
redis-server[4139]: list-compress-depth 0
redis-server[4139]: sanitize-dump-payload no
redis-server[4139]: lazyfree-lazy-user-del no
redis-server[4139]: replica-read-only yes
redis-server[4139]: repl-diskless-sync yes
redis-server[4139]: io-threads 1
redis-server[4139]: lazyfree-lazy-server-del no
redis-server[4139]: client-query-buffer-limit 1gb
redis-server[4139]: io-threads-do-reads no
redis-server[4139]: proto-max-bulk-len 512mb
redis-server[4139]: slave-read-only yes
redis-server[4139]: ------ FAST MEMORY TEST ------
redis-server[4139]: 4139:M 26 Jan 2024 17:34:26.287 # main thread terminated
redis-server[4139]: 4139:M 26 Jan 2024 17:34:26.287 # Bio worker thread #0 terminated
redis-server[4139]: 4139:M 26 Jan 2024 17:34:26.287 # Bio worker thread #1 terminated
redis-server[4139]: 4139:M 26 Jan 2024 17:34:26.288 # Bio worker thread #2 terminated
redis-server[4139]: Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.
redis-server[4139]: ------ DUMPING CODE AROUND EIP ------
redis-server[4139]: Symbol: (null) (base: (nil))
redis-server[4139]: Module: /lib64/libc.so.6 (base 0x7f38c2800000)
redis-server[4139]: $ xxd -r -p /tmp/dump.hex /tmp/dump.bin
redis-server[4139]: $ objdump --adjust-vma=(nil) -D -b binary -m i386:x86-64 /tmp/dump.bin
redis-server[4139]: ------
redis-server[4139]: === REDIS BUG REPORT END. Make sure to include from START to END. ===
@tezc
Copy link
Collaborator

tezc commented Apr 19, 2024

Previously reported here: RedisJSON/RedisJSON#1179

@cletustboone Thank you for opening it here. If it is happening again, please share logs if possible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@cletustboone @tezc