Does not work cluster mode if I use single ip address of k8s?

[Park-KwonSoo]

Hello,

I am currently utilizing k8s to deploy and manage multiple Redis pods. In the current development environment, we have about three Redis pods running, all connected via a single endpoint. That is, within k8s, we have a host named 'redis'. When accessing this single endpoint, k8s handles the load balancing across the various Redis pods.

Therefore, when using rueidis, I've been specifying only the single endpoint in InitAddress. However, it seems there are occasional issues when invoking operations like HSet.

For debugging purposes, I utilized the Nodes method of rueidis's ClusterClient, and found that the number of Redis nodes being output is four instead of three. Among these, one appears to be registered with the endpoint address instead of the actual Redis IP.

To illustrate, if the actual IPs of the Redis pods are 1.2.3.4:6379, 1.2.3.5:6379, and 1.2.3.6:6379, and the endpoint set to access them is redis:6379, then when registering []host{"redis:6379"} in InitAddress, the keys in the map output from the client's Nodes are 1.2.3.4:6379, 1.2.3.5:6379, 1.2.3.6:6379, and redis:6379.

Could this be the reason certain operations like HSet are not functioning properly? I would greatly appreciate it if you could check this for me.

Thank you!

[rueian]

Hi @simonprickett,

The address appeared in the .Nodes() output is excepted and it is only used for refreshing shard information. So that can’t be the reason for certain operations that are not working properly.

What kind of errors you saw when doing HSet?

[rueian]

Hi @Park-KwonSoo,

Thanks. That result of CLUSTER SLOTS looks normal to me.

Then what responses did you get from the HSet that were considered failures?
And what responses did you get from the HGet that were considered failures?

[Park-KwonSoo]

Thank you for checking.
Firstly, there is no error message displayed when performing HSet. It seems to be working correctly.
The issue arises intermittently when performing HGet, resulting in a Redis.Nil error.
Upon directly connecting to Redis and using the redis-cli to query with HGet or Hgetall, I was able to confirm the presence of the respective key-field-value.
When I accessed each Redis pod and observed the incoming commands using the monitor command, there was no delete operation, like the del op, for that particular key.

[rueian]

Hi @Park-KwonSoo, could you observe the problematic HGET/HGETALL from the server side using the monitor command? I know this is almost impossible but for now, I don't have more information for further debugging.

By the way, do all your redis nodes have the same response of cluster slots?

[rueian]

Hi @Park-KwonSoo, is there any update?

[Park-KwonSoo]

Oh sorry, I missed your response while I was looking into another issue. To get straight to the point, there were no changes made, but the issue isn't occurring anymore. My guess is that either the Redis we're using had a temporary glitch, or there was some network issue, or the client was making an incorrect call - one of those three, but I'm not certain. I tried various methods for debugging, but the issue stopped occurring before I could accurately pinpoint it, so I've stopped tracking it for now.
Thank you so so much for your help, and I really appreciate you providing the useful driver!
Have a great day :)