Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS RedisCache Failover requests #5890

Closed
musicallyvk opened this issue May 20, 2024 · 5 comments
Closed

AWS RedisCache Failover requests #5890

musicallyvk opened this issue May 20, 2024 · 5 comments
Labels

Comments

@musicallyvk
Copy link

Hi Everyone, Hope everyone is doing great!
so i have a 3 replica and 1 primary on aws elasticache redisCache using version 4.0, with the redis config file on the application level which has a load balancer with round robbin and using replicatedServersConfig. and many other things like retry , failedSlaveCheckInterval,timeout, readMode , subscriptionMode and many things.

if you need more details on config then -

des.redisson.config={"replicatedServersConfig":{"idleConnectionTimeout":10000,"connectTimeout":10000,"timeout":30000,"retryAttempts":3,"retryInterval":1500,"failedSlaveReconnectionInterval":3000,"failedSlaveCheckInterval":60000,"password":"*****","subscriptionsPerConnection":5,"clientName":"myClientName","loadBalancer":{"class":"org.redisson.connection.balancer.RoundRobinLoadBalancer"},"subscriptionConnectionMinimumIdleSize":1,"subscriptionConnectionPoolSize":50,"slaveConnectionMinimumIdleSize":24,"slaveConnectionPoolSize":64,"masterConnectionMinimumIdleSize":24,"masterConnectionPoolSize":64,"readMode":"SLAVE","subscriptionMode":"SLAVE","nodeAddresses":["rediss://Node1,Node2,Node3,Node4"],"scanInterval":1000},"threads":16,"nettyThreads":32,"codec":{"class":"org.redisson.codec.LZ4Codec"},"transportMode":"NIO"}

now the thing is if a failover happens then some requests are still being routed to that failed node, how to get it resolved, anyone please help me. i am stuck with this for months.
Thanks Already !

@mrniko
Copy link
Member

mrniko commented May 20, 2024

Hi,

Which Redisson version is it?

@musicallyvk
Copy link
Author

Hi, Thanks For replying,

on application, the pom.xml says
3.13.2

and on aws - the reds cache is using Engine version
4.0.10

@mrniko
Copy link
Member

mrniko commented May 20, 2024

Can you share Redisson logs?

@musicallyvk
Copy link
Author

musicallyvk commented May 20, 2024

this is the exception i see on splunk logs, if not this then
what exactly you need

2024-05-19 16:23:01.227 [redisson-netty-2-15] [ERROR] intuit_tid= correlationId= http_request_path= http_query_string= intuit_offeringid= intuit_docid= appid= authid= vendor= app_class=org.redisson.connection.ReplicatedConnectionManager app_method=: - org.redisson.client.RedisConnectionException: io.netty.resolver.dns.DnsResolveContext$SearchDomainUnknownHostException: Failed to resolve 'myNodeName-004.myNodeName.dnqsqq.usw2.cache.amazonaws.com' [A(1)] and search domain query for configured domains failed as well: [fdp-docmgmt-comprehensionprovidersvc-usw2-qal.svc.cluster.local, svc.cluster.local, cluster.local]
java.util.concurrent.CompletionException: org.redisson.client.RedisConnectionException: io.netty.resolver.dns.DnsResolveContext$SearchDomainUnknownHostException: Failed to resolve 'drertyredisecachee2ea-04.dertettyydisecacheea.dnqsqq.usw2.cache.amazonaws.com' [A(1)] and search domain query for configured domains failed as well: [fdp-docmgmt-comprehensionprovidersvc-usw2-qal.svc.cluster.local, svc.cluster.local, cluster.local]

@mrniko
Copy link
Member

mrniko commented May 21, 2024

#5726 (comment)

@mrniko mrniko closed this as completed Jun 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

No branches or pull requests

2 participants