PerfTest said: Temporary failure in name resolution

[xixiangzouyibian]

Hi,

I deployed a rabbitmq cluster by helm recently(provided by bitnami/rabbitmq).
My linux os: Ubuntu 20.04
My kubectl version: 1.23

I run command: # kubectl run perf-test --image=pivotalrabbitmq/perf-test -- --uri "amqp://${username}:${password}@${service}".
And fill above ${service} as "hrabbitmq" (rabbitmq cluster instance name).
I want to startup perfTest as a pod to test with my rabbitmq cluster:

Cluster name: rabbit@hrabbitmq-0.hrabbitmq-headless.default.svc.cluster.local

Disk Nodes

rabbit@hrabbitmq-0.hrabbitmq-headless
rabbit@hrabbitmq-1.hrabbitmq-headless

Running Nodes

rabbit@hrabbitmq-0.hrabbitmq-headless
rabbit@hrabbitmq-1.hrabbitmq-headless

Versions

rabbit@hrabbitmq-0.hrabbitmq-headless: RabbitMQ 3.10.5 on Erlang 24.3.4
rabbit@hrabbitmq-1.hrabbitmq-headless: RabbitMQ 3.10.5 on Erlang 24.3.4

Maintenance status

Node: rabbit@hrabbitmq-0.hrabbitmq-headless, status: not under maintenance
Node: rabbit@hrabbitmq-1.hrabbitmq-headless, status: not under maintenance

But logs pod and get below error:

java.net.UnknownHostException: hrabbitmq: Temporary failure in name resolution
at java.base/java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
at java.base/java.net.InetAddress$PlatformNameService.lookupAllHostAddr(InetAddress.java:933)
at java.base/java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1519)
at java.base/java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:852)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1509)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1367)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1301)
at com.rabbitmq.client.DnsRecordIpAddressResolver.resolveIpAddresses(DnsRecordIpAddressResolver.java:83)
at com.rabbitmq.client.DnsRecordIpAddressResolver.getAddresses(DnsRecordIpAddressResolver.java:73)
at com.rabbitmq.client.impl.recovery.RecoveryAwareAMQConnectionFactory.newConnection(RecoveryAwareAMQConnectionFactory.java:58)
at com.rabbitmq.client.impl.recovery.AutorecoveringConnection.init(AutorecoveringConnection.java:160)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1216)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1173)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1071)
at com.rabbitmq.perf.MulticastSet$ConnectionCreator.createConfigurationConnections(MulticastSet.java:863)
at com.rabbitmq.perf.MulticastSet.createConfigurationConnections(MulticastSet.java:355)
at com.rabbitmq.perf.MulticastSet.run(MulticastSet.java:207)
at com.rabbitmq.perf.PerfTest.main(PerfTest.java:418)
at com.rabbitmq.perf.PerfTest.main(PerfTest.java:542)

I want to figure out if wrong amqp uri provided or other reasons like pod network issue.
Could please help to suggest if anything is missing ? It's been bothering me for a long time, many thanks for any suggestions.

======
Updated:

"hrabbitmq-0.hrabbitmq-headless.default.svc.cluster.local" is still not working.
Sure, I will diagnose further more.
Maybe the article: https://www.rabbitmq.com/kubernetes/operator/using-operator.html#verify-instance should consider DNS issues in this section, to save more beginners time....
Any way, thanks Luke and Micheal, I'll update if I find something new.

[michaelklishin]

Something is up with hostname resolution on the host where you run PerfTest. I'm afraid we can't offer help on general networking issues beyond what's available in the Troubleshooting Networking guide.

[xixiangzouyibian]

Hi Michael, Thanks for your response, is there any quick fix as workaround to solve this problem.

[michaelklishin]

There is a good understanding of what the error tells you. We won't be able to diagnose a DNS/hostname resolution issue for you and reasons may vary. Please take it from here.

[lukebakken]

This error...

java.net.UnknownHostException: hrabbitmq: Temporary failure in name resolution

... means that the host name hrabbimq does not resolve in DNS. This is expected because you do not have any hosts with that name.

Your options:

• Use a load balancer to distribute connections to your two RabbitMQ nodes. If you give the load balancer the DNS/hostname hrabbimq, your PerfTest command should work.
• Update your PerfTest arguments to specify both RabbitMQ pods:

  --uris "amqp://{username}:{password}@hrabbitmq-0.hrabbitmq-headless,amqp://{username}:{password}@hrabbitmq-1.hrabbitmq-headless"
  

[xixiangzouyibian]

Hi Luke, Thanks your time for supporting,
After tried: kubectl run perf-test --image=pivotalrabbitmq/perf-test -- --uris "amqp://user:6CPcNqeVo6yiwVED@hrabbitmq-0.hrabbitmq-headless,amqp://user:6CPcNqeVo6yiwVED@hrabbitmq-1.hrabbitmq-headless"

And the error shows:

Main thread caught exception: java.net.UnknownHostException: hrabbitmq-0.hrabbitmq-headless: Temporary failure in name resolution
14:22:14.991 [main] ERROR com.rabbitmq.perf.PerfTest - Main thread caught exception
java.net.UnknownHostException: hrabbitmq-0.hrabbitmq-headless: Temporary failure in name resolution
at java.base/java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
at java.base/java.net.InetAddress$PlatformNameService.lookupAllHostAddr(InetAddress.java:933)
at java.base/java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1519)
at java.base/java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:852)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1509)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1367)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1301)
at com.rabbitmq.client.DnsRecordIpAddressResolver.resolveIpAddresses(DnsRecordIpAddressResolver.java:83)
at com.rabbitmq.client.DnsRecordIpAddressResolver.getAddresses(DnsRecordIpAddressResolver.java:73)
at com.rabbitmq.client.impl.recovery.RecoveryAwareAMQConnectionFactory.newConnection(RecoveryAwareAMQConnectionFactory.java:58)
at com.rabbitmq.client.impl.recovery.AutorecoveringConnection.init(AutorecoveringConnection.java:160)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1216)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1173)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1071)
at com.rabbitmq.perf.MulticastSet$ConnectionCreator.createConfigurationConnections(MulticastSet.java:863)
at com.rabbitmq.perf.MulticastSet.createConfigurationConnections(MulticastSet.java:355)
at com.rabbitmq.perf.MulticastSet.run(MulticastSet.java:207)
at com.rabbitmq.perf.PerfTest.main(PerfTest.java:418)
at com.rabbitmq.perf.PerfTest.main(PerfTest.java:542)

[lukebakken]

The error is clear - the host name hrabbitmq-0.hrabbitmq-headless does not resolve in DNS. You're going to have to diagnose this yourself.

Perhaps the actual host names you need to use are:

• hrabbitmq-0.hrabbitmq-headless.default.svc.cluster.local
• hrabbitmq-1.hrabbitmq-headless.default.svc.cluster.local