MG-2235 - Check gRPC Service is Healthy During Setup

[rodneyosodo]

Signed-off-by: Rodney Osodo 28790446+rodneyosodo@users.noreply.github.com

What type of PR is this?

This is a bug fix because it fixes the following issue: #2235

What does this do?

When connecting to the grpc server ensure the server is healthy before returning AuthClient during setup

Which issue(s) does this PR fix/relate to?

• Resolves Feature: Detect problems with Auth gRPC connection during setup #2235

Have you included tests for your changes?

Yes, I have modified testes and tested manually

The logs below are for test where I simulated the things auth server is unhealthy

go func() {
	if s.Name == "things" {
		s.health.SetServingStatus(s.Name, grpchealth.HealthCheckResponse_NOT_SERVING)
		time.Sleep(20 * time.Second)
		s.health.SetServingStatus(s.Name, grpchealth.HealthCheckResponse_SERVING)
	}
}()

Logs

2024/05/20 13:05:03 The binary was build using Nats as the message broker
2024/05/20 13:05:03 The binary was build using Nats as the message broker
{"time":"2024-05-20T13:05:07.477911143Z","level":"ERROR","msg":"health check failed : service is not serving"}
2024/05/20 13:05:08 The binary was build using Nats as the message broker
2024/05/20 13:05:08 The binary was build using Nats as the message broker
{"time":"2024-05-20T13:05:08.540289122Z","level":"ERROR","msg":"health check failed : service is not serving"}
2024/05/20 13:05:09 The binary was build using Nats as the message broker
2024/05/20 13:05:09 The binary was build using Nats as the message broker
{"time":"2024-05-20T13:05:09.822231503Z","level":"ERROR","msg":"health check failed : service is not serving"}
2024/05/20 13:05:10 The binary was build using Nats as the message broker
2024/05/20 13:05:10 The binary was build using Nats as the message broker
{"time":"2024-05-20T13:05:10.87364065Z","level":"ERROR","msg":"health check failed : service is not serving"}
2024/05/20 13:05:12 The binary was build using Nats as the message broker
2024/05/20 13:05:12 The binary was build using Nats as the message broker
{"time":"2024-05-20T13:05:12.159471781Z","level":"ERROR","msg":"health check failed : service is not serving"}
2024/05/20 13:05:14 The binary was build using Nats as the message broker
2024/05/20 13:05:14 The binary was build using Nats as the message broker
{"time":"2024-05-20T13:05:14.377178361Z","level":"ERROR","msg":"health check failed : service is not serving"}
2024/05/20 13:05:18 The binary was build using Nats as the message broker
2024/05/20 13:05:18 The binary was build using Nats as the message broker
{"time":"2024-05-20T13:05:18.045618441Z","level":"ERROR","msg":"health check failed : service is not serving"}
2024/05/20 13:05:24 The binary was build using Nats as the message broker
2024/05/20 13:05:24 The binary was build using Nats as the message broker
{"time":"2024-05-20T13:05:24.845535079Z","level":"ERROR","msg":"health check failed : service is not serving"}
2024/05/20 13:05:38 The binary was build using Nats as the message broker
2024/05/20 13:05:38 The binary was build using Nats as the message broker
{"time":"2024-05-20T13:05:38.051349708Z","level":"INFO","msg":"Successfully connected to things grpc server without TLS"}
{"time":"2024-05-20T13:05:38.053738507Z","level":"INFO","msg":"coap_adapter service started using http, exposed port coap-adapter:5683"}
{"time":"2024-05-20T13:05:38.053787321Z","level":"INFO","msg":"coap_adapter service  server listening at coap-adapter:5683 without TLS"}
{"time":"2024-05-20T13:05:38.053738527Z","level":"INFO","msg":"coap_adapter service http server listening at coap-adapter:5683 without TLS"}

Did you document any new/modified feature?

I have updated go docs for the setup function

Notes

[JeffMboya]

LGTM

[felixgateru]

LGTM

[WashingtonKK]

LGTM

[dborovcanin]

You can rename this error to errSvcNotServing = errors.New("service is not serving") and use it directly (not as a wrapper) in cases the service is not in the right state.