Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Worker and Bot should tolerate Redis connection issues #244

Open
dave-tucker opened this issue Apr 17, 2024 · 3 comments
Open

Worker and Bot should tolerate Redis connection issues #244

dave-tucker opened this issue Apr 17, 2024 · 3 comments
Labels
bug Something isn't working good first issue Good for newcomers

Comments

@dave-tucker
Copy link
Collaborator

If Redis isn't ready when the bot is started, it currently hangs.
If Redis isn't ready when the worker is started it retry loops forever but there is never a log once connection is established so it's hard to say for sure that it's working.

In both cases the code needs to be more tolerant of redis not being ready (yet) or a connection hiccup.

@russellb
Copy link
Member

related to #44

@dave-tucker
Copy link
Collaborator Author

omg I had the same good idea twice 😆

@vishnoianil vishnoianil added the bug Something isn't working label Apr 26, 2024
@nerdalert nerdalert added the good first issue Good for newcomers label Apr 30, 2024
@nerdalert
Copy link
Member

Here is an example to reproduce some of this behavior.

Run without a redis instance running:

<runtime> run  -it --rm ghcr.io/instructlab/instructlab-bot/instructlab-gobot:main

and you will get a blast of messages with no backoff:

2024-04-30T03:32:31.369Z	ERROR	cmd/root.go:245	Redis Client Error: dial tcp: lookup redis on 172.31.0.2:53: no such host
github.com/instructlab/instructlab-bot/gobot/cmd.receiveResults
	/src/cmd/root.go:245
github.com/instructlab/instructlab-bot/gobot/cmd.run.func4
	/src/cmd/root.go:170
2024-04-30T03:32:31.369Z	ERROR	cmd/root.go:245	Redis Client Error: dial tcp: lookup redis on 172.31.0.2:53: no such host
github.com/instructlab/instructlab-bot/gobot/cmd.receiveResults
	/src/cmd/root.go:245
github.com/instructlab/instructlab-bot/gobot/cmd.run.func4
	/src/cmd/root.go:170
2024-04-30T03:32:31.369Z	ERROR	cmd/root.go:245	Redis Client Error: dial tcp: lookup redis on 172.31.0.2:53: no such host
github.com/instructlab/instructlab-bot/gobot/cmd.receiveResults

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

4 participants