Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How To Clear Queued Renewal Work? #297

Open
surgiie opened this issue Apr 26, 2024 · 1 comment
Open

How To Clear Queued Renewal Work? #297

surgiie opened this issue Apr 26, 2024 · 1 comment

Comments

@surgiie
Copy link

surgiie commented Apr 26, 2024

Problem Description:

We are encountering issues with queued renewal work in the lua-resty-auto-ssl plugin. Specifically, we are using the Redis storage driver with the following configuration:

auto_ssl:set("storage_adapter", "resty.auto-ssl.storage_adapters.redis")
auto_ssl:set("redis", { host = "..."})

Our setup has resulted in a significant number of renewals getting queued for processing that are no longer necessary. As a consequence, our web server's CPU usage spikes, leading to severe disruptions in user traffic.

Current Workaround:

As a temporary solution, we have disabled the plugin using the following Lua block:

ssl_certificate_by_lua_block {
    auto_ssl:ssl_certificate({ generate_certs = false })
}

While this workaround has mitigated the issue, we are seeking a more permanent solution to clear out the unnecessary queued renewal work which starts to process back up once the plugin is re-enabled.

Desired Solution:

We are unsure of where this plugin is storing data for the renewals but we suspect that clearing specific key(s) from the Redis storage would resolve this problem. However, we are uncertain about which key(s) are pertinent to this issue and how to clear them effectively.

Requested Assistance:

Could someone knowledgeable about lua-resty-auto-ssl and Redis storage provide guidance on the following:

Which specific key(s) in Redis correspond to the queued/renewal work that needs to be cleared?
What is the recommended method or script to clear out this unnecessary work from Redis?
Any insights or assistance would be greatly appreciated as we aim to optimize our system's performance and minimize disruptions to user traffic.

@surgiie
Copy link
Author

surgiie commented May 1, 2024

We ended up digging into redis and purging the keys out manually. We were hoping for a method via the plugin that would of prevented the need to manually go into the redis storage but Im posting here in hopes that it helps someone out:

We wrote out the relevant keys to a file:

redis-cli -h redis-host-example.com --scan --pattern '*domain-example.com*'

Then deleted them manually:

for key in `cat /tmp/redis-keys.txt`; do
   redis-cli -h redis-host-example.com DEL $key
done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant