Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use SET NX/EX for new master locking approach #734

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

carsonreinke
Copy link
Contributor

@carsonreinke carsonreinke commented Nov 12, 2021

Fixes #553. In general, a better and more accepted approach to the master locking. Redis documentation even states:

The command SET resource-name anystring NX EX max-lock-time is a simple way to implement a locking system with Redis.

https://redis.io/commands/set

Thanks @valo

Copy link
Member

@yaauie yaauie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm all for simplifying calls where appropriate.

In this case though, any user who previously qualified for the resilient lock (because the redis they were connected to was >= 2.5) will get "downgraded" to the basic lock when they upgrade this plugin.

I would prefer an implementation based on SET NX/EX to exist along side the existing lua-based resilient lock index, not to replace it.

lib/resque/scheduler/locking.rb Outdated Show resolved Hide resolved
module Resque
module Scheduler
module Lock
class ResilientModern < Resilient
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't come up with a better name

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does inheriting from Resilient give us?

I'd prefer inheriting from base and add a implementation for locked?. This would make it easier to remove the Resilient (and Basic) lock implementations in the future, which I can't imagine we wouldn't do now that we have this awesome new lock implementation (and redis 2.x is very old at this point).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@iloveitaly The Resilient implementation offers the locked? method which retrieves and renews the lock. So, it looks like in Redis 6.2 a similar command was added.

@yaauie suggested I retain the existing implementation to avoid users downgrading to a worse master lock implementation if they upgraded resque-scheduler.

One possibility could be switching the Redis requirement to 6.2 and completely removing the inheritance and Lua script implementation.

@carsonreinke carsonreinke marked this pull request as ready for review November 15, 2021 00:45
@carsonreinke carsonreinke changed the title Use SET NX/EX for master locking instead of Lua script Use SET NX/EX for new master locking approach Nov 15, 2021
@@ -97,7 +104,7 @@ def master_lock_key
end

def redis_master_version
Resque.data_store.redis.info['redis_version'].to_f
Gem::Version.new(Resque.data_store.redis.info['redis_version'])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the change to using Gem::Version here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@iloveitaly this offers "native" version number parsing/comparison which was needed to see when the Redis feature was available. Specifically, the patch level comparison.

CHANGELOG.md Outdated Show resolved Hide resolved
module Resque
module Scheduler
module Lock
class ResilientModern < Resilient
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does inheriting from Resilient give us?

I'd prefer inheriting from base and add a implementation for locked?. This would make it easier to remove the Resilient (and Basic) lock implementations in the future, which I can't imagine we wouldn't do now that we have this awesome new lock implementation (and redis 2.x is very old at this point).

@iloveitaly
Copy link
Contributor

Thanks for this great change! Excited to get this pulled in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Master lock getting out of sync in case the LUA script gets interrupted
3 participants