Added flexible eager load configuration for Resque when used in Rails application

[georgiybykov]

Hi!

We faced a problem when upgraded from Resque 1.27.4 to > 2.0.0 version (using Resque gem as a dependency in Rails 6.0.4 application with Ruby 2.7.5) and lost many important events silently. The eager load for Resque task did not work was the reason.

Our production configuration for eager_load is set to true:

# config/environments/production.rb

Rails.application.configure do
  ...
  config.eager_load = true
  ...
end

Changes documentation between versions says the following:

However, the eager_load will never be true. Since Resque starts from Rake task, the eager_load option configuration is set to false permanently by default starting from this commit at Rails.

Only if your Rails application is 6.1.0 version or above, you are able to configure eager loading for Rake tasks environment by using rake_eager_load option that was added in this commit to provide the ability to override the default false value for every rake task of the environment.

Based on the above, I propose to add flexible configuration for eager_load:

To configure eager_load option using Resque interface you are able to add in config/initializers/resque.rb
the following:

Resque.rails_eager_load_enabled = true # one configuration for any of your application environments

Or if you would like to configure the eager_load option depends on your Rails application environment, for example:

Resque.rails_eager_load_configure do |env|
  env.development = false
  env.staging = true
  env.production = true
end

[iloveitaly]

@georgiybykov Is there a reason you don't do something like:

namespace :resque do
  task :setup => :environment do

  end
end

Pretty sure this would force rails to load before the resque app?

[georgiybykov]

The file lib/tasks/resque.rake in our Rails application contains the following:

# frozen_string_literal: true

require 'resque/tasks'

namespace :resque do
  task :setup => :environment

  ...
end

The code above is not a solution (does not force preload all classes for the application).

@iloveitaly As an alternative, do you suggest to override the setup task to force preload all classes in memory when booting the app per below?

# frozen_string_literal: true

require 'resque/tasks'

namespace :resque do
  task :setup => :environment

  task :preload => :setup do
    ActiveSupport.run_load_hooks(:before_eager_load, Rails.application)
    Rails.application.config.eager_load_namespaces.each(&:eager_load!)
  end
end

Since Rake tasks ignore the Rails application eager_load configuration setting:

eager_load leads to false value here:

Only if you use Rails >= 6.1.0, you can impact on this setting via Rails.application.config.rake_eager_load=. However, it will be applied for all rake tasks of you application for current environment.

That is why in this pull request I suggest to add flexible eager load configuration for Resque when used in Rails application without actions to override the code in Rake tasks in order to setup correctly.

[georgiybykov]

Hi @iloveitaly!

Could you look at this comment as an answer to your question please?

[iloveitaly]

@georgiybykov your explanation makes sense. Tricky problem! I need to set aside some more time to review your PR. Will hopefully get back to you soon.

[georgiybykov]

Hi @iloveitaly!

Thank you again! I would be very grateful for your feedback and I hope you get a chance to review the code soon! Please let me know if I can provide anything else in the meantime.

[fxn]

I suspect there's a bit of confusion here:

• In Rake tasks, config.eager_load has been false by default for a long time.
• In older versions, this was overridden by hand, see Rails 4.0 for example (but users could in turn reassign).
• In newer version, as it is said above, users can set config.rake_eager_load, whose value will be assigned to config.eager_load before initialization, and it is false by default.

However:

• If config.eager_load is true, Rails already does what this Rake task is doing.
• Since idempotence is not a requirement, you'll indeed run stuff twice!

I believe this task should be deleted, though I might be missing something. Let Rails eager load, and let users control this via Rails. I opened #1867 with this proposal.

[georgiybykov]

@fxn Thank you very much for your comment!

My opinion and the main points of the current pull request are:

• Prior to the 2.0.0 release change, users did not need to override the task manually to eagerly load the namespaces, as the setup task implemented this behaviour.
• If we simply delete the code for this task, users will have to override the task in their projects (if they are using Rails with a version below 6.1.0), and they should know about this "feature" by default, but the documentation says nothing about it.
• In my opinion, even if the application is using one of the latest versions of Rails, not all of them need to enable rake_eager_load which would affect every Rake task just because it uses Resque. So this request is about:
  • removing the confusion from the 2.0.0 release that points to eager_load configuration changes;
  • suggesting to add a more flexible eager load configuration using the Rasque interface.

---

I believe this task should be deleted, though I might be missing something. Let Rails eager load, and let users control this via Rails. I opened #1867 with this proposal.

Based on the above, you are correct, however the users with Rails version < 6.1.0 should always override the task:

task :preload => :setup do
  ActiveSupport.run_load_hooks(:before_eager_load, Rails.application)
  Rails.application.config.eager_load_namespaces.each(&:eager_load!)
end

because they cannot control this via the Rails configuration.