{ "version": "https://jsonfeed.org/version/1", "title": "Sidekiq on enumerator.dev", "icon": "", "home_page_url": "https://enumerator.dev/", "feed_url": "https://enumerator.dev/feed.json", "items": [ { "id": "https://enumerator.dev/migrating-from-standalone-sidekiq-to-an-activejob-adapter/", "title": "Migrating from Standalone Sidekiq to an ActiveJob Adapter", "content_html": "

I paired on this problem with my coworker Hannah Yeates. The examples below are from this pairing session. Thanks for working on this together, Hannah!

\n
\n

I am working on a project right now that uses Sidekiq without the ActiveJob adapter. This means that our Sidekiq jobs look something like this:

\n
class ExampleWorker\n  include Sidekiq::Worker\n\n  def perform(arg)\n    puts "Hello, the arg was #{arg}"\n  end\nend\n

And we call them like this:

\n
ExampleWorker.perform_async("a nice arg")\n

And in dev they’ll execute like this:

\n
2024-01-17T15:05:31.455Z pid=45599 tid=18gj\n  class=TestWorker \n  jid=8d32dcd88b1de2ed3a5419eb \n  INFO: start\nHello, the arg was foo\n

However, we want to switch over to GoodJob so that we have a transactionally safe background job processor. Our steps for doing this are:

\n
    \n
  1. Set up Sidekiq as an ActiveJob adapter instead of using Sidekiq on its own.
    2. \n
  2. Install GoodJob and replace Sidekiq with GoodJob.
    3. \n
  3. Run Sidekiq and GoodJob in parallel so that Sidekiq finishes all the work in Redis.
    4. \n
  4. Shut down Sidekiq and remove it from the application code.
    5. \n
\n

This post is about step 1, making Sidekiq the ActiveJob adapter instead of using it on its own.

\n

Step 1 : Configure ActiveJob

\n

Our first step is to configure the application to use Sidekiq for ActiveJob

\n
# in application.rb\nconfig.active_job.queue_adapter = :sidekiq\n

Then we created an empty ApplicationJob (more on this later).

\n
# application_job.rb\nclass ApplicationJob < ActiveJob::Base\n  # leave this empty for now\nend\n

Step 2: Convert Workers to ApplicationJob

\n

Next we updated our jobs to inherit from application job:

\n
class ExampleWorker < ApplicationJob # Add this\n  # remove this: include Sidekiq::Worker\n\n  # The rest of the code remains the same\nend\n

And finally, we changed all our method calls from ExampleWorker.perform_async("a nice arg") to ExampleWorker.perform_later("a nice arg").

\n

Step 3: See What Breaks

\n

So far, so good! Next, we had a question: What happens when a job is enqueued by Sidekiq but processed by ActiveJob? To emulate this we:

\n
    \n
  1. Booted a Rails console locally.
    2. \n
  2. Use the original code to enqueue a job.
    3. \n
  3. Modify the worker to use ApplicationJob.
    4. \n
  4. Boot Sidekiq locally and see what errors we get.
    5. \n
\n

In the real world, this would happen when the new code is deployed and there are jobs in the queue that were put there by Sidekiq. Here’s the first error we got:

\n
WARN: NoMethodError: undefined method `jid=' for #<TestWorker:...>\n

We resolved this by creating an alias to ActiveJob’s provider_job_id= and tested again. In the end we had three NoMethodError messages to resolve. We updated our ApplicationJob to respond to those methods.

\n
class ApplicationJob\n  def jid=(arg)\n    provider_job_id = arg\n  end\n\n  def jid\n    provider_job_id\n  end\n\n  def bid=(arg);\n    # We are lucky to not need batching, so we let this return `nil`\n  end\nend\n

Wrapping Up

\n

Making the switch to ActiveJob was easier than we expected. We now have a backwards compatible way of moving from standalone Sidekiq to ActiveJob. Our next step is to swap out Sidekiq for GoodJob.

\n

If you are planning on using GoodJob, or Solid Queue, check out this article on making the switch once you’re using ActiveJob:

\n

Migrating from Sidekiq to Solid Queue | Kyle Keesling

\n", "date_published": "2024-01-17T00:00:00+00:00", "url": "https://enumerator.dev/migrating-from-standalone-sidekiq-to-an-activejob-adapter/", "tags": ["sidekiq","activejob","ruby","rails"] } ] }