Skip to content

Latest commit

 

History

History
59 lines (41 loc) · 2.5 KB

migrating-sidekiq.md

File metadata and controls

59 lines (41 loc) · 2.5 KB
Raw
title description
Migrating Sidekiq
How to migrate Sidekiq to a new Redis instance

Migrating Sidekiq

We publish a Docker image artsy/sidekiq-migrate that can be used to migrate Sidekiq jobs from one Redis database (on the same or different instance) - source code can be found in our docker-images repo

Migrate an application

  1. Check the old Redis URL
hokusai [staging|production] env get REDIS_URL

This will be supplied to the migration container via $SIDEKIQ_OLD_REDIS_URL

  1. Update to the new Redis Url
hokusai [staging|production] env set "REDIS_URL=$SIDEKIQ_NEW_REDIS_URL"
  1. Refresh the application
hokusai [staging|production] refresh

At this point, you should see a squeaky clean Sidekiq dashboard at your application's Sidekiq admin URL.

  1. Migrate Sidekiq jobs and stats from the old Redis instance - plug in the old and new Redis URLs to $SIDEKIQ_OLD_REDIS_URL and $SIDEKIQ_NEW_REDIS_URL

4a) Run docker pull artsy/sidekiq-migrate:latest to pull the artsy/sidekiq-migrate:latest image to your local Docker image cache, busting the cache if you happen to have an older version. Note: this step will become obsolte in Docker version 19.09 with the addition of the --pull flag to docker run

4b) Connect to the staging or production VPN and ensure your local Docker is configured to use the VPN interface.

4c) Run docker run -ti --env "SIDEKIQ_OLD_REDIS_URL=$SIDEKIQ_OLD_REDIS_URL" --env "SIDEKIQ_NEW_REDIS_URL=$SIDEKIQ_NEW_REDIS_URL" artsy/sidekiq-migrate:latest

See https://github.com/artsy/docker-images/tree/master/sidekiq-migrate for further options / enviornment variables to enable debug logging, perform a dry run, or clean up the source redis database's Sidekiq-specific keys.

Refresh the Sidekiq dashboard at and confirm everything is migrated.

  1. Update your app's Redis database assignments in our staging and production Terrform config so we can track and reference it in our "shared-redis-db-assignments" Kubernetes ConfigMap