Sidekiq integration for Capistrano - providing systemd service management and deployment coordination for Sidekiq 7+.
Upgrading from v2.x? See the Migration Guide for breaking changes and upgrade instructions.
For a complete working example of this gem in action, see the capistrano-example-app which demonstrates:
- Rails 8.0 deployment with Capistrano
- Sidekiq 7.0 with Redis 7+
- Systemd service management
- rbenv integration
- Complete deployment guide
Add to your Gemfile:
gem 'capistrano-sidekiq', group: :developmentThen execute:
$ bundle installAdd to your Capfile:
# Capfile
require 'capistrano/sidekiq'
install_plugin Capistrano::Sidekiq # Default sidekiq tasks (REQUIRED!)
install_plugin Capistrano::Sidekiq::Systemd # Systemd integration (REQUIRED!)Important: Both require and install_plugin lines are necessary. The require loads the gem code, while install_plugin actually registers the Capistrano tasks. Without install_plugin, commands like cap sidekiq:start will not be available.
# config/deploy.rb
set :sidekiq_roles, :worker # Default role for Sidekiq processes
set :sidekiq_default_hooks, true # Enable default deployment hooks
set :sidekiq_env, fetch(:rack_env, fetch(:rails_env, fetch(:stage))) # Environment for Sidekiq processes
# Single config file
set :sidekiq_config_files, ['sidekiq.yml']
# Multiple config files for different Sidekiq processes
set :sidekiq_config_files, ['sidekiq.yml', 'sidekiq-high-priority.yml']This gem follows the Capistrano convention of sharing common settings across multiple service gems (e.g., capistrano-puma, capistrano-sidekiq). The following settings are shared:
:service_unit_user- Determines if services run as system or user services:systemctl_bin- Path to the systemctl binary:lingering_user- User for systemd lingering (for user services):service_unit_env_files- Shared environment files:service_unit_env_vars- Shared environment variables
Each gem can override these with prefixed versions (e.g., :sidekiq_systemctl_user).
# Shared systemd settings (can be used by multiple Capistrano gems like capistrano-puma)
set :service_unit_user, :user # Run as user or system service (:system or :user)
set :systemctl_bin, '/bin/systemctl' # Path to systemctl binary
set :lingering_user, 'deploy' # User to enable lingering for (defaults to :user)
# Sidekiq-specific overrides (optional - defaults to shared settings above)
set :sidekiq_systemctl_user, :system # Override service_unit_user for Sidekiq only
set :sidekiq_systemctl_bin, '/usr/bin/systemctl' # Override systemctl_bin for Sidekiq only
set :sidekiq_service_unit_name, "custom_sidekiq_#{fetch(:stage)}" # Custom service name
set :sidekiq_lingering_user, 'sidekiq' # Override lingering user for Sidekiq only
# Environment configuration
set :sidekiq_service_unit_env_files, ['/etc/environment'] # Environment files
set :sidekiq_service_unit_env_vars, [ # Environment variables
'RAILS_ENV=production',
'MALLOC_ARENA_MAX=2'
]
# Logging configuration
set :sidekiq_log, -> { File.join(shared_path, 'log', 'sidekiq.log') }
set :sidekiq_error_log, -> { File.join(shared_path, 'log', 'sidekiq.error.log') }
# Command customization
set :sidekiq_command, 'sidekiq' # Use 'sidekiq' or 'sidekiqswarm' for Enterprise
set :sidekiq_command_args, -> { "-e #{fetch(:sidekiq_env)}" }
# Use login shell to load environment (for rbenv, rvm, etc.)
set :sidekiq_use_login_shell, true
# Sidekiq Enterprise - Swarm configuration
set :sidekiq_command, 'sidekiqswarm'
set :sidekiq_swarm_env_vars, [
'SIDEKIQ_COUNT=5', # Number of processes
'SIDEKIQ_MAXMEM_MB=768', # Memory limit per process
'SIDEKIQ_PRELOAD_APP=1' # Preload app for memory efficiency
]
# Merge swarm env vars with regular env vars
set :sidekiq_service_unit_env_vars, -> {
fetch(:sidekiq_service_unit_env_vars, []) + fetch(:sidekiq_swarm_env_vars, [])
}You can configure Sidekiq differently for specific servers, allowing you to run different Sidekiq processes with different configurations on different servers.
# config/deploy/production.rb
server 'worker1.example.com',
roles: [:worker],
sidekiq_config_files: ['sidekiq_high_priority.yml']
server 'worker2.example.com',
roles: [:worker],
sidekiq_config_files: ['sidekiq_low_priority.yml']# Different users and multiple processes per server
server 'worker1.example.com',
roles: [:worker],
sidekiq_config_files: ['sidekiq_critical.yml', 'sidekiq_default.yml'],
sidekiq_user: 'sidekiq_critical',
sidekiq_systemctl_user: :system # Run as system service on this server
server 'worker2.example.com',
roles: [:worker],
sidekiq_config_files: ['sidekiq_batch.yml'],
sidekiq_user: 'sidekiq_batch',
sidekiq_service_unit_env_vars: ['MALLOC_ARENA_MAX=4'] # Server-specific env vars- Configuration Files: Each server can have its own set of
sidekiq_config_files - Service Creation: A separate systemd service is created for each config file
- Service Naming: Services are named as
<app>_sidekiq_<stage>for the defaultsidekiq.yml, or<app>_sidekiq_<stage>.<config_name>for additional configs - Independent Control: Each service can be started, stopped, and restarted independently
config/sidekiq_high_priority.yml:
:concurrency: 10
:queues:
- [critical, 2]
- [high, 1]config/sidekiq_low_priority.yml:
:concurrency: 5
:queues:
- [low, 1]
- [default, 1]When using per-server configurations, Capistrano will:
- Install the appropriate services on each server during
cap production sidekiq:install - Start only the configured services on each server during
cap production sidekiq:start - Manage each server's services independently
You can also target specific servers:
cap production sidekiq:restart --hosts=worker1.example.comThere are two ways to run multiple Sidekiq processes:
Create separate config files for different workloads:
# config/sidekiq_critical.yml
:concurrency: 5
:queues:
- [critical, 2]
# config/sidekiq_default.yml
:concurrency: 10
:queues:
- [default, 1]
- [low, 1]Configure in deploy.rb:
set :sidekiq_config_files, ['sidekiq_critical.yml', 'sidekiq_default.yml']This creates separate services:
myapp_sidekiq_production.sidekiq_criticalmyapp_sidekiq_production.sidekiq_default
Benefits:
- Clear purpose for each process
- Independent scaling and management
- Different configurations per process
- Meaningful service names
For simple scaling with identical processes:
set :sidekiq_command, 'sidekiqswarm'
set :sidekiq_service_unit_env_vars, ['SIDEKIQ_COUNT=10']This runs 10 identical processes with a single service.
Use the helper task to generate config files:
# Generate 5 config files
bundle exec cap production sidekiq:helpers:generate_configs[5]
# This creates:
# - config/sidekiq.yml (critical, high queues)
# - config/sidekiq_1.yml (default, medium queues)
# - config/sidekiq_2.yml (low, background queues)
# - config/sidekiq_3.yml (low, background queues)
# - config/sidekiq_4.yml (low, background queues)Then add to your deploy.rb:
set :sidekiq_config_files, ['sidekiq.yml', 'sidekiq_1.yml', 'sidekiq_2.yml', 'sidekiq_3.yml', 'sidekiq_4.yml']# View all available tasks
cap -T sidekiq
# Common commands
cap sidekiq:start # Start Sidekiq
cap sidekiq:stop # Stop Sidekiq (graceful shutdown)
cap sidekiq:restart # Restart Sidekiq
cap sidekiq:quiet # Quiet Sidekiq (stop processing new jobs)
cap sidekiq:install # Install Sidekiq systemd service
cap sidekiq:uninstall # Remove Sidekiq systemd service
cap sidekiq:enable # Enable Sidekiq systemd service
cap sidekiq:disable # Disable Sidekiq systemd service
cap sidekiq:mark_deploy # Mark deployment in Sidekiq metrics (Sidekiq 7+)
# Helper tasks
cap sidekiq:helpers:generate_configs[3] # Generate 3 config files
cap sidekiq:helpers:show_config # Show current configurationFor detailed information about systemd integration, see Systemd Integration Guide.
Sidekiq 7 introduced metrics with deployment tracking. Enable it to see deployment markers in your Sidekiq metrics graphs:
# config/deploy.rb
# Enable deployment marking
set :sidekiq_mark_deploy, true
# Optional: Custom deploy label (defaults to git commit description)
set :sidekiq_deploy_label, "v#{fetch(:current_revision)[0..6]} deployed to #{fetch(:stage)}"When enabled, the gem will:
- Automatically mark deployments after successful deploys
- Show vertical red lines in Sidekiq's metrics graphs
- Display the deploy label when hovering over the marker
This helps correlate performance changes with deployments.
This gem supports Sidekiq Enterprise's sidekiqswarm command for managing multiple processes with a single service.
# config/deploy.rb
# Use sidekiqswarm instead of sidekiq
set :sidekiq_command, 'sidekiqswarm'
# Configure swarm-specific environment variables
set :sidekiq_service_unit_env_vars, [
'SIDEKIQ_COUNT=10', # Number of processes to spawn
'SIDEKIQ_MAXMEM_MB=512', # Memory limit per process
'SIDEKIQ_PRELOAD_APP=1', # Preload Rails app for memory efficiency
'RAILS_ENV=production'
]
# Optional: Configure shutdown timeout for graceful stops
set :sidekiq_command_args, -> { "-e #{fetch(:sidekiq_env)} -t 30" }- Single Service: Manage N processes with one systemd service
- Memory Efficiency: Preload app once, fork processes
- Auto-restart: Processes that exceed memory limits are automatically restarted
- Simplified Management: No need for multiple service files
server 'worker1.example.com',
roles: [:worker],
sidekiq_service_unit_env_vars: ['SIDEKIQ_COUNT=20', 'SIDEKIQ_MAXMEM_MB=1024']
server 'worker2.example.com',
roles: [:worker],
sidekiq_service_unit_env_vars: ['SIDEKIQ_COUNT=5', 'SIDEKIQ_MAXMEM_MB=512']View Sidekiq service logs using journalctl:
# View last 100 lines of logs
journalctl -u sidekiq_my
8000
app_production -n 100
# Follow logs in real-time
journalctl -u sidekiq_myapp_production -fLog files are configured automatically using the append: functionality in the systemd service file.
For systems with older Systemd versions where append: is not supported:
- Sidekiq messages are sent to syslog by default
- Configure system logger to filter Sidekiq messages
A complete example application demonstrating the usage of this gem is available at: https://github.com/seuros/capistrano-example-app
- Fork the repository
- Create your feature branch (
git checkout -b my-new-feature) - Commit your changes (
git commit -am 'Add some feature') - Push to the branch (
git push origin my-new-feature) - Create new Pull Request