Simple monitoring status for ActiveJob, independent of your queuing backend or cache storage.
gem 'activejob-status'By default, ActiveJob::Status use the Rails.cache to store data.
You can use any compatible ActiveSupport::Cache::Store you want (memory, memcache, redis, ..)
or any storage responding to read/write/delete
Note : In Rails 5,
Rails.cachedefaults toActiveSupport::Cache::NullStorewhich will result in empty status. Setting a cache store for ActiveJob::Status is therefore mandatory.
You can set your own store:
# config/initializers/activejob_status.rb
# Use an alternative cache store:
# ActiveJob::Status.store = :file_store, "/path/to/cache/directory"
# ActiveJob::Status.store = :redis_cache_store, { url: ENV['REDIS_URL'] }
# ActiveJob::Status.store = :mem_cache_store
#
# You should avoid using cache store that are not shared between web and background processes
# (ex: :memory_store).
#
if Rails.cache.is_a?(ActiveSupport::Cache::NullStore)
ActiveJob::Status.store = :mem_cache_store
endBecause ActiveJob::Status relies on cache store, all statuses come with an expiration time.
It's set to 1 hour by default.
You can set a longer expiration:
# config/initializers/activejob_status.rb
# Set your own status expiration time:
# Default is 1 hour.
#
ActiveJob::Status.options = { expires_in: 30.days.to_i }Depending on the cache storage latency, updating a status too often can cause bottlenecks.
To narrow this effect, you can force a time interval between each updates:
# config/initializers/activejob_status.rb
# Apply a time interval in seconds between every status updates.
# Default is 0 - no throttling mechanism
#
ActiveJob::Status.options = { throttle_interval: 0.1 }Include the ActiveJob::Status module in your jobs.
class MyJob < ActiveJob::Base
include ActiveJob::Status
endThe module introduces two methods:
progressto implement a progress status
class MyJob < ActiveJob::Base
include ActiveJob::Status
def perform
progress.total = 1000
1000.time do
# ...do something...
progress.increment
end
end
endstatusto directly read/update status
class MyJob < ActiveJob::Base
include ActiveJob::Status
def perform
status[:step] = "A"
# ...do something...
status[:step] = "B"
status[:result] = "...."
end
endYou can combine both to update status and progress in a single call.
class MyJob < ActiveJob::Base
include ActiveJob::Status
def perform
status.update(step: "A", total: 100)
100.times do
# ...do something...
progress.increment
end
# Reset the progress for the next step
status.update(step: "B", total: 50, progress: 0)
50.times do
# ...do something...
progress.increment
end
end
endThrottling mechanism (see configuration) is applied when doing:
progress.increment
progress.decrement
status.update(foo: 'bar')Throttling mechanism is not applied when doing:
progress.total = 100
progress.progress = 0
progress.finish
status[:foo] = 'bar'
status.update({ foo: 'bar' }, force: true)Check the status of a job
job = MyJob.perform_later
status = ActiveJob::Status.get(job)
# => { status: :queued }You can also use the job_id
status = ActiveJob::Status.get('d11b64e6-8631-4118-ae76-e19376769171')
# => { status: :queued }Follow the progression of your job
loop do
puts status
break if status.completed?
end
# => { status: :queued }
# => { status: :working, progress: 0, total: 100, step: "A" }
# => { status: :working, progress: 60, total: 100, step: "A" }
# => { status: :working, progress: 90, total: 100, step: "A" }
# => { status: :working, progress: 0, total: 50, step: "B" }
# => { status: :completed, progress: 50, total: 50, step: "B" }The status provides you getters:
status.status # => "working"
status.queued? # => false
status.working? # => true
status.completed? # => false
status.failed? # => false
status.progress # => 0.5 (progress / total)
status[:step] # => "A"... until it's completed
status.status # => "completed"
status.completed? # => true
status.progress # => 1Within a controller, you can serialize a status to JSON:
class JobsController
def show
status = ActiveJob::Status.get(params[:id])
render json: status.to_json
end
endGET /jobs/status/d11b64e6-8631-4118-ae76-e19376769171.json
{
"status": "working",
"progress": 50
"total": 100,
"step": "A"
}
You can override default options per job:
class MyJob < ActiveJob::Base
include ActiveJob::Status
def status
@status ||= ActiveJob::Status::Status.new(self, expires_in: 3.days, throttle_interval: 0.5)
end
def perform
...
end
endInternally, ActiveJob::Status uses ActiveSupport#rescue_from to catch every Exception to apply the failed status
before throwing the exception again.
Rails says:
Handlers are inherited. They are searched from right to left, from bottom to top, and up the hierarchy. The handler of the first class for which exception.is_a?(klass) holds true is the one invoked, if any.
Thus, there are a few points to consider when using rescue_from:
1 - Do not declare rescue_from handlers before including ActiveJob::Status. They cannot be called:
class ApplicationJob < ActiveJob::Base
rescue_from Exception do |e|
ExceptionMonitoring.notify(e)
raise e
end
end
class MyJob < ApplicationJob
# The rescue handlers from ApplicationJob won't ever be executed
# and the exception monitoring won't be notified.
include ActiveJob::Status
end2 - If you're rescuing any or all exceptions, the status will never be set to failed. You need to update it by
yourself:
class ApplicationJob < ActiveJob::Base
include ActiveJob::Status
rescue_from Exception do |e|
ExceptionMonitoring.notify(e)
status.update(status: :failed)
raise e
end
end3 - Subsequent handlers will stop the rescuing mechanism:
class MyJob < ApplicationJob
# With the exceptions handled below:
# - the monitor won't be notified
# - the job status will remains to `working`
retry_on 'SomeTimeoutError', wait: 5.seconds
discard_on 'DeserializationError'
rescue_from 'AnotherCustomException' do |e|
do_something_else
end
end- Don't hesitate to submit your feature/idea/fix in issues
- Fork the repository
- Create your feature branch
- Create a pull request
Not yet provided.
Copyright (c) 2021 Savater Sebastien.
See LICENSE for further details.
Contributors:
- Valentin Ballestrino @vala
- Dennis van de Hoef @dennisvandehoef
- Olle Jonsson @olleolleolle
- Felipe Sateler @fsateler