Yesterday I had a revelation: the Heroku gem can start and stop workers.
For the last few months, I’ve been manually kicking off a rake task to start a worker to catch up on Resque queues.
Now, my cron task starts up a Heroku worker, queues up the updates that workers need to do, and shutdowns the worker once the queues are empty.
This works great for me since my app has regular processes to run, but generally empty queues so having a worker running all the time isn’t very cost effective.
Here’s the basics:
require 'heroku'
desc "This task is called by the Heroku cron add-on"
task :cron => :environment do
client = Heroku::Client.new(heroku_user, heroku_pass)
# start a worker
client.set_workers(heroku_app, 1)
# Do stuff
.....
sleep 10 # wait for worker to start up
if Resque.working.empty? # returns list of workers that are working
# stop the worker
client.set_workers(heroku_app, 0)
else
# wait for jobs to finish
sleep 5
end
end
This has pretty worked well for me so far. No longer do I need to manually start a worker. And I also am keeping my costs low by running the worker only when they are jobs to be processed.
