Set up Celery production for Django project
π Background
Celery is a cron job service commonly used in Django projects. In my case, I used Celery to reset user streaks at midnight (00:00 AM) for my application.
In development, I had to run three terminal windows:
-
One for the Django server:
python manage.py runserver
-
One for the Celery worker:
celery -A proj worker
-
One for the Celery beat:
celery -A proj beat
That setup was a bit complex. So in this blog, Iβll share how I set up Celery as a production-ready service using systemd
.
π οΈ Approach
When I first moved my app to production, I used Gunicorn to serve Django, but forgot about Celery. As a result, scheduled tasks didnβt run because both Celery Worker and Celery Beat need to run in parallel.
β Option 1: Using tmux
Initially, I used tmux
:
-
SSH into the server.
-
Start a
tmux
session and split the window withCtrl + b
β%
. -
Run the worker and beat processes in separate panes.
Even after closing SSH, the processes stayed alive (confirmed using htop
). This works, but itβs not ideal for long-term use.
β Option 2: Using systemd Services
When I got a new VPS, I wanted a better solution. After some research (and help from ChatGPT π), I found a reliable approach using systemd
services to run Celery in the background.
Reference: Django with Celery in Production
βοΈ 1. Create the Celery Worker Service
Create a new service file:
sudo nano /etc/systemd/system/celery.service
Paste the following (replace user
, paths, and [celery_app]
accordingly):
[Unit]
Description=Celery Worker Service
After=network.target
[Service]
Type=simple
User=your_username
Group=your_username
WorkingDirectory=/home/your_username/your_project
ExecStart=/home/your_username/.local/share/virtualenvs/your-venv/bin/celery -A [celery_app] worker --loglevel=INFO
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
π Replace:
-
your_username
with your actual Linux username. -
[celery_app]
with the value from yourcelery.py
file.
Then run:
sudo systemctl daemon-reload
sudo systemctl enable celery
sudo systemctl start celery
π 2. Create the Celery Beat Service
Now create the Beat service:
sudo nano /etc/systemd/system/celery-beat.service
Paste the following:
[Unit]
Description=Celery Beat Service
After=network.target
[Service]
Type=simple
User=your_username
Group=your_username
WorkingDirectory=/home/your_username/your_project
ExecStart=/home/your_username/.local/share/virtualenvs/your-venv/bin/celery -A [celery_app] beat --loglevel=INFO
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
Then run:
sudo systemctl daemon-reload
sudo systemctl enable celery-beat
sudo systemctl start celery-beat
β Check status:
sudo systemctl status celery-beat
If you see active (running)
, it means everything is set up correctly.
π§Ύ Conclusion
Thatβs how I set up Celery and Celery Beat in production using systemd
. Itβs a clean, reliable, and maintainable way to manage background tasks in Django.
π References:
Thanks for reading! π Hope this helps you in your deployment journey.
- β Previous
Touch Typing Practice March 2025 - Next β
Touch Typing Practice April 2025