Investigate the use of Python Celery for API job queueing #1037
Comments
|
I completely agree that moving from Python multithreading to a proper task queue like Celery would significantly improve Nettacker’s scalability and maintainability. As you mentioned, RabbitMQ might be overkill at this stage. I’d suggest using Redis as the message broker it's lightweight, easy to deploy, and integrates seamlessly with Celery. One of the advantages of Celery is that it abstracts the broker layer well, so switching to RabbitMQ in the future would just require changing To be transparent, I haven’t forked the repo yet but I’m very interested in contributing. I noticed the Celery discussion and wanted to jump in because I recently worked on something similar: implementation of a ticket queue in Supportix It involves agent availability checks, fallback queuing, and task delegation using Celery + Redis. I believe the core idea can align well with Nettacker’s goals too. |
at the moment, Nettacker is using Python multi-threading for API tasks, which is not very scalable. It would be good to replace it with a proper task queue system that can handle retires/tracking/background jobs and scheduled jobs. Something like RabbitMQ is probably overkill for now, but Celery is an interesting and widely used Python-based task queue that can be used to improve Nettacker
The text was updated successfully, but these errors were encountered: