8000 Use durable=True by default everywhere by trianglesis · Pull Request #2313 · celery/kombu · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Use durable=True by default everywhere #2313

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 5 commits into
base: main
Choose a base branch
from
Draft

Use durable=True by default everywhere #2313

wants to merge 5 commits into from

Conversation

trianglesis
Copy link
@trianglesis trianglesis commented Jun 10, 2025 •
edited
Loading

Description

Trying the variant where all queues and exchanges are durable by default.

Discussed at issue 2237
RabbitMQ recommendation: DOC
Related to the celery PR.

Conf: cat /etc/rabbitmq/rabbitmq.conf

# https://www.rabbitmq.com/docs/deprecated-features
deprecated_features.permit.transient_nonexcl_queues = false

A pretty simple fix just changed booleans to durable=True

Tested in my DEV environment.
Now running at prod version.

No issues with tasks, queues, managing and pinging/inspecting queues and workers.

I didn't run bundled tests, however.
Only relied upon my envs.

My celery conf 
def celery_setup():
    # Get creds and hosts
    setup_celery_env()
    # Setup django project
    django.setup()
    #  The backend is specified via the backend argument to Celery
    global app
    if app is None:
        app = Celery('server',
                     broker=RabbitMQCreds.BROKER,
                     backend=backend,
                     )

    app.conf.timezone = 'UTC'
    app.conf.enable_utc = True
    # General config:
    app.conf.update(
        result_backend=result_backend,
        accept_content=['pickle'],
        task_serializer='pickle',
        result_serializer='pickle',
        result_extended=True,
        task_track_started=True,
        beat_scheduler='django_celery_beat.schedulers:DatabaseScheduler',
        database_engine_options={'pool_timeout': 90},
        worker_prefetch_multiplier=1,
        worker_concurrency=1,
        worker_timer_precision=1.0,
        broker_heartbeat=10.0,
        broker_heartbeat_checkrate=2.0,
    )

    app.autodiscover_tasks()
    app.conf.update(
        worker_max_memory_per_child=1024 * 100,
        worker_max_tasks_per_child=100,
        worker_proc_alive_timeout=5.0,
        task_acks_late=False,
        task_acks_on_failure_or_timeout=True,
        worker_cancel_long_running_tasks_on_connection_loss=True,
        task_reject_on_worker_lost=False,
        worker_pool_restarts=True,
        worker_enable_remote_control=True,
        worker_lost_wait=20,
        broker_connection_retry_on_startup=True,
        broker_connection_retry=True,
        broker_connection_max_retries=0,
        broker_connection_timeout=4.0,
        broker_channel_error_retry=True,
        broker_pool_limit=100,
        task_send_sent_event=True,
        worker_send_task_events=True,
        worker_disable_rate_limits=True,
        task_default_queue='w_routines@tentacle.dq2',
        task_default_exchange='someserver',
        task_default_routing_key='w_routines@tentacle.dq2',
        worker_direct=True,
        result_persistent=True,
        task_default_delivery_mode='persistent',
    )

@trianglesis trianglesis mentioned this pull request Jun 10, 2025
Draft
@auvipy
Copy link
Member
auvipy commented Jun 10, 2025

Thanks

@auvipy auvipy added this to the 5.7.0 milestone Jun 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
5.7.0
Development

Successfully merging this pull request may close these issues.
2 participants
@trianglesis @auvipy
0