Skip to content

Celery sync/async DB engines are not disposed after worker fork (connection pool inherited across prefork workers) #122

Description

@DominikCywinski

In what type of project did the bug occur?
api-monolith/api-microserivce/mcp-server/agent (any project with celery)

Describe the bug
sync_engine and async_engine (app/database.py) are created once, at module import time, before Celery's prefork worker pool forks its child processes. Forked workers inherit the parent's connection pool (including any live socket file descriptors) instead of building their own. There is no worker_process_init hook disposing the engines after fork, which is SQLAlchemy's documented safeguard for this exact setup. Currently latent (the pool is lazy - no connection is actually opened before fork in this codebase), but a known footgun that can surface as random connection errors under load, worker recycling, or once anything establishes a DB connection before fork.

To Reproduce

  1. Generate a project with the celery plugin enabled.
  2. Run celery -A app.main:celery_app worker (default prefork pool, multiple worker processes).
  3. Inspect app/integrations/celery/core.py - no worker_process_init signal handler disposes sync_engine / async_engine after fork.

Expected result
Each forked worker process discards the inherited connection pool and opens its own fresh connections on first use.

Actual result
No dispose hook exists - workers keep whatever pool state they inherited from the parent process at fork time.

Environment (please complete the following information):

  • Execution method: local uv / docker

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions