Celery worker creates "control" folder when using filesystem as a broker

Question

I have a project using Celery with Redis backend to manage tasks. For local development I am trying to set up Celery with filesystem as broker instead of Redis. However, when I run Celery worker, it creates me control folder in the root directory with the following contents:

/control
├── celery.pidbox.exchange
├── Q1.exchange
├── Q2.exchange
├── ...

I have trouble finding any resources on what it is and for what it is used exactly. My goal is to possibly move this folder to another location (ex. .celery/ folder), so that it does not sit in root directory.

Here is my Celery configuration:

class CeleryLocalConfig:

    include = ["app.tasks"]
    broker_url = "filesystem://"
    result_backend = "file://.celery/broker/results"
    broker_transport_options = {
        "data_folder_in": ".celery/broker/out",
        "data_folder_out": ".celery/broker/out",
        "queue_order_strategy": "sorted"
    }

celery_app = Celery(__name__)
celery_app.config_from_object(CeleryLocalConfig)

So far I have tried:

• Running Celery worker from different directory than root, but then worker doesn't receive any tasks.

• Providing --workdir parameter when running Celery worker, but it seem to create different paths between Celery worker and Celery app and therefore fails.

Answer 1

Setting a control_folder field did the trick for me:

CELERY_BROKER_TRANSPORT_OPTIONS = {
    "data_folder_in": os.path.join(BASE_DIR, ".celery"),
    "data_folder_out": os.path.join(BASE_DIR, ".celery"),
    "control_folder": os.path.join(BASE_DIR, ".celery"),
}

So for you, I suppose, it would be:

broker_transport_options = {
    "data_folder_in": ".celery/broker/out",
    "data_folder_out": ".celery/broker/out",
    "queue_order_strategy": "sorted",
    "control_folder": ".celery/broker/out",
}