Skip to content

DatabaseScheduler ignores 'expires' option #393

Open
@hheimbuerger

Description

@hheimbuerger

I believe the DatabaseScheduler does not support the expires option properly. After struggling to make it work experimentally, I checked the source and couldn't find a place where this would be respected.

Here's how I was trying to set it up in my Celery configuration:

CELERYBEAT_SCHEDULER = 'djcelery.schedulers.DatabaseScheduler'
CELERYBEAT_SCHEDULE = {
    'periodic-task': {
        'task': 'myapp.mytask',
        'schedule': crontab(minute='*/1'),   # schedule every minute
        'options': {'expires': 5},   # expire if not run five seconds after being scheduled
    }
}

This matches the advice given here in the Celery documentation (»This can be any argument supported by apply_async(), e.g. exchange, routing_key, expires, and so on«), as well as here on SO and also matches how the default entry in the same file tries to set up its own expiration.

I believe the issue is that in djcelery/schedulers.py#L128, the ModelEntry.from_entry() function fails to read the expires from the options variable and insert it into fields.

Even worse, the database model doesn't even accomodate offsets: djcelery.models.PeriodicTask.expires is a DateTimeField, but the Celery documentation documents it as:

Either a int, describing the number of seconds, or a datetime object that describes the absolute time and date of when the task should expire. The task will not be executed after the expiration time.

The scheduler itself appears to have support for setting the expires on the task in the ModelEntry constructor, so I believe it would schedule the task with the right option if it was able to read that information from the database.

After removing the CELERYBEAT_SCHEDULER and using the default scheduler, everything seems to work as intended.

Activity

added a commit that references this issue on Dec 7, 2018
567a7b7
linked a pull request that will close this issue on Dec 7, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      Participants

      @hheimbuerger

      Issue actions

        DatabaseScheduler ignores 'expires' option · Issue #393 · celery/django-celery