Add run_on_latest_version support for backfill and clear operations #52177

ephraimbuddy · 2025-06-24T17:37:45Z

With this option, users are able to choose the dag version they want
to run their dag/task after clearing or when running backfill. This
only applies to versioned bundles as non-versioned bundles run with
the latest dag version.

When the user choose the run with latest version, the bundle_version
associated with the dagrun is updated to the latest and the associated
serialized dag version updated to the latest. Choosing not to run
with latest version which is the default means that the bundle version
and serialized dag version that the dag ran with initially would be used
in running it again.

For backfill, there's now --run-on-latest-version flag that makes it run
with the latest version, otherwise it will run with the original bundle
the dagrun was created with. Note that it's only useful when rerunning
a dagrun using backfill. The default behaviour is using the initial bundle/version
and this is intentional otherwise running backfill will fail if there was
task rename in the latest version.

Summary of changes:

Use SchedulerDagBag instead of DagBag for execution API
Add run_on_latest_version field to DAGRunClearBody and ClearTaskInstancesBody models
Add --run-on-latest-version CLI flag for backfill command
Update backfill.py to support running tasks with latest DAG version
Add UI checkbox for "Run with latest version" in clear dialogs
Update SchedulerDagBag to handle latest version parameter
Update API endpoints to support run_on_latest_version parameter

closes: #49007, closes: #49047

ephraimbuddy · 2025-06-27T15:29:49Z

jason810496

Thanks for the PR. There are some questions about the usage of SchedulerDagBag in core-api.

airflow-core/src/airflow/api_fastapi/app.py

airflow-core/src/airflow/api_fastapi/execution_api/routes/task_instances.py

airflow-core/src/airflow/api_fastapi/execution_api/routes/dag_runs.py

bbovenzi · 2025-06-30T19:10:48Z

UI for dagrun clear and task instance clear look good. We'll add this to backfills next?

kaxil

Design question: instead of --run-on-latest-version, what do you think of --bundle-version "latest" as default?

ephraimbuddy · 2025-07-03T10:39:40Z

Design question: instead of --run-on-latest-version, what do you think of --bundle-version "latest" as default?

I think it makes more sense for backfill but for clearing tasks, we can still keep run on the latest version or do you suggest we should show a form for the user to fill the version they want to run after clearing?

ephraimbuddy · 2025-07-03T10:50:46Z

Design question: instead of --run-on-latest-version, what do you think of --bundle-version "latest" as default?

I think it makes more sense for backfill but for clearing tasks, we can still keep run on the latest version or do you suggest we should show a form for the user to fill the version they want to run after clearing?

Also, by default, it runs with the version the DAG initially used, if it has run before. if we default --bundle-version with the latest, then most rerun backfill will fail if the bundle has changed

kaxil · 2025-07-03T18:28:24Z

Design question: instead of --run-on-latest-version, what do you think of --bundle-version "latest" as default?

I think it makes more sense for backfill but for clearing tasks, we can still keep run on the latest version or do you suggest we should show a form for the user to fill the version they want to run after clearing?

Also, by default, it runs with the version the DAG initially used, if it has run before. if we default --bundle-version with the latest, then most rerun backfill will fail if the bundle has changed

aah ok

kaxil · 2025-07-03T18:29:59Z

Will we ever have a situation where we have both --run-on-latest-version and --bundle-version -- this is the only thing to avoid or have it list as mutually exclusive or somrthign

ephraimbuddy · 2025-07-04T13:15:23Z

Will we ever have a situation where we have both --run-on-latest-version and --bundle-version -- this is the only thing to avoid or have it list as mutually exclusive or somrthign

I think if we need it, then it should be mutually exclusive.

ephraimbuddy · 2025-07-09T14:10:57Z

Will we ever have a situation where we have both --run-on-latest-version and --bundle-version -- this is the only thing to avoid or have it list as mutually exclusive or somrthign

I connected with Jed, and we agreed to mark the flag as experimental for now. This way, if we decide to introduce --bundle-version in the future, we can remove --run-on-latest-version from the CLI without breaking expectations.

kaxil · 2025-07-10T06:43:00Z

Will we ever have a situation where we have both --run-on-latest-version and --bundle-version -- this is the only thing to avoid or have it list as mutually exclusive or somrthign

I connected with Jed, and we agreed to mark the flag as experimental for now. This way, if we decide to introduce --bundle-version in the future, we can remove --run-on-latest-version from the CLI without breaking expectations.

Sounds good

bugraoz93

Looks good! I have one comment around UI-backend, not a blocker for the PR if it is needed. Thanks, Ephraim!

Found the change in migrations, which is already done

airflow-core/src/airflow/ui/src/components/Clear/TaskInstance/ClearTaskInstanceDialog.tsx

airflow-core/src/airflow/cli/cli_config.py

airflow-core/src/airflow/ui/public/i18n/locales/en/dags.json

airflow-core/src/airflow/ui/openapi-gen/requests/schemas.gen.ts

airflow-core/src/airflow/ui/src/components/Clear/Run/ClearRunDialog.tsx

With this option, users are able to choose the dag version they want to run their dag/task after clearing or when running backfill. This only applies to versioned bundles as non-versioned bundles run with the latest dag version. When the user choose the run with latest version, the bundle_version associated with the dagrun is updated to the latest and the associated serialized dag version updated to the latest. Choosing not to run with latest version which is the default means that the bundle version and serialized dag version that the dag ran with initially would be used in running it again. For backfill, there's now --run-on-latest-version flag that makes it run with the latest version, otherwise it will run with the original bundle the dagrun was created with. Note that it's only useful when rerunning a dagrun using backfill. The default behaviour is using the initial bundle/version and this is intentional otherwise running backfill will fail if there was task rename in the latest version. Summary of changes: - Use SchedulerDagBag instead of DagBag for execution API - Add run_on_latest_version field to DAGRunClearBody and ClearTaskInstancesBody models - Add --run-on-latest-version CLI flag for backfill command - Update backfill.py to support running tasks with latest DAG version - Add UI checkbox for "Run with latest version" in clear dialogs - Update SchedulerDagBag to handle latest version parameter - Update API endpoints to support run_on_latest_version parameter

…undle

Co-authored-by: Jed Cunningham <[email protected]>

…s.gen.ts

boring-cyborg bot added area:airflow-ctl area:API Airflow's REST/HTTP API area:Scheduler including HA (high availability) scheduler area:translations area:UI Related to UI/UX. For Frontend Developers. translation:default labels Jun 24, 2025

ephraimbuddy force-pushed the properly-clear-ti-specific-version branch 5 times, most recently from 1f4b2e3 to bfc2a49 Compare June 27, 2025 15:04

ephraimbuddy changed the title ~~Fix clearing dags of versioned bundles and allow running on previous version~~ Add run_on_latest_version support for backfill and clear operations Jun 27, 2025

ephraimbuddy force-pushed the properly-clear-ti-specific-version branch from bfc2a49 to 661e02a Compare June 27, 2025 15:37

ephraimbuddy marked this pull request as ready for review June 27, 2025 15:38

ephraimbuddy requested review from ashb, kaxil, amoghrajesh, bbovenzi, pierrejeambrun, ryanahamilton, jscheffl, shubhamraj-git, bugraoz93, potiuk, rawwar, jason810496 and XD-DENG as code owners June 27, 2025 15:38

jason810496 reviewed Jun 29, 2025

View reviewed changes

kaxil reviewed Jul 2, 2025

View reviewed changes

tirkarthi mentioned this pull request Jul 6, 2025

Allow to switch DAG run to newer dag version #52884

Closed

2 tasks

ephraimbuddy force-pushed the properly-clear-ti-specific-version branch from 9e4faa4 to 8713b05 Compare July 9, 2025 14:39

ephraimbuddy force-pushed the properly-clear-ti-specific-version branch from 8713b05 to 87aa44c Compare July 10, 2025 13:48

ephraimbuddy requested a review from eladkal as a code owner July 10, 2025 13:48

bugraoz93 approved these changes Jul 10, 2025

View reviewed changes

airflow-core/src/airflow/ui/src/components/Clear/TaskInstance/ClearTaskInstanceDialog.tsx Outdated Show resolved Hide resolved

ephraimbuddy mentioned this pull request Jul 10, 2025

WIP: Refactor SchedularDagBag and use it across the RESTAPI #53153

Draft

jedcunningham reviewed Jul 10, 2025

View reviewed changes

ephraimbuddy force-pushed the properly-clear-ti-specific-version branch 2 times, most recently from 13f44b4 to b7ab41b Compare July 14, 2025 09:17

ephraimbuddy and others added 12 commits July 14, 2025 19:48

Only show the run on latest version button when we are on versioned b…

65e5db1

…undle

Fix test

2d825ef

Fix test by using serialized dag

065300b

fixup! Fix test by using serialized dag

2e43d42

Mark the --run-on-latest-version as experimental

c8a95a9

properly save dag to the db in tests

07c9384

Fix ui as taskinstance.dag_version is no longer nullable

400b7a7

Apply suggestions from code review

92f5776

Co-authored-by: Jed Cunningham <[email protected]>

Update airflow-core/src/airflow/ui/openapi-gen/requests/schemas.gen.ts

f068789

Co-authored-by: Jed Cunningham <[email protected]>

fixup! Update airflow-core/src/airflow/ui/openapi-gen/requests/schema…

2c7f12e

…s.gen.ts

Do not show run on latest button if run was on latest

71ec6c1

ephraimbuddy force-pushed the properly-clear-ti-specific-version branch from b7ab41b to 71ec6c1 Compare July 14, 2025 18:49

Add run_on_latest_version support for backfill and clear operations #52177

Are you sure you want to change the base?

Add run_on_latest_version support for backfill and clear operations #52177

Conversation

ephraimbuddy commented Jun 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ephraimbuddy commented Jun 27, 2025

Uh oh!

jason810496 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bbovenzi commented Jun 30, 2025

Uh oh!

kaxil left a comment

Choose a reason for hiding this comment

Uh oh!

ephraimbuddy commented Jul 3, 2025

Uh oh!

ephraimbuddy commented Jul 3, 2025

Uh oh!

kaxil commented Jul 3, 2025

Uh oh!

kaxil commented Jul 3, 2025

Uh oh!

ephraimbuddy commented Jul 4, 2025

Uh oh!

ephraimbuddy commented Jul 9, 2025

Uh oh!

kaxil commented Jul 10, 2025

Uh oh!

bugraoz93 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ephraimbuddy commented Jun 24, 2025 •

edited

Loading

bugraoz93 left a comment •

edited

Loading