Feat: Visualize DAG version changes and mixed versions in Grid view #53216

choo121600 · 2025-07-12T03:13:23Z

Summary

Adds visual indicators in the Grid view to help users identify DAG version changes and detect mixed-version TaskInstances within a single DagRun.

Problem

After updating a DAG file, it's difficult to know from which DagRun the new version is applied.
Some TaskInstances within the same DagRun may use different DAG versions, but this is not visible in the UI.

Solution

This PR introduces visual cues in the Grid view to address these pain points:

Version Change Indicator(Vertical):
- A vertical orange line on the left of a DagRun indicates a DAG version change.
- The label shows the new version number (e.g., v2).
Mixed Version Indicator(Vertical):
- A vertical orange line on the right of a DagRun denotes that multiple DAG versions were used in its TaskInstances.
TaskInstance Version Dividers(Horizontal):
- Horizontal orange lines separate TaskInstances using different DAG versions within the same DagRun.

Indicator Scenarios

Scenario 1: Normal

All tasks run with the same version
No indicator (default state)

Scenario 2: Version Change

Entire run with new version
Left indicator + version number

Scenario 3: Mixed Version

Multiple versions within single run
Right indicator + latest version number

Scenario 4: Priority Applied

Version Change occurs after Mixed
Previous Mixed indicator hidden → Version Change displayed

Scenario 5: Complex Pattern

All situations occurring consecutively
Priority rules applied multiple times

Indicator Position Rules

🟧┃ Left: Version Change (entire run with new version)
┃🟧 Right: Mixed Version (multiple versions in Dag Run)

Priority

Version Change > Mixed Version

When Version Change occurs, previous Mixed indicator is automatically hidden
Information priority applied for visual clarity

flowchart TB
    subgraph "Indicators Scenarios"
        subgraph scenario1["Scenario 1"]
            direction LR
            bar1_1["<br/>█<br/>█<br/>█<br/>█<br/>█<br/>v1"]
            bar1_2["<br/>█<br/>█<br/>█<br/>█<br/>█<br/>v1"]
            bar1_3["<br/>█<br/>█<br/>█<br/>█<br/>█<br/>v1"]
            bar1_1 --- bar1_2
            bar1_2 --- bar1_3
        end
        
        subgraph scenario2["Scenario 2"]
            direction LR
            bar2_1["<br/>█<br/>█<br/>█<br/>█<br/>█<br/>v1"]
            bar2_2["<br/>█<br/>█<br/>█<br/>█<br/>█<br/>v1"]
            change_indicator2["🟧<br/>┃<br/>┃<br/>┃<br/>┃<br/>v2"]
            bar2_3["<br/>█<br/>█<br/>█<br/>█<br/>█<br/>v2"]
            bar2_4["<br/>█<br/>█<br/>█<br/>█<br/>█<br/>v2"]
            bar2_1 --- bar2_2
            bar2_2 --- change_indicator2
            change_indicator2 --- bar2_3
            bar2_3 --- bar2_4
        end
        
        subgraph scenario3["Scenario 3"]
            direction LR
            bar3_1["<br/>█<br/>█<br/>█<br/>█<br/>█<br/>v1"]
            bar3_2["<br/>█<br/>█<br/>█<br/>█<br/>█<br/>v1"]
            mixed_bar3["<br/>█<br/>█<br/>█<br/>█<br/>█<br/>v1→v3"] 
            mixed_indicator3["🟧<br/>┃<br/>┃<br/>┃<br/>┃<br/>v3"]
            bar3_4["<br/>█<br/>█<br/>█<br/>█<br/>█<br/>v3"]
            bar3_1 --- bar3_2
            bar3_2 --- mixed_bar3
            mixed_bar3 --- mixed_indicator3
            mixed_indicator3 --- bar3_4
        end
        
        subgraph scenario4["Scenario 4"]
            direction LR
            bar4_1["<br/>█<br/>█<br/>█<br/>█<br/>█<br/>v1"]
            prev_mixed4["<br/>█<br/>█<br/>█<br/>█<br/>█<br/>v2→v3"]
            prev_mixed_ind["🟧<br/>┃<br/>┃<br/>┃<br/>┃<br/>v3"]
            change_ind4["🟧<br/>┃<br/>┃<br/>┃<br/>┃<br/>v4"]
            new_bar4["<br/>█<br/>█<br/>█<br/>█<br/>█<br/>v4"]
            bar4_5["<br/>█<br/>█<br/>█<br/>█<br/>█<br/>v4"]
            
            bar4_1 --- prev_mixed4
            prev_mixed4 --- prev_mixed_ind
            prev_mixed_ind -.-> |"Hidden by Priority"| change_ind4
            change_ind4 --- new_bar4
            new_bar4 --- bar4_5
        end
        
        subgraph scenario5["Scenario 5"]
            direction LR
            c1["<br/>█<br/>█<br/>█<br/>█<br/>█<br/>v1"]
            
            ci2["🟧<br/>┃<br/>┃<br/>┃<br/>┃<br/>v2"]
            c2["<br/>█<br/>█<br/>█<br/>█<br/>█<br/>v2"]
            
            c3["<br/>█<br/>█<br/>█<br/>█<br/>█<br/>v2→v4"]
            mi3["🟧<br/>┃<br/>┃<br/>┃<br/>┃<br/>v4"]
            
            ci4["🟧<br/>┃<br/>┃<br/>┃<br/>┃<br/>v5"]
            c4["<br/>█<br/>█<br/>█<br/>█<br/>█<br/>v5"]
            
            c5["<br/>█<br/>█<br/>█<br/>█<br/>█<br/>v5→v6"]
            mi5["🟧<br/>┃<br/>┃<br/>┃<br/>┃<br/>v6"]
            
            c6["<br/>█<br/>█<br/>█<br/>█<br/>█<br/>v6"]
            
            c1 --- ci2
            ci2 --- c2
            c2 --- c3
            c3 --- mi3
            mi3 -.-> |"Hidden by Priority"| ci4
            ci4 --- c4
            c4 --- c5
            c5 --- mi5
            mi5 --- c6
        end
    end
    
    style bar1_1 fill:#e8f5e8
    style bar1_2 fill:#e8f5e8
    style bar1_3 fill:#e8f5e8
    style bar2_1 fill:#e8f5e8
    style bar2_2 fill:#e8f5e8
    style bar2_3 fill:#d4edda
    style bar2_4 fill:#d4edda
    style bar3_1 fill:#e8f5e8
    style bar3_2 fill:#e8f5e8
    style mixed_bar3 fill:#fff3cd
    style bar3_4 fill:#d4edda
    style bar4_1 fill:#e8f5e8
    style prev_mixed4 fill:#fff3cd
    style prev_mixed_ind fill:#f8f9fa,stroke-dasharray: 5 5,opacity:0.5
    style new_bar4 fill:#d4edda
    style bar4_5 fill:#d4edda
    style c1 fill:#e8f5e8
    style c2 fill:#d4edda
    style c3 fill:#fff3cd
    style mi3 fill:#f8f9fa,stroke-dasharray: 5 5,opacity:0.5
    style c4 fill:#d4edda
    style c5 fill:#fff3cd
    style c6 fill:#d4edda
    style mixed_indicator3 fill:#fd7e14,color:#fff
    style change_indicator2 fill:#fd7e14,color:#fff
    style change_ind4 fill:#fd7e14,color:#fff
    style ci2 fill:#fd7e14,color:#fff
    style ci4 fill:#fd7e14,color:#fff
    style mi5 fill:#fd7e14,color:#fff

Change

Backend

Introduced a new DagVersionService that:

Efficiently detects version changes and mixed-version runs using a single optimized query.
Returns version metadata to the UI through the /grid/runs/{dag_id} API.

Extended API response models to include:

dag_version_id, dag_version_number, latest_version_number
is_version_changed, has_mixed_versions

Frontend

Added VersionIndicator component in Grid bars to display version change and mixed-version status.
Integrated version dividers between TaskInstances in TaskInstancesColumn.tsx.
Ensured accessibility with aria-label for screen reader support.
Used orange theme (orange.400 for background, orange.700 for version text).

Testing

Add extract_dynamic_fields() helper for UUID validation and dynamic field extraction
Replace hardcoded test expectations with dynamic validation for version indicator fields
Ensure all grid API responses properly validate dag_version_id, dag_version_number, is_version_changed, has_mixed_versions
Refactor tests to handle dynamic UUID fields while maintaining comprehensive coverage

closes: #52286

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

bugraoz93

Thanks for the PR! I think this is bringing back the deprecated and removed monolith endpoint with its methods. I think we should integrate the fix into only new structure
cc: @dstandish @pierrejeambrun

choo121600 · 2025-07-12T08:52:27Z

@bugraoz93 Thanks for your quick feedback! I saw something strange while fixing the conflict,
So I started checking the main branch — your comment helped me confirm it.

I think my endpoints follow the old style. I’ll change them to use the new structure.

bugraoz93 · 2025-07-12T09:07:17Z

@bugraoz93 Thanks for your quick feedback! I saw something strange while fixing the conflict,
So I started checking the main branch — your comment helped me confirm it.

I think my endpoints follow the old style. I’ll change them to use the new structure.

Amazing, thanks a lot!

airflow-core/src/airflow/api_fastapi/core_api/routes/ui/grid.py

choo121600 · 2025-07-13T07:58:13Z

I’ve updated the tests and removed the old structure,
but I'm testing whether the feature I added works as before🥲
After verifying the tests, I will leave a comment!

choo121600 · 2025-07-13T09:08:44Z

I’ve tested the scenarios I had in mind, and everything looks good so far 🙌
Reviews are welcome from this point.

bugraoz93

Thanks for the changes! I believe this PR still contains old deprecated code :)

bugraoz93 · 2025-07-14T12:09:09Z

airflow-core/src/airflow/api_fastapi/core_api/services/ui/grid.py

+    return 1
+
+
+def fill_task_instance_summaries(


This is still from old method

bugraoz93 · 2025-07-14T12:10:16Z

airflow-core/src/airflow/api_fastapi/core_api/services/ui/grid.py


 log = structlog.get_logger(logger_name=__name__)


+def get_task_group_map(dag: DAG) -> dict[str, dict[str, Any]]:


This as well

kyungjunleeme · 2025-07-14T13:32:25Z

airflow-core/src/airflow/api_fastapi/core_api/services/ui/dag_version_service.py

+from airflow.models.dag_version import DagVersion
+from airflow.models.dagrun import DagRun
+from airflow.models.taskinstance import TaskInstance
+


if TYPE_CHECKING: from foo import bar ~~~

how about adding some object which only indicate type?

jason810496

Nice! Thanks for the PR!

jason810496 · 2025-07-14T13:29:27Z

airflow-core/src/airflow/api_fastapi/core_api/services/ui/dag_version_service.py

+    def __init__(self, session: SessionDep):
+        self.session = session
+
+    def detect_mixed_versions(self, dag_id: str, dag_run_ids: list[str]) -> dict[str, dict]:


How about adding DagVersionInfo(TypedDict) as type annotation for the return type?

jason810496 · 2025-07-14T13:39:55Z

airflow-core/src/airflow/api_fastapi/core_api/services/ui/dag_version_service.py

+            unique_version_ids = set(v["version_id"] for v in versions)
+            has_mixed_versions = len(unique_version_ids) > 1
+
+            # Get the latest version number if mixed versions exist
+            latest_version_number = None
+            if has_mixed_versions:
+                latest_version_number = max(
+                    v["version_number"] for v in versions if v["version_number"] is not None
+                )


I think we can get the has_mixed_versions and latest_version_number in on loop.

jason810496 · 2025-07-14T13:41:44Z

airflow-core/src/airflow/api_fastapi/core_api/services/ui/dag_version_service.py

+
+    def detect_version_changes(
+        self, dag_runs: list[DagRun], mixed_versions_info: dict[str, dict]
+    ) -> list[dict]:


How about adding TypedDict here for the return type as well ?

choo121600 · 2025-07-15T15:57:13Z

I thought I deleted the old version, but it still there.
Right now, my new code and the old APIs are mixed together, and it looks like the old version is included when I commit 🤔
This might be a problem with my local setup, so I will check and fix it this week.

ephraimbuddy · 2025-07-15T16:50:11Z

If we want to visualize version change, in my opinion, we should do it for bundle version and not dag version that changes everytime especially for dynamic dags cc @jedcunningham

bbovenzi · 2025-07-15T19:11:18Z

If we want to visualize version change, in my opinion, we should do it for bundle version and not dag version that changes everytime especially for dynamic dags cc @jedcunningham

In that case, we might need to use an icon and then show the bundle version in a tooltip. Maybe something like <FiGitCommit />

pierrejeambrun · 2025-07-16T09:31:17Z

If we want to visualize version change, in my opinion, we should do it for bundle version and not dag version that changes everytime especially for dynamic dags cc @jedcunningham

Maybe we need both with different colors, dag version change (to task level) + bundle version change (dag run level only).

If we were to keep only one, I would be in favor of keeping the DagVersion because it is consistant with the Graph(Graph is showing structures of specific dag versions), And because I think source code is less important than what was actually executed.

pierrejeambrun

I think the code looks very complicated for what we are trying to achieve. (And also there are some unused stuff I believe from the legacy grid)

The backend need to serialize the dag_run.versions in the GridRunsResponse, same for tasks.

And in the UI, we just need a simple logic to check if dag_run.version of Nth run is equal to dag_run.version of (N-1)th run.

(Same logic goes for tasks, and yes if DagRun.dag_versions.lenght === 1 no need to go try the task level logic)

Maybe I missed something though.

pierrejeambrun · 2025-07-16T09:53:31Z

airflow-core/src/airflow/ui/src/layouts/Details/Grid/TaskInstancesColumn.tsx

+          <Box bg="orange.400" height="2px" left="0" position="absolute" top="-1px" width="18px" zIndex={3}>
+            <Text
+              bg="white"
+              borderRadius="2px"
+              color="orange.700"
+              fontSize="8px"
+              position="absolute"
+              px="1px"
+              right="-8px"
+              top="-4px"
+              whiteSpace="nowrap"
+            >
+              {`v${taskInstance.dag_version_number ?? ""}`}


Horizontal and Vertical version divider should look exactly the same (Background, space between the head and the bar, etc...)

pierrejeambrun · 2025-07-16T09:57:26Z

In that case, we might need to use an icon and then show the bundle version in a tooltip. Maybe something like

I'm afraid that we can potentially have as many tooltips as TI in the grid. Which would most likely bring back tooltip performance issue we observed in the past.

choo121600 · 2025-07-16T09:59:16Z

Yes, you're right. I missed the Grid part while working on the feature, so I wrote the code based on the old structure 😥
I need to make many changes to match the current Grid.
Also, when I commit from my local setup, some old Grid code is included too. So I think I need to fix my local environment.
It might take some time, so I’ll keep working on it during this week.

choo121600 requested review from bbovenzi, pierrejeambrun, ryanahamilton, jscheffl, shubhamraj-git, ephraimbuddy, rawwar, jason810496 and bugraoz93 as code owners July 12, 2025 03:13

boring-cyborg bot added area:API Airflow's REST/HTTP API area:UI Related to UI/UX. For Frontend Developers. labels Jul 12, 2025

choo121600 force-pushed the feature/version-indicator branch from b3eecb4 to 309e2ea Compare July 12, 2025 04:54

choo121600 added 2 commits July 12, 2025 15:45

Feat: Visualize DAG version changes and mixed versions in Grid view

5f7edbd

Fix some logic error and build fail

bd021e8

choo121600 force-pushed the feature/version-indicator branch from 09ecc2b to bd021e8 Compare July 12, 2025 07:04

bugraoz93 requested changes Jul 12, 2025

View reviewed changes

choo121600 added 2 commits July 12, 2025 22:51

remove the old system

ccbbc17

handle dynamic UUID fields in grid API tests for dag_version_id

7924351

kyungjunleeme reviewed Jul 13, 2025

View reviewed changes

airflow-core/src/airflow/api_fastapi/core_api/routes/ui/grid.py Show resolved Hide resolved

choo121600 requested a review from bugraoz93 July 13, 2025 09:10

Merge branch 'main' into feature/version-indicator

cc88d90

bugraoz93 reviewed Jul 14, 2025

View reviewed changes

kyungjunleeme reviewed Jul 14, 2025

View reviewed changes

jason810496 reviewed Jul 14, 2025

View reviewed changes

pierrejeambrun reviewed Jul 16, 2025

View reviewed changes


		log = structlog.get_logger(logger_name=__name__)


		def get_task_group_map(dag: DAG) -> dict[str, dict[str, Any]]:

Feat: Visualize DAG version changes and mixed versions in Grid view #53216

Are you sure you want to change the base?

Feat: Visualize DAG version changes and mixed versions in Grid view #53216

Conversation

choo121600 commented Jul 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Problem

Solution

Indicator Scenarios

Scenario 1: Normal

Scenario 2: Version Change

Scenario 3: Mixed Version

Scenario 4: Priority Applied

Scenario 5: Complex Pattern

Indicator Position Rules

Priority

Change

Backend

Frontend

Testing

Uh oh!

bugraoz93 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

choo121600 commented Jul 12, 2025

Uh oh!

bugraoz93 commented Jul 12, 2025

Uh oh!

Uh oh!

choo121600 commented Jul 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

choo121600 commented Jul 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bugraoz93 left a comment

Choose a reason for hiding this comment

Uh oh!

bugraoz93 Jul 14, 2025

Choose a reason for hiding this comment

Uh oh!

bugraoz93 Jul 14, 2025

Choose a reason for hiding this comment

Uh oh!

kyungjunleeme Jul 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jason810496 left a comment

Choose a reason for hiding this comment

Uh oh!

jason810496 Jul 14, 2025

Choose a reason for hiding this comment

Uh oh!

jason810496 Jul 14, 2025

Choose a reason for hiding this comment

Uh oh!

jason810496 Jul 14, 2025

Choose a reason for hiding this comment

Uh oh!

choo121600 commented Jul 15, 2025

Uh oh!

ephraimbuddy commented Jul 15, 2025

Uh oh!

bbovenzi commented Jul 15, 2025

Uh oh!

pierrejeambrun commented Jul 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pierrejeambrun left a comment

Choose a reason for hiding this comment

Uh oh!

pierrejeambrun Jul 16, 2025

Choose a reason for hiding this comment

Uh oh!

pierrejeambrun commented Jul 16, 2025

Uh oh!

choo121600 commented Jul 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

choo121600 commented Jul 12, 2025 •

edited

Loading

bugraoz93 left a comment •

edited

Loading

choo121600 commented Jul 13, 2025 •

edited

Loading

choo121600 commented Jul 13, 2025 •

edited

Loading

kyungjunleeme Jul 14, 2025 •

edited

Loading

pierrejeambrun commented Jul 16, 2025 •

edited

Loading

choo121600 commented Jul 16, 2025 •

edited

Loading