Skip to content

HIVE-28773: CBO fails when a materialized view is dropped but its pre-compiled plan remains in the registry. #5665

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

kasakrisz
Copy link
Contributor

@kasakrisz kasakrisz commented Feb 28, 2025

What changes were proposed in this pull request?

  • The materialized view registry is initialized by loading all MVs from HMS that are enabled for query rewrite and precompiled. Later, a scheduled thread refreshes the registry periodically in the same way. This patch adds logic to remove MVs from the registry that no longer exist in HMS or have been disabled for query rewrite since the last refresh.
  • Rename MaterializedViewCache to MaterializedViewMap because it is not a cache.
  • Modify the way MVs are stored in the map: remove the database grouping and add the database name as a key prefix to the MV name. Since all MVs are queried and added to the planner during query rewrite, this approach aligns better with the most common function of the registry/map.

Why are the changes needed?

Dropped MVs may casue CBO failure. See jira HIVE-28773 for details

Does this PR introduce any user-facing change?

Yes. CBO failure is intermittent in clusters with multiple HS2s and depending on the query the whole compilation process could fail. This patch fixes these intermittent issues.

Is the change a dependency upgrade?

No.

How was this patch tested?

mvn test -Dtest=TestQueryRewrite,TestHiveMaterializedViewRegistry -pl itests/hive-unit -Pitests
mvn test -Dtest=TestMaterializedViewMap -pl ql

Copy link

…-compiled plan remains in the registry.
Copy link

Copy link

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the [email protected] list if the patch is in need of reviews.

@github-actions github-actions bot added the stale label Jul 12, 2025
@deniskuzZ deniskuzZ removed the stale label Jul 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants