Skip to content

[Delta] Keep table protocol intact before and after REPLACE regardless of default CC enablement #4782

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

xzhseh
Copy link
Contributor

@xzhseh xzhseh commented Jun 17, 2025

Which Delta project/connector is this regarding?

  • Spark
  • Standalone
  • Flink
  • Kernel
  • Other (fill in here)

Description

This PR keeps table protocol intact before and after REPLACE commands.

Under the current implementation, we have the following scenario that may automatically/silently upgrade a normal delta table's Protocol when using REPLACE commands. It would be upgraded from (x, x) (i.e., any combinations of protocol versions) to (3, 7), plus an additional (unintended) VacuumProtocolCheckTableFeature.

-- w/ protocol version (1, 2) if DV is turned off.
1. CREATE TABLE t (id LONG) USING delta;

-- Enable CO by default in the current SparkSession.
2. spark.conf.set("default CO conf", "supported")

-- The protocol would be automatically upgraded to (3, 7) w/ an extra VacuumProtocolCheckTableFeature.
3. REPLACE TABLE t (id LONG) USING delta;

How was this patch tested?

N/A

Does this PR introduce any user-facing changes?

N/A

@xzhseh xzhseh force-pushed the xzhseh/keep-protocol-intact-before-and-after-replace branch from 2d6f37e to f2c3ef7 Compare June 24, 2025 19:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant