Allow read+append against data rearrangement #289
Conversation
[SC-24892] Add typesafe bintray repo for sbt-mima-plugin
|
|
There was a problem hiding this comment.
Thank you for sending this PR! When doing the last round of conflict detection changes, @tdas realized that this case could possibly be allowed but hadn't gotten time to code + test it. He's on vacation right now; do you mind waiting a couple weeks until he's back and can look over it in detail?
| } | ||
| } | ||
|
|
||
| test("bloc delete against data rearrange") { |
There was a problem hiding this comment.
Hi @mukulmurthy, thank you for your response. No problem :)
| @@ -486,6 +486,11 @@ trait OptimisticTransactionImpl extends TransactionalWrite with SQLMetricsReport | |||
| } else { | |||
| changedDataAddedFiles ++= winningCommitActions.collect { case a: AddFile => a } | |||
| } | |||
| val rearrangeOnly = | |||
There was a problem hiding this comment.
You might consider turning both checks into functions, they are used 1-2 more times throughout the file.
|
@tdas can you please take a look on this PR please. |
|
Will take a look. Currently a bit busy trying to get all the critical things in for 0.6.0 release. This PR is a bit tricky, so it wont make it in this release. I will take a look at it after the 0.6.0 release, probably next week. |
|
@tdas still waiting for you to evaluate this PR :) can you please take a look at it. |
This PR improves the conflict detection logic by treating the following case.
when the winning commit rearranges data (enabled by #223) and the current transaction is just reading and appending files, the current commit should be allowed.