Skip to content

Commit 6c6d36c

Browse files
cloud-fangengliangwang
authored andcommitted
[SPARK-52521][SQL] Right#replacement should not access SQLConf dynamically
### What changes were proposed in this pull request? `Right#replacement` is a lazy val that has non-deterministic initialization timing. It's fragile to access SQLConf there as the conf value can be different with different lazy val initialization timing. For example, if we initialize the lazy val during view plan resolution, it will use the ANSI conf from the view's recorded SQL confs, which can be different from the current session's SQL conf. If the `Right` expression appears more than once in the query plan, and their lazy val initialization timing is different, it will lead to inconsistency issues that fail the analysis. This PR fixes `Right#replacement` to always create `Substring` with `failOnError = false`. The length parameter is guaranteed to be positive here, so the `failOnError` flag doesn't matter. Setting it to false makes the generated java code simpler. ### Why are the changes needed? bug fix ### Does this PR introduce _any_ user-facing change? yes, the query failed to analysis can now work ### How was this patch tested? a new test ### Was this patch authored or co-authored using generative AI tooling? no Closes #51210 from cloud-fan/ansi. Lead-authored-by: Wenchen Fan <[email protected]> Co-authored-by: Wenchen Fan <[email protected]> Signed-off-by: Gengliang Wang <[email protected]> (cherry picked from commit 18faa83) Signed-off-by: Gengliang Wang <[email protected]>
1 parent 774db40 commit 6c6d36c

File tree

2 files changed

+18
-1
lines changed

2 files changed

+18
-1
lines changed

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2354,7 +2354,7 @@ case class Right(str: Expression, len: Expression) extends RuntimeReplaceable
23542354
If(
23552355
LessThanOrEqual(len, Literal(0)),
23562356
Literal(UTF8String.EMPTY_UTF8, str.dataType),
2357-
new Substring(str, UnaryMinus(len))
2357+
new Substring(str, UnaryMinus(len, failOnError = false))
23582358
)
23592359
)
23602360

sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1323,4 +1323,21 @@ abstract class SQLViewSuite extends QueryTest with SQLTestUtils {
13231323
assert(ts1._1.getTime < ts2._1.getTime)
13241324
}
13251325
}
1326+
1327+
test("SPARK-52521: view with ANSI expressions") {
1328+
withView("v1") {
1329+
withSQLConf(ANSI_ENABLED.key -> "true") {
1330+
sql(
1331+
"""
1332+
|CREATE VIEW v1 AS
1333+
|SELECT RIGHT(CAST(id AS STRING), 1) AS c
1334+
|FROM range(1)
1335+
|GROUP BY RIGHT(CAST(id AS STRING), 1)
1336+
|""".stripMargin)
1337+
}
1338+
withSQLConf(ANSI_ENABLED.key -> "false") {
1339+
checkAnswer(sql("SELECT * FROM v1"), Row("0"))
1340+
}
1341+
}
1342+
}
13261343
}

0 commit comments

Comments
 (0)