You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[SPARK-52290][SQL][BUILD] Fix the Maven compilation and testing issues for sql/pipelines module
### What changes were proposed in this pull request?
The following modifications are mainly made in this PR:
1. Add the dependency `org.scala-lang:scala-library` to fix:
```
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:4.9.2:doc-jar (attach-scaladocs) on project spark-pipelines_2.13: MavenReportException: Error while creating archive: wrap: Process exited with an error: 1 (Exit value: 1) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <args> -rf :spark-pipelines_2.13
```
2. Add the dependency `org.apache.spark:spark-core` to fix:
```
*** RUN ABORTED ***
A needed class was not found. This could be due to an error in your runpath. Missing class: org/apache/spark/internal/config/Tests$
java.lang.NoClassDefFoundError: org/apache/spark/internal/config/Tests$
at org.apache.spark.SparkFunSuite.beforeAll(SparkFunSuite.scala:94)
at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:69)
at org.scalatest.Suite.callExecuteOnSuite$1(Suite.scala:1178)
at org.scalatest.Suite.$anonfun$runNestedSuites$1(Suite.scala:1225)
at scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1324)
at org.scalatest.Suite.runNestedSuites(Suite.scala:1223)
at org.scalatest.Suite.runNestedSuites$(Suite.scala:1156)
...
Cause: java.lang.ClassNotFoundException: org.apache.spark.internal.config.Tests$
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
at org.apache.spark.SparkFunSuite.beforeAll(SparkFunSuite.scala:94)
at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:69)
at org.scalatest.Suite.callExecuteOnSuite$1(Suite.scala:1178)
at org.scalatest.Suite.$anonfun$runNestedSuites$1(Suite.scala:1225)
```
### Why are the changes needed?
Fix the Maven compilation and testing issues for `sql/pipelines` module, maven daily test failed after SPARK-52166
- https://github.com/apache/spark/actions/runs/15212144791
- https://github.com/apache/spark/actions/runs/15211074246
- https://github.com/apache/spark/actions/runs/15213443142
- https://github.com/apache/spark/actions/runs/15218341216

### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
- Pass GitHub Actions
- Local check:
```
build/mvn clean install -pl sql/pipelines -am -DskipTests
build/mvn test -pl sql/pipelines
```
```
Run completed in 356 milliseconds.
Total number of tests run: 9
Suites: completed 3, aborted 0
Tests: succeeded 9, failed 0, canceled 0, ignored 0, pending 0
All tests passed.
```
### Was this patch authored or co-authored using generative AI tooling?
No
Closes#51008 from LuciferYang/SPARK-52290.
Authored-by: yangjie01 <[email protected]>
Signed-off-by: yangjie01 <[email protected]>
0 commit comments