Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-48341][CONNECT] Allow plugins to use QueryTest in their tests #46667

Closed
wants to merge 4 commits into from

Conversation

tomvanbussel
Copy link
Contributor

What changes were proposed in this pull request?

This PR changes QueryTest to no longer depend on RemoteSparkSession.

Why are the changes needed?

This allows the tests for Spark Connect plugin to provide their version of RemoteSparkSession (which depends on some idiosyncrasies of how Spark is built).

Does this PR introduce any user-facing change?

No

How was this patch tested?

Existing tests to ensure that nothing breaks. Manually tested that this allows a plugin to use QueryTest.

Was this patch authored or co-authored using generative AI tooling?

No

@HyukjinKwon
Copy link
Member

Can we file a JIRA ticket please?

Copy link
Member

@HyukjinKwon HyukjinKwon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise, seems fine.

@zhengruifeng
Copy link
Contributor

the failed tests seems related:

[error] /home/runner/work/spark/spark/connector/connect/client/jvm/src/test/scala/org/apache/spark/sql/CatalogSuite.scala:29:28: illegal inheritance;
[error]  self-type org.apache.spark.sql.CatalogSuite does not conform to org.apache.spark.sql.test.RemoteSparkSession's selftype org.apache.spark.sql.test.RemoteSparkSession with org.scalatest.Suite
[error] class CatalogSuite extends RemoteSparkSession with SQLHelper {
[error]                            ^
[error] /home/runner/work/spark/spark/connector/connect/client/jvm/src/test/scala/org/apache/spark/sql/CatalogSuite.scala:31:3: package org.apache.spark.sql.test is not a value

@tomvanbussel tomvanbussel changed the title [CONNECT] Allow plugins to use QueryTest in their tests [SPARK-48341][CONNECT] Allow plugins to use QueryTest in their tests May 21, 2024
@tomvanbussel
Copy link
Contributor Author

@HyukjinKwon Sorry, I filed a ticket (see branch name) but forgot to include it in the PR title. (I knew I was missing something...)

@zhengruifeng Thanks, should be fixed now!

@HyukjinKwon
Copy link
Member

Merged to master.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants