Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Issues found by Spark UT Framework on RapidsStringExpressionsSuite #10775

Open
1 of 8 tasks
binmahone opened this issue May 7, 2024 · 6 comments
Open
1 of 8 tasks
Assignees
Labels
bug Something isn't working

Comments

@binmahone
Copy link
Collaborator

binmahone commented May 7, 2024

Describe the bug

Spark UT Framework enabled RapidsStringExpressionsSuite (#10743), with the following test cases explicitly excluded:

These excluded test cases needs further investigating!!!
Notice: Other test cases in this suite may pass with falling back!

Steps/Code to reproduce bug

  1. Compile everything with mvn -Dbuildver=330 install -DskipTests
  2. Pick a test case name in the above table
  3. Go to RapidsTestSettings and find the line starting with ".exclude" and containing the test case name, comment it out
  4. Run the Suite then you'll see one failed test case. E.g. mvn -nsu -Dbuildver=330 -pl tests -Dsuites="org.apache.spark.sql.rapids.suites.RapidsXXXSuite" test (replace RapidsXXXSuite with the right name in issue header). ALWAYS double check if your suite name coincide with in source code, as it may contain typos!

Expected behavior
The suite can pass without excluding any test case.

@binmahone binmahone added bug Something isn't working ? - Needs Triage Need team to review and classify labels May 7, 2024
@mattahrens
Copy link
Collaborator

Initial scope is triaging unit test failures to determine priorities of individual issues.

@mythrocks
Copy link
Collaborator

All but one of the reported failures disappear when whole-stage codegen is disabled.

There does seem to be a genuine failure in the tests for ParseUrl. I will raise a separate bug to track that one.

@mythrocks
Copy link
Collaborator

I've filed #10810, for which a fix is available at #10811.

@mythrocks
Copy link
Collaborator

@binmahone, @NVnavkumar, what are your thoughts on disabling whole-stage codegen for these tests? It should resolve almost all the failures listed here (save for the last one, which will be fixed with #10811).

@NVnavkumar
Copy link
Collaborator

I think it's the right move. We do need further investigation to disable any of the codegen specific-tests here. I know that in the regexp suite I saw a handful of places that were testing the java output of the codegen step.

@binmahone, @NVnavkumar, what are your thoughts on disabling whole-stage codegen for these tests? It should resolve almost all the failures listed here (save for the last one, which will be fixed with #10811).

@binmahone
Copy link
Collaborator Author

after discussion with Gary and Ferdinand, we have refined the Spark UT Framework by #10851 , which eliminates the necessity of disabling wholeStage. Now the list of the failed tests is refreshed. After PR 10851 is merged, we can move forward to investigate the remaining failed test cases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants