Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Incorrect cast of string columns containing various infinity notations with trailing spaces #10794

Open
Tracked by #10771
gerashegalov opened this issue May 10, 2024 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@gerashegalov
Copy link
Collaborator

gerashegalov commented May 10, 2024

The actual value that the UT caught is infinity but it boils down to the trailing space

Repro:

$ ~/dist/spark-3.3.0-bin-hadoop3/bin/spark-shell \
  --conf spark.plugins=com.nvidia.spark.SQLPlugin \
  --conf spark.rapids.sql.test.enabled=true \
  --conf spark.rapids.sql.explain=ALL \
  --jars dist/target/rapids-4-spark_2.12-24.06.0-SNAPSHOT-cuda11.jar 

GPU:

scala> Seq("infinity ", "Infinity ", "+Inf ").toDF.coalesce(1).selectExpr("cast(value as float)").collect()
24/05/10 12:02:57 WARN GpuOverrides: 
*Exec <ProjectExec> will run on GPU
  *Expression <Alias> cast(value#1 as float) AS value#5 will run on GPU
    *Expression <Cast> cast(value#1 as float) will run on GPU
  *Exec <CoalesceExec> will run on GPU
    ! <LocalTableScanExec> cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.execution.LocalTableScanExec
      @Expression <AttributeReference> value#1 could run on GPU

res0: Array[org.apache.spark.sql.Row] = Array([null], [null], [null])

CPU:

scala> spark.conf.set("spark.rapids.sql.enabled", false)

scala> Seq("infinity ", "Infinity ", "+Inf ").toDF.coalesce(1).selectExpr("cast(value as float)").collect()
res2: Array[org.apache.spark.sql.Row] = Array([Infinity], [Infinity], [Infinity])
@gerashegalov gerashegalov changed the title cast( infinity as float), actual: null, expected: Infinity [BUG] cast( infinity as float), actual: null, expected: Infinity May 10, 2024
@gerashegalov gerashegalov changed the title [BUG] cast( infinity as float), actual: null, expected: Infinity [BUG] Incorrect cast of string columns containing various infinity notations with trailing spaces May 10, 2024
@gerashegalov gerashegalov added bug Something isn't working ? - Needs Triage Need team to review and classify labels May 10, 2024
@sameerz sameerz removed the ? - Needs Triage Need team to review and classify label May 14, 2024
@Feng-Jiang28 Feng-Jiang28 self-assigned this May 17, 2024
@Feng-Jiang28
Copy link
Collaborator

Failed on corner case: cast_string_to_float.cu: check_for_inf didn't consider trailing white spaces.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants