-
Notifications
You must be signed in to change notification settings - Fork 370
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[VL] Unsupported spark function list [please leave a comment if you plan to pick some] #4039
Comments
I'd like support hex and unhex. update: hex and unhex has already supported in Gluten. |
Hi i'd like to give a try with hour function. |
Hi, I'd like to have a look into |
Hi I'd like to support |
Hi, I'd like to support date_trunc/trunc. |
Hi, I'd like to support |
|
The two stand for the same function I assume? I'll take these two if nobody is working on it. |
Yes, they are one thing. Just unify them into one checkbox. Thanks! |
I will take a look |
ubase64: |
Is there any plan to suppport from_json function? |
I'd like take |
I'd like to give |
Just removed the below functions from the list, since they have been supported. Thanks! @acvictor, @Yohahaha, @fyp711, @zwangsheng, @JkSelf, etc.
|
@PHILO-HE I see support for |
|
Thanks so much for your feedback! Just removed it from the list. |
Will do minute as well. |
I'd like to work on locate and arrayintersect. |
I would like to work on |
@PHILO-HE Should we uncheck these two? I ran a test and the two functions are both fallen back (in 3.3). |
I would like to give |
and exists(array) also support |
hey @zhouyuan could you help add |
@gaoyangxiaozhu, just added them into the list. Thanks! |
I'd like to take |
I'd like to take cot, thanks. |
I'd like and am working in the math function expm1. |
PR for width_bucket support, #5634 looks still need velox side change for to support case as bucket_number <=0, will send PR in velox repository to fix |
I'd like to implement |
I'd like to take a look at |
@xumingming Currently, 4 generator functions are supported : |
@marin-ma Thanks for the advice, I will take a look. |
I'd like to take unix_date, thanks. |
@NEUpanning, we have supported it in both Gluten & Velox. Just changed its state in the list. Thanks! |
@PHILO-HE Thanks for your feedback. So i'd like to take date_part. Is to_date supported in gluten now? It doesn't shows in the list. I also would like to pick it. |
@NEUpanning, this list only maintains working-in-progress functions. I think
Line 808 in d74fc97
|
I can't find any implementation of date_part and to_date function in Velox. Would you like to help me find it? Thanks. |
|
I will take a look at |
@NEUpanning, not a direct replacement. date_part is covered here. to_date is converted to Cast + GetTimestamp by Spark. |
@xumingming, seems |
As I see only |
@PHILO-HE array_sort is marked as supported in the doc:
And there is a test for Line 846 in d35d1dc
|
@xumingming, this test only confirms aggregate is offloaded. In my local test, |
@PHILO-HE I can try to support |
I see you've map the from_base64 to unbase64, and respectively I find the base64 is almost the same as to_base64, so it's just a missing or is there any other consideration? |
@Donvi, seems there are a few semantic differences between Spark's |
Description
Here listed spark functions still not supported by Gluten Velox backend. Please leave a comment if you'd like to pick some. In the below list, [√] means someone is working in progress for the corresponding function.
You can find all functions' support status from this gluten doc.
To avoid duplicate work, before starting, please check whether a PR has been submitted in Velox community or whether it has already been implemented in Velox who holds most sql functions in its sparksql folder & prestosql folder.
Reference:
The text was updated successfully, but these errors were encountered: