You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A partial named columns list in spark_read_x seems to rewrite the names of the first x columns up to the length of the partial list and thereby likely refer to the wrong columns. So it seems like a partial columns list would not be useful and could be misleading / produce unexpected results for most users.
Suggestion: sparklyr should disable this partial list option (error out) or warn that not all columns are included or the columns you thought you were selecting and specifying type for are not the actual ones you are reading.
The text was updated successfully, but these errors were encountered:
A partial named columns list in spark_read_x seems to rewrite the names of the first x columns up to the length of the partial list and thereby likely refer to the wrong columns. So it seems like a partial columns list would not be useful and could be misleading / produce unexpected results for most users.
Suggestion: sparklyr should disable this partial list option (error out) or warn that not all columns are included or the columns you thought you were selecting and specifying type for are not the actual ones you are reading.
The text was updated successfully, but these errors were encountered: