Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for BOOLEAN, TIMESTAMP, and VARBINARY types for data column in Delta Lake connector $partitions system table #21878

Open
jayzzh opened this issue May 8, 2024 · 0 comments
Labels
delta-lake Delta Lake connector

Comments

@jayzzh
Copy link
Contributor

jayzzh commented May 8, 2024

We should add support for BOOLEAN, TIMESTAMP, and VARBINARY types for data column in Delta Lake connector $partitions system table.

For example, consider the following query with TIMESTAMP case:

CREATE TABLE randomTable(_nonpartition TIMESTAMP(6), _partition VARCHAR) WITH (partitioned_by = ARRAY['_partition']
INSERT INTO randomTable VALUES "VALUES (TIMESTAMP '2001-05-06 12:34:56.123456', 'a'), ( TIMESTAMP '2001-05-06 12:34:56.567890', 'a'), (TIMESTAMP '2001-05-06 12:34:56.123456', 'b'), (TIMESTAMP '2001-05-06 12:34:56.123457', 'b')
SELECT data._nonpartition  FROM "randomTable$partitions"

This results in:

VALUES 
 ROW(ROW(NULL, NULL, 0)),
 ROW(ROW(NULL, NULL, 0))

We should change it so it results in:

VALUES 
 ROW(ROW(TIMESTAMP '2001-05-06 12:34:56.123456', TIMESTAMP '2001-05-06 12:34:567890', 0)),
 ROW(ROW(TIMESTAMP '2001-05-06 12:34:56.123456', TIMESTAMP '2001-05-06 12:34:56.123457', 0))

Related: #20993

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
delta-lake Delta Lake connector
Development

No branches or pull requests

1 participant