Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ISSUE] row_filter name returned from API not parsed correctly #604

Open
wsilveira-usp opened this issue Apr 1, 2024 · 3 comments
Open
Assignees

Comments

@wsilveira-usp
Copy link

wsilveira-usp commented Apr 1, 2024

Description
Databricks /api/2.1/unity-catalog/tables/{full_name} API actually returns unction_name for row_filters and not name as described in the docs.

Reproduction
A minimal code sample demonstrating the bug.

Row Filter Function creation:

CREATE FUNCTION some_catalog.some_schema.us_filter(b INT)
RETURN IF(IS_ACCOUNT_GROUP_MEMBER('admin'), true, b=1);

Table creation with Row Filter:

CREATE TABLE some_catalog.some_schema.rectangles_v12(
  a integer,
  b integer,
  c decimal(10, 2),
  `interval` INTERVAL YEAR TO MONTH,
  identity_generated_default BIGINT GENERATED BY DEFAULT AS IDENTITY,
  identity_generated_always BIGINT GENERATED ALWAYS AS IDENTITY ( START WITH 2 INCREMENT BY 2 ),
  identity_default timestamp DEFAULT current_timestamp,
  area INT GENERATED ALWAYS AS (a * b)
) 
USING delta 
LOCATION 'abfss://container@storage_acc.dfs.core.windows.net/some_path'
WITH ROW FILTER some_catalog.some_schema.us_filter ON (b)
TBLPROPERTIES (
  delta.feature.allowColumnDefaults = 'supported',
  custom = 'custom_feature'
);

Fetch table's detail via SDK:

from databricks.sdk import WorkspaceClient
w = WorkspaceClient()

print(w.tables.get("some_catalog.some_schema.rectangles_v12"))

Note the Row filter function name is returned empty

Expected behavior

curl -X GET https://<<WORKSPACE ID>>/api/2.1/unity-catalog/tables/some_catalog.some_schema.rectangles_v12 --header "Authorization: Bearer <<PAT TOKEN>>"

Row Filter function_name returnd not empty

Is it a regression?
Don't have past information, but I believe it never worked.

PR Link
PR 603

@mgyucht
Copy link
Contributor

mgyucht commented Apr 4, 2024

Thanks for submitting this and the PR to fix! Unfortunately, this code is autogenerated and will be overwritten by the next SDK release. I'll submit a fix upstream to the underlying API spec so that this is fixed in this SDK and the others as well.

@mgyucht mgyucht self-assigned this Apr 4, 2024
@mgyucht
Copy link
Contributor

mgyucht commented Apr 8, 2024

The issue was fixed in our upstream API definition. This will be released in this week's SDK release.

@wsilveira-usp
Copy link
Author

Ty, @mgyucht !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants