You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have a table that is parittioned by a very long URL. When I try to create a share using the PARTITION command I am getting an Internal Error.
Command:
ALTER SHARE my_share ADD table my_catalog.my_schema.my_table
partition (url = 'https://some-files-from-the-internet.com/machine-readable-files/Public/2023-03-13_usable_company_dba_this_is_a_company_name_here_are_some_files_1_of_2.json.gz');
Error:
AnalysisException Traceback (most recent call last)
File <command-3224760991686733>:7
5 display(df)
6 return df
----> 7 _sqldf = ____databricks_percent_sql()
8 finally:
9 del ____databricks_percent_sql
File /databricks/spark/python/pyspark/instrumentation_utils.py:48, in _wrap_function.<locals>.wrapper(*args, **kwargs)
46 start = time.perf_counter()
47 try:
---> 48 res = func(*args, **kwargs)
49 logger.log_success(
50 module_name, class_name, function_name, time.perf_counter() - start, signature
51 )
52 return res
File /databricks/spark/python/pyspark/sql/session.py:1387, in SparkSession.sql(self, sqlQuery, args, **kwargs)
1385 try:
1386 litArgs = {k: _to_java_column(lit(v)) for k, v in (args or {}).items()}
-> 1387 return DataFrame(self._jsparkSession.sql(sqlQuery, litArgs), self)
1388 finally:
1389 if len(kwargs) > 0:
File /databricks/spark/python/lib/py4j-0.10.9.5-src.zip/py4j/java_gateway.py:1321, in JavaMember.__call__(self, *args)
1315 command = proto.CALL_COMMAND_NAME +\
1316 self.command_header +\
1317 args_command +\
1318 proto.END_COMMAND_PART
1320 answer = self.gateway_client.send_command(command)
-> 1321 return_value = get_return_value(
1322 answer, self.gateway_client, self.target_id, self.name)
1324 for temp_arg in temp_args:
1325 temp_arg._detach()
File /databricks/spark/python/pyspark/errors/exceptions.py:234, in capture_sql_exception.<locals>.deco(*a, **kw)
230 converted = convert_exception(e.java_exception)
231 if not isinstance(converted, UnknownException):
232 # Hide where the exception came from that shows a non-Pythonic
233 # JVM exception message.
--> 234 raise converted from None
235 else:
236 raise
AnalysisException: [RequestId=a822a302-12f2-4fb4-b181-a638b39f3171 ErrorClass=INTERNAL_ERROR]
From testing around it seems like there may be a character length issue. If I keep the character length under 117 characters it will work. Is there a work around without having to modify the partition key?
The text was updated successfully, but these errors were encountered:
I am using Databricks on AWS using DBR 11.3LTS.
I have a table that is parittioned by a very long URL. When I try to create a share using the PARTITION command I am getting an Internal Error.
Command:
Error:
From testing around it seems like there may be a character length issue. If I keep the character length under 117 characters it will work. Is there a work around without having to modify the partition key?
The text was updated successfully, but these errors were encountered: