Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AnalysisException: [RequestId=a822a302-12f2-4fb4-b181-a638b39f3171 ErrorClass=INTERNAL_ERROR] #299

Open
gardnmi opened this issue May 1, 2023 · 2 comments
Assignees

Comments

@gardnmi
Copy link

gardnmi commented May 1, 2023

I am using Databricks on AWS using DBR 11.3LTS.

I have a table that is parittioned by a very long URL. When I try to create a share using the PARTITION command I am getting an Internal Error.

Command:

ALTER SHARE my_share ADD table my_catalog.my_schema.my_table 
partition (url = 'https://some-files-from-the-internet.com/machine-readable-files/Public/2023-03-13_usable_company_dba_this_is_a_company_name_here_are_some_files_1_of_2.json.gz');

Error:

AnalysisException                         Traceback (most recent call last)
File <command-3224760991686733>:7
      5     display(df)
      6     return df
----> 7   _sqldf = ____databricks_percent_sql()
      8 finally:
      9   del ____databricks_percent_sql

File /databricks/spark/python/pyspark/instrumentation_utils.py:48, in _wrap_function.<locals>.wrapper(*args, **kwargs)
     46 start = time.perf_counter()
     47 try:
---> 48     res = func(*args, **kwargs)
     49     logger.log_success(
     50         module_name, class_name, function_name, time.perf_counter() - start, signature
     51     )
     52     return res

File /databricks/spark/python/pyspark/sql/session.py:1387, in SparkSession.sql(self, sqlQuery, args, **kwargs)
   1385 try:
   1386     litArgs = {k: _to_java_column(lit(v)) for k, v in (args or {}).items()}
-> 1387     return DataFrame(self._jsparkSession.sql(sqlQuery, litArgs), self)
   1388 finally:
   1389     if len(kwargs) > 0:

File /databricks/spark/python/lib/py4j-0.10.9.5-src.zip/py4j/java_gateway.py:1321, in JavaMember.__call__(self, *args)
   1315 command = proto.CALL_COMMAND_NAME +\
   1316     self.command_header +\
   1317     args_command +\
   1318     proto.END_COMMAND_PART
   1320 answer = self.gateway_client.send_command(command)
-> 1321 return_value = get_return_value(
   1322     answer, self.gateway_client, self.target_id, self.name)
   1324 for temp_arg in temp_args:
   1325     temp_arg._detach()

File /databricks/spark/python/pyspark/errors/exceptions.py:234, in capture_sql_exception.<locals>.deco(*a, **kw)
    230 converted = convert_exception(e.java_exception)
    231 if not isinstance(converted, UnknownException):
    232     # Hide where the exception came from that shows a non-Pythonic
    233     # JVM exception message.
--> 234     raise converted from None
    235 else:
    236     raise

AnalysisException: [RequestId=a822a302-12f2-4fb4-b181-a638b39f3171 ErrorClass=INTERNAL_ERROR] 

From testing around it seems like there may be a character length issue. If I keep the character length under 117 characters it will work. Is there a work around without having to modify the partition key?

@linzhou-db linzhou-db assigned linzhou-db and zhuansunxt and unassigned linzhou-db May 2, 2023
@linzhou-db
Copy link
Collaborator

@gardnmi Thanks for reporting the issue. We'll investigate.
Are you on aws us west 2?

@gardnmi
Copy link
Author

gardnmi commented May 2, 2023

@gardnmi Thanks for reporting the issue. We'll investigate.
Are you on aws us west 2?

Yes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants