question

AlexanderBHurtado-6467 avatar image
0 Votes"
AlexanderBHurtado-6467 asked PRADEEPCHEEKATLA-MSFT commented

Cannot Create or List any spark Database

Im trying to create a spark database either using spark.sql("CREATE DATABASE dbname") or df.write.mode("overwrite").saveAsTable(dbname.tablename) and is giving me this error:

AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient;
Traceback (most recent call last):
File “/opt/spark/python/lib/pyspark.zip/pyspark/sql/session.py”, line 767, in sql
return DataFrame(self.jsparkSession.sql(sqlQuery), self.wrapped)
File “/opt/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py”, line 1257, in
call

answer, self.gateway_client, self.target_id, self.name)
File “/opt/spark/python/lib/pyspark.zip/pyspark/sql/utils.py”, line 75, in deco
raise AnalysisException(s.split(‘: ’, 1)[1], stackTrace)
pyspark.sql.utils.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient;

It algo gives me the same error when I try to list the tables using: print(spark.catalog.listTables())

azure-synapse-analytics
· 7
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @AlexanderBHurtado-6467,

Welcome to the Microsoft Q&A platform.

In order to investigate further, could you please share the below details:

  • This could be a transient issue. Could you please retry and do let us know.

  • Are you facing this issue only specific to single Spark pool?

  • Could you please try to create a new spark pool and check if the problem exists?

0 Votes 0 ·

Hi, I have been trying for the last 3 days, same result.

I have tested with 3 different Spark Pools with different specs, all of them give me the same result.

0 Votes 0 ·

Hello @AlexanderBHurtado-6467,

Thanks for the confirmation.

We are reaching out to product team to get help on this issue and will update you as soon as we have a response.

Thank you for your patience.

0 Votes 0 ·
Show more comments

Hi @PRADEEPCHEEKATLA-MSFT,

Workspacename: occu-synapse-test

Region: US West

0 Votes 0 ·

Hello @AlexanderBHurtado-6467,

Thanks for sharing the details.

I had shared these details with the product team to get help on this issue and will update you as soon as we have a response.

Thank you for your patience.

0 Votes 0 ·

1 Answer

PRADEEPCHEEKATLA-MSFT avatar image
0 Votes"
PRADEEPCHEEKATLA-MSFT answered PRADEEPCHEEKATLA-MSFT commented

Hello @AlexanderBHurtado-6467,

We had fixed the issue for your workspace. Could you please check and confirm on the same?

This was one of the small set workspaces where we failed to enable SyMS. We created a hotfix from backend to resolve the issue.

Hope this helps. Do let us know if you any further queries.


Please "Accept the answer" if the information helped you. This will help us and others in the community as well.

· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @PRADEEPCHEEKATLA-MSFT

Thanks for the help. Can you please explained a little bit about what happened, so we don't make the same mistake in the future?

0 Votes 0 ·

Hello @AlexanderBHurtado-6467,

It's was an issue from the backend, which we had fixed to resolve the issue.

0 Votes 0 ·

Hello @AlexanderBHurtado-6467,

Just checking in to see if the above answer helped. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.

0 Votes 0 ·