Im trying to create a spark database either using spark.sql("CREATE DATABASE dbname") or df.write.mode("overwrite").saveAsTable(dbname.tablename) and is giving me this error:
AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient;
Traceback (most recent call last):
File “/opt/spark/python/lib/pyspark.zip/pyspark/sql/session.py”, line 767, in sql
return DataFrame(self.jsparkSession.sql(sqlQuery), self.wrapped)
File “/opt/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py”, line 1257, in call
answer, self.gateway_client, self.target_id, self.name)
File “/opt/spark/python/lib/pyspark.zip/pyspark/sql/utils.py”, line 75, in deco
raise AnalysisException(s.split(‘: ’, 1)[1], stackTrace)
pyspark.sql.utils.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient;
It algo gives me the same error when I try to list the tables using: print(spark.catalog.listTables())