question

ShwetaPrabhu-1076 avatar image
0 Votes"
ShwetaPrabhu-1076 asked PRADEEPCHEEKATLA-MSFT edited

Databricks Hive upgradation from version 1.2.0 to 2.3.7

  • I want to upgrade my Databricks hive version from 1.2.0 to 2.3.7.

  • I am using internal hive metastore and To upgrade hive version I need to upgrade existing schema version of metastore.

  1. what are the steps involved in it? can I use hive schema tool and How?

  2. Is there any custom upgradation script available or any alternate way to upgrade it.

  3. Also How can I take backup of my existing internal hive metastore?


azure-databricks
· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @ShwetaPrabhu-1076,

Welcome to the Microsoft Q&A platform.

I’m working with the product team and get back to you when I have more information.

1 Vote 1 ·

Hello @ShwetaPrabhu-1076,

We are waiting for the product team's response and I will get back to you when I have more information.

Stay tuned!

1 Vote 1 ·

1 Answer

PRADEEPCHEEKATLA-MSFT avatar image
1 Vote"
PRADEEPCHEEKATLA-MSFT answered PRADEEPCHEEKATLA-MSFT edited

Hello @ShwetaPrabhu-1076,

Welcome to the Microsoft Q&A platform.

what are the steps involved in it? can I use hive schema tool and How?

You can try to set "spark.sql.hive.metastore.version" in the Azure Databricks cluster configuration and restart.

41764-image.png

41716-image.png

41817-adb-hivemetastoreupdate.gif

Is there any custom upgradation script available or any alternate way to upgrade it.

You may checkout the Github page which contains hive metastore upgrade scripts: https://github.com/apache/hive/tree/master/metastore/scripts/upgrade

How can I take a backup of my existing internal hive metastore?

You can create table DDLs to import to an external metastore.

Reference: https://kb.databricks.com/metastore/create-table-ddl-for-metastore.html

Hope this helps. Do let us know if you any further queries.


  • Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how.

  • Want a reminder to come back and check responses? Here is how to subscribe to a notification.


image.png (106.2 KiB)
image.png (1.5 KiB)
· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

@PRADEEPCHEEKATLA-MSFT Thank you so much for help.
- Using upgradation script I am able to upgrade hive schema version. I have configured external metastore and upgraded hive schema version using hive upgradation script.
- but in case of internal hive metastore how can we login to underlying database and execute upgradation script?


0 Votes 0 ·

Hello @ShwetaPrabhu-1076,

You can use this notebook to dump the metastore database from MySql or MariaDB used with a Databricks cluster. This can be used for external metastore migration or backup.


0 Votes 0 ·