question

RajD-9527 avatar image
0 Votes"
RajD-9527 asked azure-cxp-api edited

Automate spin up of a cluster and an azure databricks workspace

Hi, I am working on spinning up a cluster and a databricks workspace in a Resource Group. The idea is to create a template and minimize manual intervention, using the below code and not sure on how to setup the yaml file. If there any resources for doing same could you please share.

 import json
 import logging
 import yaml
 import base64
 import requests
 import time
 from azure.common.credentials import ServicePrincipalCredentials
 from azure.mgmt.resource import ResourceManagementClient
 from azure.mgmt.resource.resources.models import DeploymentMode
     
 YAML_VARS = "DemoDBWorkspace\\databricks_workspace_vars.yaml"
 JSON_TEMPLATE_PATH = "DemoDBWorkspace\\databricks_premium_workspaceLab.json"
 RESOURCE_GROUP_PARAMS = {"location": "eastus"}
 RESOURCE_GROUP_NAME = "DemoRG"
 JSON_REQUEST_PATH = "DemoDBWorkspace\\deploy_databricks_cluster_restapi.json"

Thank you

azure-databricks
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

PRADEEPCHEEKATLA-MSFT avatar image
0 Votes"
PRADEEPCHEEKATLA-MSFT answered RajD-9527 commented

Hello @RajD-9527,

Thanks for the question and using Microsoft Q&A platform.

To deploy Workspace using the ARM template.

Note: ARM templates are utilized in order to deploy Azure Databricks workspace.

Automation options - There are a few options available to use the Azure Databricks APIs:

  • Databricks Terraform Resource Provider could be combined with Azure provider to create an end-to-end architecture, utilizing Terraform’s dependency and state management features.

  • Python (or any other programming language) could be used to invoke the APIs (sample solution) providing a way to integrate with third-party or homegrown DevOps tooling.

  • A readymade API client like Postman could be used to invoke the API directly.

For more details, refer Automate Azure Databricks Platform Provisioning and Configuration.

Hope this helps. Do let us know if you any further queries.


Please don’t forget to Accept Answer and Up-Vote wherever the information provided helps you, this can be beneficial to other community members.

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi PRADEEPCHEEKATLA-MSFT, Thanks very much for your response.

Regards

0 Votes 0 ·