question

JoshRobinson-0107 avatar image
0 Votes"
JoshRobinson-0107 asked JoshRobinson-0107 commented

Using databricks workspace in the same configuration as the databricks provider

I'm having some trouble getting the azurerm & databricks provider to work together.

With the azurerm provider, setup my workspace

 resource "azurerm_databricks_workspace" "ws" {
   name                        = var.workspace_name
   resource_group_name         = azurerm_resource_group.rg.name
   location                    = azurerm_resource_group.rg.location
   sku                         = "premium"
   managed_resource_group_name = "${azurerm_resource_group.rg.name}-mng-rg"
   custom_parameters {
     virtual_network_id  = data.azurerm_virtual_network.vnet.id
     public_subnet_name  = var.public_subnet
     private_subnet_name = var.private_subnet
   }
 }

No matter how I structure this, I can't say seem to get the azurerm_databricks_workspace.ws.id to work in the provider statement for databricks in the the same configuration. If it did work, the above workspace would be defined in the same configuration and I'd have a provider statement that looks like this:

 provider "databricks" {
   azure_workspace_resource_id = azurerm_databricks_workspace.ws.id
 }

I've tried in the same configuration & in a module and consuming outputs. The only way I can get it to work is by running one configuration for the workspace and a second configuration to consume the workspace.

This is super suboptimal in that I have a fair amount of repeating values across those configurations and it would be ideal just to have one.

Has anyone been able to do this? Thank you :)


azure-databricks
· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @JoshRobinson-0107,

Welcome to the Microsoft Q&A platform.

When you try to run the above configuration are experiencing any error message?

Could you please share the error message which you are experiencing for deeper investigation?

1 Vote 1 ·

95334-image.png



The error message #3 is the one I'm attempting. I'm operating as a service principal with Contributor or above on subscription. I have all of my ARM_* environment variables successfully set.

I should note that if I split my configuration, one for the workspace and one for databricks provider and output the host. If I do this, I'm able to create the workspace and output the host with "http://" wrapping it and set it as my host in the configuration. This works well.. but it's a lot more workflow than I'd prefer and others have found this unnecessary.

0 Votes 0 ·
image.png (21.3 KiB)

Here's the error message.
95603-image.png



The authentication method that I'm using is #3 - workspace id & AZ CLI auth.
In this case I'm using a bash script file which I execute like source set_env.sh. It includes ARM_CLIENT_ID, ARM_CLIENT_SECRET, ARM_TENANT_ID, ARM_SUBSCRIPTION_ID and anything for TF_VARS*, or terraform debug flags when I need them.

I use this file for all of my terraform development and these settings work great for everything but this scenario.

NB: attaching the networking was pretty straight forward and didn't give me much issue after I figured out how to properly specify it.

I'd love to say you have the answer.. but I might be onto supplying an issue. There might be a bug in my version of the databricks provider and I had a similar response from one of the maintainers of the code on an SO posting that mirrors this one.

0 Votes 0 ·
image.png (21.3 KiB)

1 Answer

PRADEEPCHEEKATLA-MSFT avatar image
0 Votes"
PRADEEPCHEEKATLA-MSFT answered

Hello @JoshRobinson-0107,

From the error message, it looks like Authentication is not configured for provider could you please configure it through the one of following options mentioned above.

For more details, refer Databricks provider - Authentication.

For passing the custom_parameters, you may checkout the SO thread which addressing the similar issue.

In case if you need more help on this issue, I would suggest to open an issue here: https://github.com/terraform-providers/terraform-provider-azurerm/issues

Hope this helps. Do let us know if you any further queries.


Please don’t forget to Accept Answer and Up-Vote wherever the information provided helps you, this can be beneficial to other community members.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.