question

PorscheMe-6235 avatar image
0 Votes"
PorscheMe-6235 asked ·

Databricks error: Failed to download bootstrap script from artifact storage

Azure Firewall: Hub & Spoke architecture
Databricks: 7.4 runtime
Azure region: West Us2

I followed this data-exfiltration-protection-with-azure-databricks.html detailed walkthrough guide to setup protect data exfiltration with Azure Firewall and Databricks in our dev environment. After the setup, I was never be able to start my Databricks cluster.

Any help to resolve this error is very much appreciated.

Logs on the cluster

 Cluster terminated. Reason: Storage Download Failure
    
 Failed to download bootstrap script from artifact storage. Please check that instances have connectivity to cloud storage.
 Instance bootstrap failed command: DownloadBootstrapScript
 Failure message: (Base64 encoded) VM extension code: ProvisioningState/succeeded
 ----REDACTED----
 instanceId: InstanceId(3cf2fd8a96e5407cb868d636fdee9d48)
 workerEnv: workerenv-979924835046728
 Additional details (may be truncated):
    
     [Bootstrap Event] Command DownloadBootstrapScript finished. Storage Account: arprodwestus2a1.blob.core.windows.net [FAILED] [Will try other storage accounts]
     [Bootstrap Event] Command DownloadBootstrapScript finished. Storage Account: arprodwestus2a3.blob.core.windows.net [FAILED] [Will try other storage accounts]
     [Bootstrap Event] Command DownloadBootstrapScript finished. Storage Account: dbartifactsprodwestus2.blob.core.windows.net [FAILED] [Will try other storage accounts]
     [Bootstrap Event] Command DownloadBootstrapScript finished. Storage Account: arprodwestusa14.blob.core.windows.net [FAILED] [Will try other storage accounts]
     [Bootstrap Event] Command DownloadBootstrapScript finished. Storage Account: arprodwestusa11.blob.core.windows.net [FAILED] [Will try other storage accounts]
     [Bootstrap Event] Command DownloadBootstrapScript finished. Storage Account: dbartifactsprodwestus.blob.core.windows.net [FAILED] [Will try other storage accounts]
     [Bootstrap Event] {FAILED_COMMAND:DownloadBootstrapScript}
     [Bootstrap Event] {FAILED_MESSAGE:(Base64 encoded) ----REDACTED----}
     [Bootstrap Event] SELF BOOTSTRAP FAILED
     [Bootstrap Event] Can reach arprodwestus2a1.blob.core.windows.net: [SUCCEEDED]
     [Bootstrap Event] Can reach arprodwestus2a3.blob.core.windows.net: [SUCCEEDED]
     [Bootstrap Event] Can reach dbartifactsprodwestus2.blob.core.windows.net: [SUCCEEDED]
     [Bootstrap Event] Can reach arprodwestusa14.blob.core.windows.net: [SUCCEEDED]
     [Bootstrap Event] Can reach arprodwestusa11.blob.core.windows.net: [SUCCEEDED]
     [Bootstrap Event] Can reach dbartifactsprodwestus.blob.core.windows.net: [SUCCEEDED]
     [Bootstrap Event] Can reach westus.azuredatabricks.net: [SUCCEEDED]

Firewall Application Rule Collection

79819-application-databricks-control-plane-services.png

Firewall Network Rule Collection

79874-network-databricks-control-plane-services.png





azure-virtual-networkazure-databricksazure-firewall
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

PorscheMe-6235 avatar image
1 Vote"
PorscheMe-6235 answered ·

This issue is resolved now.

This linkudr was very useful in getting the job done.
I mean data exfiltration protection with Azure Databricks and Azure Firewall Premium.

That being said, working with the firewall was never an easy task.

· 1 ·
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @PorscheMe-6235 ,

Glad to know that your issue has resolved.

Now you can accept it as answer and thanks for sharing the solution, which might be beneficial to other community members reading this thread.

0 Votes 0 ·