This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(96, 80%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EEM%3C/text%3E%3C/svg%3E)
Dear all,
I am using dbx to deploy and launch jobs on ephemeral clusters on databricks.
I have initialized the the cicd-sample-project and connected to a fresh empty Databricks Free trial environment and everything works.
But when I try to do the same on my company's project, the launch commands fails. In the UI I get this error 
and in the log I get the following
WARN MetastoreMonitor: Failed to connect to the metastore InternalMysqlMetastore(DbMetastoreConfig{host=consolidated-westeuropec2-prod-metastore-3.mysql.database.azure.com, port=3306, dbName=organization5367007285973203, user=[REDACTED]}). (timeSinceLastSuccess=0) org.skife.jdbi.v2.exceptions.UnableToObtainConnectionException: java.sql.SQLTransientConnectionException: metastore-monitor - Connection is not available, request timed out after 15090ms. at org.skife.jdbi.v2.DBI.open(DBI.java:230)
My company's databricks environment connects to Azure through a vpn so I think that there lies the problem.
This is an overview of the ports, protocol and destination opened in our firewall:
PROTOCOL: TCP; PORT: 5671,5672,9350-9354,9093; DESTINATION: *
PROTOCOL: UDP; PORT: 123; DESTINATION: *
PROTOCOL: ICMP; PORT: *; DESTINATION: *
PROTOCOL: MSSQL; PORT 3306; DESTINATION:
westeurope-prod-metastore.mysql.database.azure.com,consolidated-westeurope-prod-metastore-addl-1.mysql.database.azure.com,consolidated-westeurope-prod-metastore-addl-2.mysql.database.azure.com,consolidated-westeurope-prod-metastore-addl-3.mysql.database.azure.com,consolidated-westeuropec2-prod-metastore-0.mysql.database.azure.com,consolidated-westeuropec2-prod-metastore-1.mysql.database.azure.com,consolidated-westeuropec2-prod-metastore-2.mysql.database.azure.com,consolidated-westeuropec2-prod-metastore-3.mysql.database.azure.com
Here are the screenshots of my current firewall configuration:
I am quite stuck and I don't know what to try further, could you help me out?
I have attached a full log in case someone wants to have a look
269999-fail.txt
Any help or suggestion is welcomed, thanks and have a great day.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 9%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKM%3C/text%3E%3C/svg%3E)
Hello @Enrico Mosca ,
Thanks for the question and using MS Q&A platform.
Yes, seems like a networking/firewall issue. This involves multiple things that need to be investigated along with networking team, hence for better assistance and thorough investigation, we recommend filing a support ticket or this issue. Please do share the SR# once the ticket is filed so that we can track it internally and in case if you don't have a support plan, please let me know here.
Thank you.
Hi @KranthiPakala-MSFT ,
thanks for your reply.
I don't have a support plan, but the organization I am working for (ATOS) is a Microsoft Partner. Is there a way I could speak with someone helping me out with the issue at hand?
Thanks and have a great day.
Hello @Enrico Mosca ,
Thanks for your response. If your client is having a support plan, then administrator of Azure subscription should be able to help you file a support ticket for this issue. In case if they don't have a support plan, please let me know here, so that I can check on alternate options.
thanks for your reply.
Unfortunately my client also does not have a support plan.
Do you think that changing the rules priority of the application rules (i.e. the one responsible for opening the connection towards consolidated-westeuropec2-prod-metastore-3.mysql.database.azure.com on port 3306) could lead to any change?
Thanks and have a great day.
Hello @Enrico Mosca ,
Thanks for your response. We are not sure about the root cause as this is a pretty new feature/integration. As dbx by Databricks Labs, which is provided as-is and is not supported by Databricks through customer technical support channels. It is recommended that Questions and feature requests can be communicated through the Issues page of the databrickslabs/dbx repo on GitHub.
I would recommend reaching out in Issues page as mentioned above so that one the developer team can better assist on this issue. Please do share the issue link once it is posted in GitHub repository so that other readers in this community can follow the thread for solution.
Hope this info helps.
Thanks
Thanks for your reply!
A colleague find out that some routes were missing and adding them fixed the problem.
For everyone else that may encounter the same problem, please have a look at Configure user-defined routes with Azure service tags
Hello @Enrico Mosca ,
Glad to know that your issue has been resolved and appreciate much for sharing your findings and helping the community here with the root cause and resolution details.
Thank you
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(246.4, 53%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
@Enrico Mosca The route in the UDR will work, but the Firewall should also work. I think you have made an mistake. Note that the UDR grant access to all Azure hosted SQL services (SQL ServiceTag).
You have created a Application Rule (L7) for MSSQL:3306 (not MySQL used by Databricks!) MSSQL does NOT use port 3306.
Instead configure the rule in Network Rules (L4) for Destination consolidated-westeuropec2-prod-metastore-0.mysql.database.azure.com (all domains here: Databricks) TCP port 3306.
No need for "wide open" UDR's to next hop internet. Please verify this.