Azure SQL Database
An Azure relational database service.
5,148 questions
any way to reach more professionals through this post?
load data into SQL Server via Pyspark.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(227.2, 8%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EWV%3C/text%3E%3C/svg%3E)
Welder Vicente Moreira Martins
1
Reputation point
Hi everyone, I don't know what else to do. I tested it with several cluster versions. Need help.
thanks.
Error return in the description below:
org.apache.spark.SparkException: Job aborted due to stage failure: Task 31 in stage 1.0 failed 4 times, most recent failure: Lost task 31.3 in stage 1.0 (TID 109) (10.139.64.9 executor 8): com.microsoft.sqlserver.jdbc.SQLServerException: Database 'dw' on server 'dw' is not currently available. Please retry the connection later. If the problem persists, contact customer support, and provide them the session tracing ID of '27977670-F9B7-42AF-8F1F-3E3EB972FA65'
Azure SQL Database
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
1,938 questions
.NET Machine learning
.NET Machine learning
.NET: Microsoft Technologies based on the .NET software framework.Machine learning: A type of artificial intelligence focused on enabling computers to use observed data to evolve new behaviors that have not been explicitly programmed.
150 questions
{count} votes
-
ShaikMaheer-MSFT 37,896 Reputation points Microsoft Employee2022-02-14T15:28:09.077+00:00 Hi @Welder Vicente Moreira Martins ,
Thank you for posting query in Microsoft Q&A Platform.
From error message it seems Database is not available. Kindly check database status and make sure its available and running.
In below thread some details shared around SQL issues with Azure Databricks. Kindly check it once if that helps.
https://learn.microsoft.com/en-us/answers/questions/251843/unable-to-connect-to-azure-sql-database-through-da.htmlIf above not helps then couple please help sharing more details on your setup. Meaning, Azure SQL details such as serverless or provisioned and Databricks code which throwing this error. So that we can try to repro scenario and help better. Thank you.
-
Welder Vicente Moreira Martins 1 Reputation point
2022-02-14T17:15:02.777+00:00 Hi,
Inserting a small mass of data does not cause the database connection to be lost.
The first data load, for example, was a thousand records and there were no failures.
But with a load of 48 million records the connection to the database was lost.
Data loading was done at several different times and has the same failure.
Att,
Martins welder
-
Welder Vicente Moreira Martins 1 Reputation point
2022-02-15T17:03:09.65+00:00 Hi ShaikMaheer-MSFT, I processed 48 million records in an hour in my on-premises environment. On azure I'm using a cluster with 4 workers and in addition to spending more than 1 hour processing, it loses the connection to the database. Do you have any solution suggestions?
-
ShaikMaheer-MSFT 37,896 Reputation points Microsoft Employee
2022-02-16T12:54:01.99+00:00 Hi @Welder Vicente Moreira Martins ,
It feels this issue needs more debug and go to logs deeper and understand the cause. I encourage you to have to support ticket for this issue to deeper investigation and resolution. Below is the link which explains how to create support ticket. Thank you.
https://learn.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request
Sign in to comment