question

bjorneev-8087 avatar image
0 Votes"
bjorneev-8087 asked MartinKruse-0387 edited

Synapse workspace has a 0 core limit

I have a Synapse workspace running a bunch of pipelines every night. The pipelines are starting notebooks which use a spark pool. Since April 1st, all the pipelines failed with this error message:

"message": "Operation on target failed: Activity failed because an inner activity failed; Inner activity name: , Error: Exception: Failed to create Livy session for executing notebook. Error: Your Spark job requested 12 vcores. However, the workspace has a 0 core limit. Try reducing the numbers of vcores requested or increasing your vcore quota. HTTP status code: 400."

These pipelines have been running fine since I scheduled them 3 weeks ago. The workspace suddenly have a 0 core limit. Anyone experienced anything similar? I can't start dataflow debug clusters anymore either. They fail with message: "Failed to setup debug session. Debug session is already terminated."

Seems like there is a problem between Synapse and spark.

azure-synapse-analyticsdotnet-ml-big-data
5 |1600 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Visible to all users

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

MartinKruse-0387 avatar image
0 Votes"
MartinKruse-0387 answered MartinKruse-0387 edited

Hi. Possible duplicate: https://docs.microsoft.com/en-us/answers/questions/796922/your-spark-job-requested-8-vcores-however-the-work.html

Best regards
Martin

5 |1600 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Visible to all users

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.