question

BrianCuster-6063 avatar image
0 Votes"
BrianCuster-6063 asked PRADEEPCHEEKATLA-MSFT commented

Can't Start a Spark Cluster in my Synapse Environment

I have a synapse workspace with a managed network and am unable to start a spark cluster or do data flow debugging in my workspace studio. Does anyone know what might be the problem here? I've looked online and can't find any solutions to this problem.

azure-synapse-analytics
· 7
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @BrianCuster-6063,

Welcome to the Microsoft Q&A platform.

When you say "unable to start a spark cluster or do data flow debugging in my workspace studio" - could you please help us understand what happen when you click on Data Flow debug ? Are you experiencing any error message? If yes, please do share the screenshot of the error message.

131449-image.png

Meanwhile, you can checkout the Mapping data flow Debug Mode documentation.


0 Votes 0 ·
image.png (95.6 KiB)
BrianCuster-6063 avatar image BrianCuster-6063 PRADEEPCHEEKATLA-MSFT ·

Here is the error message:

LIVY_JOB_STATE_DEAD: Livy session has failed. Session state: Dead. Error code: LIVY_JOB_STATE_DEAD. [plugins.blc-workspace-synapse.mysparkpool.3 WorkspaceType:<Synapse> CCID:<082f19ee-3ce2-49b4-b6a7-02ab98d84ef9>] [Monitoring] Livy Endpoint=[https://hubservice1.westus2.azuresynapse.net:8001/api/v1.0/publish/50140127-f1b8-498e-a545-838a9e215dec]. Livy Id=[0] Job failed during run time with state=[dead]. Source: Dependency.

0 Votes 0 ·

Hi @BrianCuster-6063,

Thanks for sharing the error message.

  • Could you please add the stack trace (complete) detailed error message?

  • Have you tried to create data flow debug multiple times by using different browser?

  • Could you please share the integration runtime details which you are using to turn on data flow debug?

131740-image.png



0 Votes 0 ·
image.png (25.8 KiB)
Show more comments

0 Answers