LivyHttpRequestFailure: Something went wrong while processing your request. Please try again later. HTTP status code: 500. Trace ID: d2425fce-9179-49eb-829c-a7b2a0d963ed.

Devender 61 Reputation points
2022-11-22T06:13:46.587+00:00

Hi Everyone,
I am getting the Below Error Code while running my cells in Synapse Notebook:

> LivyHttpRequestFailure: Something went wrong while processing your request. Please try again later. HTTP status code: 500. Trace ID: d2425fce-9179-49eb-829c-a7b2a0d963ed.

I am getting this error in my 30th cell previous to 30th cell all cells are running fine, but cell 29 is taking 3 min to execute but when coming to 30th cell it is running for more than 30 minutes and than throwing the error.

Can anybody help me in fixing this issue. The Operations i am doing in that cell is JOINS using Pyspark

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,395 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
1,939 questions

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. BhargavaGunnam-MSFT 26,306 Reputation points Microsoft Employee
    2022-11-22T21:53:34.397+00:00

    Hello @Devender ,
    Welcome to the MS Q&A platform.
    HTTP 500 is a server error. It could be a synapse service issue.
    Do you see the error on any other notebooks? or is this failing at the 30th cell every time?
    Also, please check the available cores in your pool when running the notebook.

  2. Dom Sadie 0 Reputation points
    2024-04-23T08:26:35.41+00:00

    Hi I am getting the same issue and not sure how to solve it - LivyHttpRequestFailure: Something went wrong while processing your request. Please try again later. HTTP status code: 500. Trace ID: 3a315ef4-ca34-4c36-9862-b9ee109036da. So I am using a medium sever-less pool to stream an hour batch worth of data at a time (12500 files). It runs about 36hours (day and a half worth of data) then throws this error after 4hrs30minuntes of the pipeline running. Please can I get help, is it due to a cache issue, how can it run 36 batches then throw this error?