(UserError) Error when parsing request; unable to deserialize request body

Question

question

SadeghHosseinpoor-9324 asked · Jun 08 at 11:39 AM

(UserError) Error when parsing request; unable to deserialize request body

Hi, getting this error when i run an azureml experiment with custom_docker_image (basegpu image of mcr) - can anybody help me understand this? Have tested this in local compute and it works, not sure why this does not work on a training cluster vm?

    azureml._restclient.exceptions.ServiceException: ServiceException:
         Code: 400
         Message: (UserError) Error when parsing request; unable to deserialize request body
         Details:
        
         Headers: {
             "Date": "Mon, 08 Jun 2020 11:03:52 GMT",
             "Content-Type": "application/json; charset=utf-8",
             "Transfer-Encoding": "chunked",
             "Connection": "keep-alive",
             "Request-Context": "appId=cid-v1:6a27ce65-5555-41a3-85f7-b7a1ce31fd6b",
             "x-ms-response-type": "error",
             "Strict-Transport-Security": "max-age=15724800; includeSubDomains; preload"
         }
         InnerException: {
         "additional_properties": {},
         "error": {
             "additional_properties": {},
             "code": "UserError",
             "message": "Error when parsing request; unable to deserialize request body",
             "details_uri": null,
             "target": null,
             "details": [],
             "inner_error": null,
             "debug_info": null,
             "message_format": null,
             "message_parameters": null,
             "reference_code": null
         },
         "correlation": {
             "operation": "e96d6285280f5849a4a5e3f172d65d36",
             "request": "1beee8ecb7180147"
         },
         "environment": "westeurope",
         "location": "westeurope",
         "time": {}
     }

azure-machine-learning

10 |1000 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Answer 1 · 2020-06-11T14:05:05.637Z

SadeghHosseinpoor-9324 answered · Jun 11 at 02:05 PM

My bad, after giving it some days and looking at the code, I noticed i had forgotten to add the parameters for the estimator configuration. Here is the estimator configuration that works for me:

 estimator = Estimator(source_directory=experiment_folder,
                       compute_target=compute_target,
                       script_params=script_params,
                       entry_script='rps_efn_b0.py',
                       node_count=1,        
                       conda_packages=['ipykernel'],
                       pip_packages = ['azureml-sdk',
                                       'pyarrow',
                                       'pyspark',
                                       'azureml-mlflow',
                                       'joblib',
                                       'matplotlib',
                                       'Pillow',
                                       'tensorflow==2.2',
                                       'tensorflow-datasets',
                                       'tensorflow-hub',
                                       'azureml-defaults',
                                       'azureml-dataprep[fuse,pandas]'],
                       custom_docker_image='mcr.microsoft.com/azureml/base-gpu:openmpi3.1.2-cuda10.1-cudnn7-ubuntu18.04')

· Share

10 |1000 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Answer 2 · 2020-06-09T14:44:45.27Z

romungi-MSFT answered · Jun 09 at 02:44 PM

@sadeghHosseinpoor-9324 Apologies for not responding on the previous thread as we are having platform issues on providing response to some threads.

In this case having access to the backend details of your experiment run could help to suggest what changes are required in your current experiment setup. This can be addressed by our support team if you can raise a support ticket against this resource from the azure portal.

Meanwhile, I have tried to lookup similar error messages faced by other users and based on your scenario are you installing any package in your environment with package name and version which is failing to read the request json? I found one user who corrected this error during environment setup by correcting the request for spark package installation in their environment.

Before:

 environment.spark.packages=[SparkPackage(group='com.microsoft.azure', artifact='azure:azure-sqldb-spark', version='1.0.2')]

After:

 environment.spark.packages=[{'group':'com.microsoft.azure', 'artifact': 'azure-sqldb-spark', 'version':'1.0.2'}]

Is there any similar requests in your case?

· 1 · Share

10 |1000 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

SadeghHosseinpoor-9324 · Jun 11 at 02:06 PM

Thanks for the reply, figured it out, see answer*

0 · ·

question