Trigger Azure ML Pipeline from Azure Data Factory

Question

question

VinothKumarK-8698 asked Feb 17, '22 AhmedMahmoud-6610 commented Feb 28, '22

Trigger Azure ML Pipeline from Azure Data Factory

I have created and published a Azure ML pipeline. I want to trigger the ML pipeline from Azure Data Factory.

In ADF, i have chosen Machine learning execute pipeline and created the linked service to azure machine learning and able to choose the published pipeline endpoint. However while running, i am getting the below error. I couldn't find much information how to resolve the error.

"Convert Failed. The value type 'System.String', in key 'azureCloudType' is not expected type 'Microsoft.DataTransfer.Common.Models.AzureCloudType"

azure-data-factory azure-machine-learning

· 10

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

RazvanVacaru-8719 · Feb 17 at 10:25 AM

We have ADF pipelines running in production already for sometime, we didn't update anything in relation to either the ADF pipeline or the AzureML pipeline.
Nevertheless from this morning at ~8 CET our scheduled pipelines failed with the same error. We've tried re creating the "Machine Learning Execute Pipeline" activity or using PipelineId instead of PipelineEndpointId but without any success so far. It seems to be un underlying issue in Azure/ADF with how ADF invokes the ML Pipeline

Please prioritize this.

5 ·

RodrigoBarraza-7111 · Feb 17 at 12:35 PM

I have exactly the same problem.
I have a pipeline in synapse which executes a pipeline in ML, I don't know any modifications have been made in two months in either of the two environments and it has always been executed correctly, but today it fails throwing the same message:

"errorCode": "2105",
"message": "Convert Failed. The value type 'System.String', in key 'azureCloudType' is not expected type 'Microsoft.DataTransfer.Common.Models.AzureCloudType'"

3 ·

Tka32-3742 · Feb 17 at 10:22 AM

I also have the same issue.
We have a pipeline triggering ML pipelines, which used to run successfully until yesterday, started failing this morning with the same error.

2 ·

VeselinValchev-9446 · Feb 17 at 03:15 PM

Same problem here - anyone has any feedback from Microsoft re this?

0 ·

96861701 · Feb 17 at 03:20 PM

Same problem here

0 ·

Show more comments

Answer 1 · 2022-02-17T15:32:33.53Z

AnnuKumari-MSFT answered Feb 17, '22 AhmedMahmoud-6610 commented Feb 28, '22

Hi @VinothKumarK-8698 ,
Welcome to Microsoft Q&A platform and thankyou for posting your query.
As per the details you have shared in the query, it looks like a product bug. I have raised this issue with the internal Product team. Once I hear back from them, I will keep everyone posted on this. Thanks for your patience!

· 14

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

AnnuKumari-MSFT · Feb 17 at 04:55 PM

Hi @VinothKumarK-8698 ,
Product team has suggested to use SHIR instead of Azure IR as the current workaround which might help in mitigating the issue. Please do let us know if this solution works in your case. Although, the team is doing further investigation on the same.

0 ·

VinothKumarK-8698 AnnuKumari-MSFT · Feb 18 at 06:18 AM

Since my workload is still on the development stage, i would prefer to wait for the product team to fix the issue rather than setup the SHIR. Please do keep posted if the issue is resolved or any further developments on this.

0 ·

BirsanDEDumitruEugen-2501 AnnuKumari-MSFT · Feb 18 at 09:02 AM

The SHIR solution is not working. I get the same error.

0 ·

AnnuKumari-MSFT BirsanDEDumitruEugen-2501 · Feb 18 at 09:33 AM

Hi ,
Apologies for the inconvenience . Product team is working on this issue taking it as priority. Hot fix for this issue is in progress. It should be resolved in next 12 hours.

2 ·

Show more comments

JoseManuelEspinoza-2261 · Feb 24 at 10:54 PM

Hi,

I'm facing the same issue posted here calling an ML pipeline through the endpoint from Synapse.
Region AU East.

Thanks for the help in advance.

0 ·

AhmedMahmoud-6610 JoseManuelEspinoza-2261 · Feb 28 at 08:09 AM

@JoseManuelEspinoza-2261 Can you retry again and let us know if you still face the same issue.

Regards,
Ahmed

0 ·

Answer 2 · 2022-02-28T10:12:25.843Z

JoseManuelEspinoza-2261 answered Feb 28, '22 AhmedMahmoud-6610 commented Feb 28, '22

Hi Ahmed,

Working now :)
Many thanks.

Regards,
José.

· 1

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

AhmedMahmoud-6610 · Feb 28 at 10:15 AM

Hi Jose Manuel,

Thankyou for the update. Glad to know that the issue has been resolved!

Regards,
Ahmed

1 ·

question