question

SampatVarun-0080 avatar image
0 Votes"
SampatVarun-0080 asked YutongTie-MSFT commented

Integrate Azure ML with Azure Data Factory

Hello,

I have an Azure ML Batch Endpoint that successfully submits. I have made the input dataset a parameter. Now, ideally, I'd want to do the following:
- Execute the Endpoint from Azure Data Factory, while passing in a dynamic dataset that exists on Blob Storage.
- Receive the output dataset and continue consuming the returned dataset with other activities on Azure Data Factory.

So far, I've had no luck or access to clear documentation that can help me achieve this functionality:
- I followed this tutorial but it seems you can use this Activity on Azure Data Factory only to trigger an experiment on Azure ML and this doesn't mention passing in datasets from Azure Data Factory.
- I set up my batch endpoint using this tutorial and the only way this mentions the consumption of an endpoint is either manually through Azure ML or through the REST Api.

Overall, I'd like to know the feasibility of my desired solution. The worst case seems that I'd have to write a Python script that can create datasets on Azure ML, then trigger the batch endpoint pipeline (using the REST Api), and then re-upload the model output on a desired location in Blob Storage, and run this Python script on a Execute Batch Activity.

Additionally, I was wondering if it's possible to get sample code for the REST API code for consuming the batch endpoint like I did for a real-time endpoint.

Thanks,
Varun.

azure-data-factoryazure-machine-learning
· 4
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @SampatVarun-0080 and welcome to Microsoft Q&A.

To clarify, are you using ML Classic, or Azure ML?

0 Votes 0 ·

Hi Martin,

I'm using Azure ML.

0 Votes 0 ·

Hi @MartinJaffer-MSFT , are there any updates?

0 Votes 0 ·

@SampatVarun-0080 Sorry for the delay response. I will suggest you to raise a support ticket for this issue since it mixed two services and need your private information. Do you have a support plan? We can enable a One Time Free Ticket to you to assign a support engineer helping you for this issue. Please let me know if you need that and share the ticket number to us. Thanks a lot.


Regards,
Yutong

0 Votes 0 ·

0 Answers