question

JoshGriffith-2717 avatar image
1 Vote"
JoshGriffith-2717 asked hunga-1270 answered

How do I access an input parameter in Azure Machine Learning endpoints?

I've created an Azure ML Endpoint Pipeline with a single 'Execute Python Script'. From the script, I am looking for a way to access the input 'ParameterAssignments' that I POST to the endpoint to trigger the pipeline. I expected to see them somewhere in Run.get_context(), but I haven't had any luck. I simply need a way to POST arbitrary values that my Python scripts can access. Thank you!

azure-machine-learning
· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Are you using "Execute Python Script" module in Azure Machine Learning designer? Is it possible if you can share the screenshot of your pipeline and the Python script you put in the module?

0 Votes 0 ·

Thank you for replying. I've attached screenshots to illustrate what I'm trying to do. Basically, once I deploy the Pipeline endpoint, I want to access the pipeline parameter circled in red from within my python script. I feel like I've tried everything. I've read all the documentation there is and have spent 20+ hours trying to find a way to accomplish this. Any guidance or ideas you can provide would be well-received.

32233-screenshot2.png


32148-screenshot1.png


0 Votes 0 ·
screenshot2.png (33.8 KiB)
screenshot1.png (68.6 KiB)
LuZhangAI-1027 avatar image
1 Vote"
LuZhangAI-1027 answered LuZhang-4441 commented

I just confirmed with our engineer that you cannot set up a pipeline parameter and use it without tying it with any of the module parameter. So the workaround is - make the pipeline parameter as one of the inputs (i.e. dataset) to "Execute Python Script" module and set it as pipeline parameter. Then you can change it every time when calling the pipeline.

· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

With Lu Zhang's help, I was able to successfully achieve what I needed to do. The relevant documentation to import data from a regular pipeline endpoint to an 'Execute Python Script' can be found here: https://docs.microsoft.com/en-us/azure/machine-learning/how-to-retrain-designer

Basically, an 'Import Data' node can be attached as a parameter to the Execute Python Script. From there, you can define various parameters of the Import Data node as pipeline parameters. Pipeline parameters can be submitted to the pipeline endpoint invocation with a JSON serialized http request body with a 'ParameterAssignments' object which contains the parameters you wish to override.

1 Vote 1 ·

I have tried this in 10 different ways, but I may be doing something wrong. The documentation is extremely difficult to follow or understand. Endpoint pipelines appear to disregard the web service input node. It seems that web service inputs are only applicable to 'real-time endpoints', but I could be wrong. Are you able to confirm this with the engineering team or provide some sort of example? I really appreciate your time on this. I've been attempting to deliver a working solution for over a month now.

0 Votes 0 ·

Yes. Web service input/output are only applicable to real-time endpoints. Could you please reach out to me via: luzhan@microsoft.com? We can have a quick call to discuss the potential solutions to your problem, as well as the documentation improvement we can do.

0 Votes 0 ·
hunga-1270 avatar image
0 Votes"
hunga-1270 answered

Hi @LuZhangAI-1027, I am finding a way to connect to Postgres in my pipeline. I don't think 'Import Data' supports it, so I am thinking to write my own 'Execute Python Script' to load the data from Postgres. As I don't use 'Import Data', is there another way to access the pipeline parameters inside 'Execute Python Script'?

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.