question

AmroGhoneim-1570 avatar image
0 Votes"
AmroGhoneim-1570 asked RoderickBant74 commented

What is the best way to deploy my machine learning model using GPUs, specifically as a web based API?

I am trying to find the best way to run my machine learning models on GPUs for inference as an http request. Do Azure functions support GPUs? if not, what are other options I can look into?

note: I also want to use packaged models, not necessarily ones of my own creation (such as easyOCR for python)


azure-functionsazure-machine-learning
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

RoderickBant74 avatar image
1 Vote"
RoderickBant74 answered RoderickBant74 commented

Hi,

If you need GPU support on ML inference the only supported option is the Azure Kubernetes Service as stated in this documentation

For guidance on deploying an ML model to AKS, please refer to this documenation on deploying to AKS


· 4
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thanks!

The link you provided contains a note that mentions that azure machine learning endpoints "provide an improved, simpler deployment experience. Endpoints support both real-time and batch inference scenarios"

Should I then explore this option instead of AKS?

I also found this Article that says that it supports GPUs. Is that the case?


0 Votes 0 ·

The managed endpoint are currently in Preview and documetation indeed indicates that GPU's will be supported. However the documentation states that this option is not recommended for production workloads.

So IMHO the managed endpoint option is excellent for research, development and testing use-cases. But for production workloads I would recommend using AKS.


1 Vote 1 ·

Right I understand!

One more thing, I also found this repository that describes a way of deploying GPU enabled Azure functions using AKS




Is this a viable option?

0 Votes 0 ·
Show more comments