question

2JK avatar image
0 Votes"
2JK asked PramodValavala-MSFT answered

Loading a custom model once in Azure Functions

Hi.

I have an Azure Function with a Cosmos trigger that listens to a container, processes the content (a text field), and then outputs a result. To process the text, I have a custom model trained in Tensorflow (.h5). Problem is, every time there's a change in the container or after every batch is captured, the model, which sits in a models folder outside the function folder, has to be loaded. This obviously isn't ideal.

Is there a way to have it load once? Maybe it would be best to host the model somewhere then call it, but is it possible to do it in the Azure Function or using any other Azure service?

Thanks.

azure-functionsazure-cosmos-db
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

PramodValavala-MSFT avatar image
1 Vote"
PramodValavala-MSFT answered

@2JK One straightforward way would be to load the model once per instance (like how the manage connections doc suggests for sharing expensive connection objects).

Since this would be per instance, as your function app scales out, each instance would load the model when starts up. Similarly, if the function app scales down to zero, then when it starts back up, the model would have to be loaded again.

Another option is to use the Premium Plan which allows at-least one instance to be always available for requests. This way there is no cold start and as your function app scales out, the model would be loaded for each instance.


5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.