How to ensure only one Azure Function BlobTrigger runs at a time?

ddx1 171 Reputation points
2020-10-23T19:39:59.65+00:00

I have a use case to implement multiple BlobTriggers in Azure Functions (using the Linux Consumption Plan). For example in Azure Storage I would have 5 different clients with a directory structure like:

client1/file.txt
client2.file.txt
client3/file.txt
client4/file.txt
client5/file.txt

It's possible for both client1/file.txt and client2/file.txt to be dropped off at the same time in Azure Storage. To prevent race conditions and exceeding the 1.5 GB memory limit, I would like the BlobTrigger for client1/file.txt to wait for the BlobTrigger for client2/file.txt to finish or vice versa (the order doesn't matter here, just that both of them eventually execute).

Do I have to set up a queue process separately? Can I use the preview setting WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT to achieve this easily?

Would using durable functions be a better/more reliable solution?

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,349 questions
Azure Functions
Azure Functions
An Azure service that provides an event-driven serverless compute platform.
4,300 questions
Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,436 questions
0 comments No comments
{count} votes

Accepted answer
  1. Mike Urnun 9,676 Reputation points Microsoft Employee
    2020-10-23T21:13:08.057+00:00

    Hi @TomTakiyama-3838

    Blob trigger uses Queue storage under the covers, so in order to process your blobs one-by-one, you'll want to set batchSize setting to 1 (default is 16) in your host.json file:

    {  
        "version": "2.0",  
        "extensions": {  
            "queues": {  
                ...  
                "batchSize": 1  
                ...  
            }  
        }  
    }  
    
    2 people found this answer helpful.

0 additional answers

Sort by: Most helpful