Azure Queue storage trigger for Azure Functions

The queue storage trigger runs a function as messages are added to Azure Queue storage.

Encoding

Functions expect a base64 encoded string. Any adjustments to the encoding type (in order to prepare data as a base64 encoded string) need to be implemented in the calling service.

Example

Use the queue trigger to start a function when a new item is received on a queue. The queue message is provided as input to the function.

The following example shows a C# function that polls the myqueue-items queue and writes a log each time a queue item is processed.

public static class QueueFunctions
{
    [FunctionName("QueueTrigger")]
    public static void QueueTrigger(
        [QueueTrigger("myqueue-items")] string myQueueItem, 
        ILogger log)
    {
        log.LogInformation($"C# function processed: {myQueueItem}");
    }
}

Attributes and annotations

In C# class libraries, use the following attributes to configure a queue trigger:

  • QueueTriggerAttribute

    The attribute's constructor takes the name of the queue to monitor, as shown in the following example:

    [FunctionName("QueueTrigger")]
    public static void Run(
        [QueueTrigger("myqueue-items")] string myQueueItem, 
        ILogger log)
    {
        ...
    }
    

    You can set the Connection property to specify the app setting that contains the storage account connection string to use, as shown in the following example:

    [FunctionName("QueueTrigger")]
    public static void Run(
        [QueueTrigger("myqueue-items", Connection = "StorageConnectionAppSetting")] string myQueueItem, 
        ILogger log)
    {
        ....
    }
    

    For a complete example, see example.

  • StorageAccountAttribute

    Provides another way to specify the storage account to use. The constructor takes the name of an app setting that contains a storage connection string. The attribute can be applied at the parameter, method, or class level. The following example shows class level and method level:

    [StorageAccount("ClassLevelStorageAppSetting")]
    public static class AzureFunctions
    {
        [FunctionName("QueueTrigger")]
        [StorageAccount("FunctionLevelStorageAppSetting")]
        public static void Run( //...
    {
        ...
    }
    

The storage account to use is determined in the following order:

  • The QueueTrigger attribute's Connection property.
  • The StorageAccount attribute applied to the same parameter as the QueueTrigger attribute.
  • The StorageAccount attribute applied to the function.
  • The StorageAccount attribute applied to the class.
  • The "AzureWebJobsStorage" app setting.

Configuration

The following table explains the binding configuration properties that you set in the function.json file and the QueueTrigger attribute.

function.json property Attribute property Description
type n/a Must be set to queueTrigger. This property is set automatically when you create the trigger in the Azure portal.
direction n/a In the function.json file only. Must be set to in. This property is set automatically when you create the trigger in the Azure portal.
name n/a The name of the variable that contains the queue item payload in the function code.
queueName QueueName The name of the queue to poll.
connection Connection The name of an app setting or setting collection that specifies how to connect to Azure Queues. See Connections.

When you're developing locally, app settings go into the local.settings.json file.

Connections

The connection property is a reference to environment configuration which specifies how the app should connect to Azure Queues. It may specify:

If the configured value is both an exact match for a single setting and a prefix match for other settings, the exact match is used.

Connection string

To obtain a connection string, follow the steps shown at Manage storage account access keys.

This connection string should be stored in an application setting with a name matching the value specified by the connection property of the binding configuration.

If the app setting name begins with "AzureWebJobs", you can specify only the remainder of the name here. For example, if you set connection to "MyStorage", the Functions runtime looks for an app setting that is named "AzureWebJobsMyStorage." If you leave connection empty, the Functions runtime uses the default Storage connection string in the app setting that is named AzureWebJobsStorage.

Identity-based connections

If you are using version 5.x or higher of the extension, instead of using a connection string with a secret, you can have the app use an Azure Active Directory identity. To do this, you would define settings under a common prefix which maps to the connection property in the trigger and binding configuration.

In this mode, the extension requires the following properties:

Property Environment variable template Description Example value
Queue Service URI <CONNECTION_NAME_PREFIX>__queueServiceUri1 The data plane URI of the queue service to which you are connecting, using the HTTPS scheme. https://<storage_account_name>.queue.core.windows.net

1 <CONNECTION_NAME_PREFIX>__serviceUri can be used as an alias. If both forms are provided, the queueServiceUri form will be used. The serviceUri form cannot be used when the overall connection configuration is to be used across blobs, queues, and/or tables.

Additional properties may be set to customize the connection. See Common properties for identity-based connections.

When hosted in the Azure Functions service, identity-based connections use a managed identity. The system-assigned identity is used by default, although a user-assigned identity can be specified with the credential and clientID properties. When run in other contexts, such as local development, your developer identity is used instead, although this can be customized. See Local development with identity-based connections.

Grant permission to the identity

Whatever identity is being used must have permissions to perform the intended actions. You will need to assign a role in Azure RBAC, using either built-in or custom roles which provide those permissions.

Important

Some permissions might be exposed by the target service that are not necessary for all contexts. Where possible, adhere to the principle of least privilege, granting the identity only required privileges. For example, if the app only needs to be able to read from a data source, use a role that only has permission to read. It would be inappropriate to assign a role that also allows writing to that service, as this would be excessive permission for a read operation. Similarly, you would want to ensure the role assignment is scoped only over the resources that need to be read.

You will need to create a role assignment that provides access to your queue at runtime. Management roles like Owner are not sufficient. The following table shows built-in roles that are recommended when using the Queue Storage extension in normal operation. Your application may require additional permissions based on the code you write.

Binding type Example built-in roles
Trigger Storage Queue Data Reader, Storage Queue Data Message Processor
Output binding Storage Queue Data Contributor, Storage Queue Data Message Sender

Usage

Default

Access the message data by using a method parameter such as string paramName. You can bind to any of the following types:

  • Object - The Functions runtime deserializes a JSON payload into an instance of an arbitrary class defined in your code.
  • string
  • byte[]
  • CloudQueueMessage

If you try to bind to CloudQueueMessage and get an error message, make sure that you have a reference to the correct Storage SDK version.

Additional types

Apps using the 5.0.0 or higher version of the Storage extension may also use types from the Azure SDK for .NET. This version drops support for the legacy CloudQueueMessage type in favor of the following types:

For examples using these types, see the GitHub repository for the extension.

Message metadata

The queue trigger provides several metadata properties. These properties can be used as part of binding expressions in other bindings or as parameters in your code. The properties are members of the CloudQueueMessage class.

Property Type Description
QueueTrigger string Queue payload (if a valid string). If the queue message payload is a string, QueueTrigger has the same value as the variable named by the name property in function.json.
DequeueCount int The number of times this message has been dequeued.
ExpirationTime DateTimeOffset The time that the message expires.
Id string Queue message ID.
InsertionTime DateTimeOffset The time that the message was added to the queue.
NextVisibleTime DateTimeOffset The time that the message will next be visible.
PopReceipt string The message's pop receipt.

Poison messages

When a queue trigger function fails, Azure Functions retries the function up to five times for a given queue message, including the first try. If all five attempts fail, the functions runtime adds a message to a queue named <originalqueuename>-poison. You can write a function to process messages from the poison queue by logging them or sending a notification that manual attention is needed.

To handle poison messages manually, check the dequeueCount of the queue message.

Peek lock

The peek-lock pattern happens automatically for queue triggers. As messages are dequeued, they are marked as invisible and associated with a timeout managed by the Storage service.

When the function starts, it starts processing a message under the following conditions.

  • If the function is successful, then the function execution completes and the message is deleted.
  • If the function fails, then the message visibility is reset. After being reset, the message is reprocessed the next time the function requests a new message.
  • If the function never completes due to a crash, the message visibility expires and the message re-appears in the queue.

All of the visibility mechanics are handled by the Storage service, not the Functions runtime.

Polling algorithm

The queue trigger implements a random exponential back-off algorithm to reduce the effect of idle-queue polling on storage transaction costs.

The algorithm uses the following logic:

  • When a message is found, the runtime waits 100 milliseconds and then checks for another message
  • When no message is found, it waits about 200 milliseconds before trying again.
  • After subsequent failed attempts to get a queue message, the wait time continues to increase until it reaches the maximum wait time, which defaults to one minute.
  • The maximum wait time is configurable via the maxPollingInterval property in the host.json file.

For local development the maximum polling interval defaults to two seconds.

Note

In regards to billing when hosting function apps in the Consumption plan, you are not charged for time spent polling by the runtime.

Concurrency

When there are multiple queue messages waiting, the queue trigger retrieves a batch of messages and invokes function instances concurrently to process them. By default, the batch size is 16. When the number being processed gets down to 8, the runtime gets another batch and starts processing those messages. So the maximum number of concurrent messages being processed per function on one virtual machine (VM) is 24. This limit applies separately to each queue-triggered function on each VM. If your function app scales out to multiple VMs, each VM will wait for triggers and attempt to run functions. For example, if a function app scales out to 3 VMs, the default maximum number of concurrent instances of one queue-triggered function is 72.

The batch size and the threshold for getting a new batch are configurable in the host.json file. If you want to minimize parallel execution for queue-triggered functions in a function app, you can set the batch size to 1. This setting eliminates concurrency only so long as your function app runs on a single virtual machine (VM).

The queue trigger automatically prevents a function from processing a queue message multiple times simultaneously.

host.json properties

The host.json file contains settings that control queue trigger behavior. See the host.json settings section for details regarding available settings.

Next steps