Azure Queue storage bindings for Azure Functions

This article explains how to work with Azure Queue storage bindings in Azure Functions. Azure Functions supports trigger and output bindings for queues.

This is reference information for Azure Functions developers. If you're new to Azure Functions, start with the following resources:

Packages

The Queue storage bindings are provided in the Microsoft.Azure.WebJobs NuGet package. Source code for the package is in the azure-webjobs-sdk GitHub repository.

C# class library

For C# class library development, the package is automatically installed in the project.

Binding extensions in Functions 2.x

For local development in Azure Functions version 2.x, the package is automatically registered as a binding extension.

Version numbers

Package version numbers follow WebJobs SDK version numbering. WebJobs SDK 2.x corresponds to Functions 1.x, as shown in the following table.

Functions 1.x Functions 2.x
NuGet package version 2.x 3.x
Source code branch v2.x master, dev

Azure Storage SDK version

The Microsoft.Azure.WebJobs package includes the Azure Storage SDK (WindowsAzure.Storage) as a dependency. If you use the Storage SDK directly in your code, we recommend that you reference the same Storage SDK version that the Functions runtime does.

The Storage SDK version used by the Functions runtime is shown in the dependency list for the Microsoft.Azure.WebJobs package. For example, Microsoft.Azure.WebJobs 2.2 (Functions 1.x) depends on Storage SDK version 7.2.1, and Microsoft.Azure.WebJobs 3.0.0-beta5 (Functions 2.x) depends on Storage SDK version 8.6.

Trigger

Use the queue trigger to start a function when a new item is received on a queue. The queue message is provided as input to the function.

Trigger - example

See the language-specific example:

Trigger - C# example

The following example shows a C# function that polls the myqueue-items queue and writes a log each time a queue item is processed.

public static class QueueFunctions
{
    [FunctionName("QueueTrigger")]
    public static void QueueTrigger(
        [QueueTrigger("myqueue-items")] string myQueueItem, 
        TraceWriter log)
    {
        log.Info($"C# function processed: {myQueueItem}");
    }
}

Trigger - C# script example

The following example shows a queue trigger binding in a function.json file and C# script (.csx) code that uses the binding. The function polls the myqueue-items queue and writes a log each time a queue item is processed.

Here's the function.json file:

{
    "disabled": false,
    "bindings": [
        {
            "type": "queueTrigger",
            "direction": "in",
            "name": "myQueueItem",
            "queueName": "myqueue-items",
            "connection":"MyStorageConnectionAppSetting"
        }
    ]
}

The configuration section explains these properties.

Here's the C# script code:

#r "Microsoft.WindowsAzure.Storage"

using Microsoft.WindowsAzure.Storage.Queue;
using System;

public static void Run(CloudQueueMessage myQueueItem, 
    DateTimeOffset expirationTime, 
    DateTimeOffset insertionTime, 
    DateTimeOffset nextVisibleTime,
    string queueTrigger,
    string id,
    string popReceipt,
    int dequeueCount,
    TraceWriter log)
{
    log.Info($"C# Queue trigger function processed: {myQueueItem.AsString}\n" +
        $"queueTrigger={queueTrigger}\n" +
        $"expirationTime={expirationTime}\n" +
        $"insertionTime={insertionTime}\n" +
        $"nextVisibleTime={nextVisibleTime}\n" +
        $"id={id}\n" +
        $"popReceipt={popReceipt}\n" + 
        $"dequeueCount={dequeueCount}");
}

The usage section explains myQueueItem, which is named by the name property in function.json. The message metadata section explains all of the other variables shown.

Trigger - JavaScript example

The following example shows a queue trigger binding in a function.json file and a JavaScript function that uses the binding. The function polls the myqueue-items queue and writes a log each time a queue item is processed.

Here's the function.json file:

{
    "disabled": false,
    "bindings": [
        {
            "type": "queueTrigger",
            "direction": "in",
            "name": "myQueueItem",
            "queueName": "myqueue-items",
            "connection":"MyStorageConnectionAppSetting"
        }
    ]
}

The configuration section explains these properties.

Here's the JavaScript code:

module.exports = function (context) {
    context.log('Node.js queue trigger function processed work item', context.bindings.myQueueItem);
    context.log('queueTrigger =', context.bindingData.queueTrigger);
    context.log('expirationTime =', context.bindingData.expirationTime);
    context.log('insertionTime =', context.bindingData.insertionTime);
    context.log('nextVisibleTime =', context.bindingData.nextVisibleTime);
    context.log('id =', context.bindingData.id);
    context.log('popReceipt =', context.bindingData.popReceipt);
    context.log('dequeueCount =', context.bindingData.dequeueCount);
    context.done();
};

The usage section explains myQueueItem, which is named by the name property in function.json. The message metadata section explains all of the other variables shown.

Trigger - attributes

In C# class libraries, use the following attributes to configure a queue trigger:

  • QueueTriggerAttribute

    The attribute's constructor takes the name of the queue to monitor, as shown in the following example:

    [FunctionName("QueueTrigger")]
    public static void Run(
        [QueueTrigger("myqueue-items")] string myQueueItem, 
        TraceWriter log)
    {
        ...
    }
    

    You can set the Connection property to specify the storage account to use, as shown in the following example:

    [FunctionName("QueueTrigger")]
    public static void Run(
        [QueueTrigger("myqueue-items", Connection = "StorageConnectionAppSetting")] string myQueueItem, 
        TraceWriter log)
    {
        ....
    }
    

    For a complete example, see Trigger - C# example.

  • StorageAccountAttribute

    Provides another way to specify the storage account to use. The constructor takes the name of an app setting that contains a storage connection string. The attribute can be applied at the parameter, method, or class level. The following example shows class level and method level:

    [StorageAccount("ClassLevelStorageAppSetting")]
    public static class AzureFunctions
    {
        [FunctionName("QueueTrigger")]
        [StorageAccount("FunctionLevelStorageAppSetting")]
        public static void Run( //...
    {
        ...
    }
    

The storage account to use is determined in the following order:

  • The QueueTrigger attribute's Connection property.
  • The StorageAccount attribute applied to the same parameter as the QueueTrigger attribute.
  • The StorageAccount attribute applied to the function.
  • The StorageAccount attribute applied to the class.
  • The "AzureWebJobsStorage" app setting.

Trigger - configuration

The following table explains the binding configuration properties that you set in the function.json file and the QueueTrigger attribute.

function.json property Attribute property Description
type n/a Must be set to queueTrigger. This property is set automatically when you create the trigger in the Azure portal.
direction n/a In the function.json file only. Must be set to in. This property is set automatically when you create the trigger in the Azure portal.
name n/a The name of the variable that represents the queue in function code.
queueName QueueName The name of the queue to poll.
connection Connection The name of an app setting that contains the Storage connection string to use for this binding. If the app setting name begins with "AzureWebJobs", you can specify only the remainder of the name here. For example, if you set connection to "MyStorage", the Functions runtime looks for an app setting that is named "AzureWebJobsMyStorage." If you leave connection empty, the Functions runtime uses the default Storage connection string in the app setting that is named AzureWebJobsStorage.

When you're developing locally, app settings go into the local.settings.json file.

Trigger - usage

In C# and C# script, access the message data by using a method parameter such as string paramName. In C# script, paramName is the value specified in the name property of function.json. You can bind to any of the following types:

  • Object - The Functions runtime deserializes a JSON payload into an instance of an arbitrary class defined in your code.
  • string
  • byte[]
  • CloudQueueMessage

In JavaScript, use context.bindings.<name> to access the queue item payload. If the payload is JSON, it's deserialized into an object.

Trigger - message metadata

The queue trigger provides several metadata properties. These properties can be used as part of binding expressions in other bindings or as parameters in your code. These are properties of the CloudQueueMessage class.

Property Type Description
QueueTrigger string Queue payload (if a valid string). If the queue message payload as a string, QueueTrigger has the same value as the variable named by the name property in function.json.
DequeueCount int The number of times this message has been dequeued.
ExpirationTime DateTimeOffset The time that the message expires.
Id string Queue message ID.
InsertionTime DateTimeOffset The time that the message was added to the queue.
NextVisibleTime DateTimeOffset The time that the message will next be visible.
PopReceipt string The message's pop receipt.

Trigger - poison messages

When a queue trigger function fails, Azure Functions retries the function up to five times for a given queue message, including the first try. If all five attempts fail, the functions runtime adds a message to a queue named <originalqueuename>-poison. You can write a function to process messages from the poison queue by logging them or sending a notification that manual attention is needed.

To handle poison messages manually, check the dequeueCount of the queue message.

Trigger - polling algorithm

The queue trigger implements a random exponential back-off algorithm to reduce the effect of idle-queue polling on storage transaction costs. When a message is found, the runtime waits two seconds and then checks for another message; when no message is found, it waits about four seconds before trying again. After subsequent failed attempts to get a queue message, the wait time continues to increase until it reaches the maximum wait time, which defaults to one minute. The maximum wait time is configurable via the maxPollingInterval property in the host.json file.

Trigger - concurrency

When there are multiple queue messages waiting, the queue trigger retrieves a batch of messages and invokes function instances concurrently to process them. By default, the batch size is 16. When the number being processed gets down to 8, the runtime gets another batch and starts processing those messages. So the maximum number of concurrent messages being processed per function on one virtual machine (VM) is 24. This limit applies separately to each queue-triggered function on each VM. If your function app scales out to multiple VMs, each VM will wait for triggers and attempt to run functions. For example, if a function app scales out to 3 VMs, the default maximum number of concurrent instances of one queue-triggered function is 72.

The batch size and the threshold for getting a new batch are configurable in the host.json file. If you want to minimize parallel execution for queue-triggered functions in a function app, you can set the batch size to 1. This setting eliminates concurrency only so long as your function app runs on a single virtual machine (VM).

The queue trigger automatically prevents a function from processing a queue message multiple times; functions do not have to be written to be idempotent.

Trigger - host.json properties

The host.json file contains settings that control queue trigger behavior.

{
    "queues": {
      "maxPollingInterval": 2000,
      "visibilityTimeout" : "00:00:30",
      "batchSize": 16,
      "maxDequeueCount": 5,
      "newBatchThreshold": 8
    }
}
Property Default Description
maxPollingInterval 60000 The maximum interval in milliseconds between queue polls.
visibilityTimeout 0 The time interval between retries when processing of a message fails.
batchSize 16 The number of queue messages that the Functions runtime retrieves simultaneously and processes in parallel. When the number being processed gets down to the newBatchThreshold, the runtime gets another batch and starts processing those messages. So the maximum number of concurrent messages being processed per function is batchSize plus newBatchThreshold. This limit applies separately to each queue-triggered function.

If you want to avoid parallel execution for messages received on one queue, you can set batchSize to 1. However, this setting eliminates concurrency only so long as your function app runs on a single virtual machine (VM). If the function app scales out to multiple VMs, each VM could run one instance of each queue-triggered function.

The maximum batchSize is 32.
maxDequeueCount 5 The number of times to try processing a message before moving it to the poison queue.
newBatchThreshold batchSize/2 Whenever the number of messages being processed concurrently gets down to this number, the runtime retrieves another batch.

Output

Use the Azure Queue storage output binding to write messages to a queue.

Output - example

See the language-specific example:

Output - C# example

The following example shows a C# function that creates a queue message for each HTTP request received.

[StorageAccount("AzureWebJobsStorage")]
public static class QueueFunctions
{
    [FunctionName("QueueOutput")]
    [return: Queue("myqueue-items")]
    public static string QueueOutput([HttpTrigger] dynamic input,  TraceWriter log)
    {
        log.Info($"C# function processed: {input.Text}");
        return input.Text;
    }
}

Output - C# script example

The following example shows an HTTP trigger binding in a function.json file and C# script (.csx) code that uses the binding. The function creates a queue item with a CustomQueueMessage object payload for each HTTP request received.

Here's the function.json file:

{
  "bindings": [
    {
      "type": "httpTrigger",
      "direction": "in",
      "authLevel": "function",
      "name": "input"
    },
    {
      "type": "http",
      "direction": "out",
      "name": "return"
    },
    {
      "type": "queue",
      "direction": "out",
      "name": "$return",
      "queueName": "outqueue",
      "connection": "MyStorageConnectionAppSetting",
    }
  ]
}

The configuration section explains these properties.

Here's C# script code that creates a single queue message:

public class CustomQueueMessage
{
    public string PersonName { get; set; }
    public string Title { get; set; }
}

public static CustomQueueMessage Run(CustomQueueMessage input, TraceWriter log)
{
    return input;
}

You can send multiple messages at once by using an ICollector or IAsyncCollector parameter. Here's C# script code that sends multiple messages, one with the HTTP request data and one with hard-coded values:

public static void Run(
    CustomQueueMessage input, 
    ICollector<CustomQueueMessage> myQueueItems, 
    TraceWriter log)
{
    myQueueItems.Add(input);
    myQueueItems.Add(new CustomQueueMessage { PersonName = "You", Title = "None" });
}

Output - JavaScript example

The following example shows an HTTP trigger binding in a function.json file and a JavaScript function that uses the binding. The function creates a queue item for each HTTP request received.

Here's the function.json file:

{
  "bindings": [
    {
      "type": "httpTrigger",
      "direction": "in",
      "authLevel": "function",
      "name": "input"
    },
    {
      "type": "http",
      "direction": "out",
      "name": "return"
    },
    {
      "type": "queue",
      "direction": "out",
      "name": "$return",
      "queueName": "outqueue",
      "connection": "MyStorageConnectionAppSetting",
    }
  ]
}

The configuration section explains these properties.

Here's the JavaScript code:

module.exports = function (context, input) {
    context.done(null, input.body);
};

You can send multiple messages at once by defining a message array for the myQueueItem output binding. The following JavaScript code sends two queue messages with hard-coded values for each HTTP request received.

module.exports = function(context) {
    context.bindings.myQueueItem = ["message 1","message 2"];
    context.done();
};

Output - attributes

In C# class libraries, use the QueueAttribute.

The attribute applies to an out parameter or the return value of the function. The attribute's constructor takes the name of the queue, as shown in the following example:

[FunctionName("QueueOutput")]
[return: Queue("myqueue-items")]
public static string Run([HttpTrigger] dynamic input,  TraceWriter log)
{
    ...
}

You can set the Connection property to specify the storage account to use, as shown in the following example:

[FunctionName("QueueOutput")]
[return: Queue("myqueue-items", Connection = "StorageConnectionAppSetting")]
public static string Run([HttpTrigger] dynamic input,  TraceWriter log)
{
    ...
}

For a complete example, see Output - C# example.

You can use the StorageAccount attribute to specify the storage account at class, method, or parameter level. For more information, see Trigger - attributes.

Output - configuration

The following table explains the binding configuration properties that you set in the function.json file and the Queue attribute.

function.json property Attribute property Description
type n/a Must be set to queue. This property is set automatically when you create the trigger in the Azure portal.
direction n/a Must be set to out. This property is set automatically when you create the trigger in the Azure portal.
name n/a The name of the variable that represents the queue in function code. Set to $return to reference the function return value.
queueName QueueName The name of the queue.
connection Connection The name of an app setting that contains the Storage connection string to use for this binding. If the app setting name begins with "AzureWebJobs", you can specify only the remainder of the name here. For example, if you set connection to "MyStorage", the Functions runtime looks for an app setting that is named "AzureWebJobsMyStorage." If you leave connection empty, the Functions runtime uses the default Storage connection string in the app setting that is named AzureWebJobsStorage.

When you're developing locally, app settings go into the local.settings.json file.

Output - usage

In C# and C# script, write a single queue message by using a method parameter such as out T paramName. In C# script, paramName is the value specified in the name property of function.json. You can use the method return type instead of an out parameter, and T can be any of the following types:

In C# and C# script, write multiple queue messages by using one of the following types:

In JavaScript functions, use context.bindings.<name> to access the output queue message. You can use a string or a JSON-serializable object for the queue item payload.

Exceptions and return codes

Binding Reference
Queue Queue Error Codes
Blob, Table, Queue Storage Error Codes
Blob, Table, Queue Troubleshooting

Next steps