Azure Event Hubs bindings for Azure Functions

This article explains how to work with Azure Event Hubs bindings for Azure Functions. Azure Functions supports trigger and output bindings for Event Hubs.

This is reference information for Azure Functions developers. If you're new to Azure Functions, start with the following resources:

Trigger

Use the Event Hubs trigger to respond to an event sent to an event hub event stream. You must have read access to the event hub to set up the trigger.

When an Event Hubs trigger function is triggered, the message that triggers it is passed into the function as a string.

Trigger - scaling

Each instance of an Event Hub-Triggered Function is backed by only 1 EventProcessorHost (EPH) instance. Event Hubs ensures that only 1 EPH can get a lease on a given partition.

For example, suppose we begin with the following setup and assumptions for an Event Hub:

  1. 10 partitions.
  2. 1000 events distributed evenly across all partitions => 100 messages in each partition.

When your function is first enabled, there is only 1 instance of the funciton. Let's call this function instance Function_0. Function_0 will have 1 EPH that manages to get a lease on all 10 partitions. It will start reading events from partitions 0-9. From this point forward, one of the following will happen:

  • Only 1 function instance is needed - Function_0 is able to process all 1000 before the Azure Functions' scaling logic kicks in. Hence, all 1000 messages are processed by Function_0.

  • Add 1 more function instance - Azure Functions' scaling logic determines that Function_0 has more messages than it can process, so a new instance, Function_1, is created. Event Hubs detects that a new EPH instance is trying read messages. Event Hubs will start load balancing the partitions across the EPH instances, e.g., partitions 0-4 are assigned to Function_0 and partitions 5-9 are assigned to Function_1.

  • Add N more function instances - Azure Functions' scaling logic determines that both Function_0 and Function_1 have more messages than they can process. It will scale again for Function_2...N, where N is greater than the Event Hub paritions. Event Hubs will load balance the partitions across Function_0...9 instances.

Unique to Azure Functions' current scaling logic is the fact that N is greater than the number of partitions. This is done to ensure that there are always instances of EPH readily available to quickly get a lock on the partition(s) as they become available from other instances. Users are only charged for the resources used when the function instance executes, and are not charged for this over-provisioning.

If all function executions succeed without errors, checkpoints are added to the associated storage account. When check-pointing succeeds, all 1000 messages should never be retrieved again.

Trigger - example

See the language-specific example:

Trigger - C# example

The following example shows precompiled C# code that logs the message body of the event hub trigger.

[FunctionName("EventHubTriggerCSharp")]
public static void Run([EventHubTrigger("samples-workitems", Connection = "EventHubConnection")] string myEventHubMessage, TraceWriter log)
{
    log.Info($"C# Event Hub trigger function processed a message: {myEventHubMessage}");
}

To get access to the event metadata, bind to an EventData object (requires a using statement for Microsoft.ServiceBus.Messaging).

[FunctionName("EventHubTriggerCSharp")]
public static void Run([EventHubTrigger("samples-workitems", Connection = "EventHubConnection")] EventData myEventHubMessage, TraceWriter log)
{
    log.Info($"{Encoding.UTF8.GetString(myEventHubMessage.GetBytes())}");
}

To receive events in a batch, make string or EventData an array:

[FunctionName("EventHubTriggerCSharp")]
public static void Run([EventHubTrigger("samples-workitems", Connection = "EventHubConnection")] string[] eventHubMessages, TraceWriter log)
{
    foreach (var message in eventHubMessages)
    {
        log.Info($"C# Event Hub trigger function processed a message: {message}");
    }
}

Trigger - C# script example

The following example shows an event hub trigger binding in a function.json file and a C# script function that uses the binding. The function logs the message body of the event hub trigger.

Here's the binding data in the function.json file:

{
  "type": "eventHubTrigger",
  "name": "myEventHubMessage",
  "direction": "in",
  "path": "MyEventHub",
  "connection": "myEventHubReadConnectionString"
}

Here's the C# script code:

using System;

public static void Run(string myEventHubMessage, TraceWriter log)
{
    log.Info($"C# Event Hub trigger function processed a message: {myEventHubMessage}");
}

To get access to the event metadata, bind to an EventData object (requires a using statement for Microsoft.ServiceBus.Messaging).

#r "Microsoft.ServiceBus"
using System.Text;
using Microsoft.ServiceBus.Messaging;

public static void Run(EventData myEventHubMessage, TraceWriter log)
{
    log.Info($"{Encoding.UTF8.GetString(myEventHubMessage.GetBytes())}");
}

To receive events in a batch, make string or EventData an array:

public static void Run(string[] eventHubMessages, TraceWriter log)
{
    foreach (var message in eventHubMessages)
    {
        log.Info($"C# Event Hub trigger function processed a message: {message}");
    }
}

Trigger - F# example

The following example shows an event hub trigger binding in a function.json file and an F# function that uses the binding. The function logs the message body of the event hub trigger.

Here's the binding data in the function.json file:

{
  "type": "eventHubTrigger",
  "name": "myEventHubMessage",
  "direction": "in",
  "path": "MyEventHub",
  "connection": "myEventHubReadConnectionString"
}

Here's the F# code:

let Run(myEventHubMessage: string, log: TraceWriter) =
    log.Info(sprintf "F# eventhub trigger function processed work item: %s" myEventHubMessage)

Trigger - JavaScript example

The following example shows an event hub trigger binding in a function.json file and a JavaScript function that uses the binding. The function logs the message body of the event hub trigger.

Here's the binding data in the function.json file:

{
  "type": "eventHubTrigger",
  "name": "myEventHubMessage",
  "direction": "in",
  "path": "MyEventHub",
  "connection": "myEventHubReadConnectionString"
}

Here's the JavaScript code:

module.exports = function (context, myEventHubMessage) {
    context.log('Node.js eventhub trigger function processed work item', myEventHubMessage);    
    context.done();
};

Trigger - attributes

For precompiled C# functions, use the EventHubTriggerAttribute attribute, which is defined in NuGet package Microsoft.Azure.WebJobs.ServiceBus.

The attribute's constructor takes the name of the event hub, the name of the consumer group, and the name of an app setting that contains the connection string. For more information about these settings, see the trigger configuration section. Here's an EventHubTriggerAttribute attribute example:

[FunctionName("EventHubTriggerCSharp")]
public static void Run([EventHubTrigger("samples-workitems", Connection = "EventHubConnection")] string myEventHubMessage, TraceWriter log)
{
    ...
}

For a complete example, see Trigger - precompiled C# example.

Trigger - configuration

The following table explains the binding configuration properties that you set in the function.json file and the EventHubTrigger attribute.

function.json property Attribute property Description
type n/a Must be set to eventHubTrigger. This property is set automatically when you create the trigger in the Azure portal.
direction n/a Must be set to in. This property is set automatically when you create the trigger in the Azure portal.
name n/a The name of the variable that represents the event item in function code.
path EventHubName The name of the event hub.
consumerGroup ConsumerGroup An optional property that sets the consumer group used to subscribe to events in the hub. If omitted, the $Default consumer group is used.
connection Connection The name of an app setting that contains the connection string to the event hub's namespace. Copy this connection string by clicking the Connection Information button for the namespace, not the event hub itself. This connection string must have at least read permissions to activate the trigger.

When you're developing locally, app settings go into the local.settings.json file.

Trigger - host.json properties

The host.json file contains settings that control Event Hubs trigger behavior.

{
    "eventHub": {
      "maxBatchSize": 64,
      "prefetchCount": 256,
      "batchCheckpointFrequency": 1
    }
}
Property Default Description
maxBatchSize 64 The maximum event count received per receive loop.
prefetchCount n/a The default PrefetchCount that will be used by the underlying EventProcessorHost.
batchCheckpointFrequency 1 The number of event batches to process before creating an EventHub cursor checkpoint.

Output

Use the Event Hubs output binding to write events to an event stream. You must have send permission to an event hub to write events to it.

Output - example

See the language-specific example:

Output - C# example

The following example shows a precompiled C# function that writes a message to an event hub, using the method return value as the output:

[FunctionName("EventHubOutput")]
[return: EventHub("outputEventHubMessage", Connection = "EventHubConnection")]
public static string Run([TimerTrigger("0 */5 * * * *")] TimerInfo myTimer, TraceWriter log)
{
    log.Info($"C# Timer trigger function executed at: {DateTime.Now}");
    return $"{DateTime.Now}";
}

Output - C# script example

The following example shows an event hub trigger binding in a function.json file and a C# script function that uses the binding. The function writes a message to an event hub.

Here's the binding data in the function.json file:

{
    "type": "eventHub",
    "name": "outputEventHubMessage",
    "path": "myeventhub",
    "connection": "MyEventHubSend",
    "direction": "out"
}

Here's C# script code that creates one message:

using System;

public static void Run(TimerInfo myTimer, out string outputEventHubMessage, TraceWriter log)
{
    String msg = $"TimerTriggerCSharp1 executed at: {DateTime.Now}";
    log.Verbose(msg);   
    outputEventHubMessage = msg;
}

Here's C# script code that creates multiple messages:

public static void Run(TimerInfo myTimer, ICollector<string> outputEventHubMessage, TraceWriter log)
{
    string message = $"Event Hub message created at: {DateTime.Now}";
    log.Info(message);
    outputEventHubMessage.Add("1 " + message);
    outputEventHubMessage.Add("2 " + message);
}

Output - F# example

The following example shows an event hub trigger binding in a function.json file and an F# function that uses the binding. The function writes a message to an event hub.

Here's the binding data in the function.json file:

{
    "type": "eventHub",
    "name": "outputEventHubMessage",
    "path": "myeventhub",
    "connection": "MyEventHubSend",
    "direction": "out"
}

Here's the F# code:

let Run(myTimer: TimerInfo, outputEventHubMessage: byref<string>, log: TraceWriter) =
    let msg = sprintf "TimerTriggerFSharp1 executed at: %s" DateTime.Now.ToString()
    log.Verbose(msg);
    outputEventHubMessage <- msg;

Output - JavaScript example

The following example shows an event hub trigger binding in a function.json file and a JavaScript function that uses the binding. The function writes a message to an event hub.

Here's the binding data in the function.json file:

{
    "type": "eventHub",
    "name": "outputEventHubMessage",
    "path": "myeventhub",
    "connection": "MyEventHubSend",
    "direction": "out"
}

Here's JavaScript code that sends a single message:

module.exports = function (context, myTimer) {
    var timeStamp = new Date().toISOString();
    context.log('Event Hub message created at: ', timeStamp);   
    context.bindings.outputEventHubMessage = "Event Hub message created at: " + timeStamp;
    context.done();
};

Here's JavaScript code that sends multiple messages:

module.exports = function(context) {
    var timeStamp = new Date().toISOString();
    var message = 'Event Hub message created at: ' + timeStamp;

    context.bindings.outputEventHubMessage = [];

    context.bindings.outputEventHubMessage.push("1 " + message);
    context.bindings.outputEventHubMessage.push("2 " + message);
    context.done();
};

Output - attributes

For precompiled C# functions, use the EventHubAttribute attribute, which is defined in NuGet package Microsoft.Azure.WebJobs.ServiceBus.

The attribute's constructor takes the name of the event hub and the name of an app setting that contains the connection string. For more information about these settings, see Output - configuration. Here's an EventHub attribute example:

[FunctionName("EventHubOutput")]
[return: EventHub("outputEventHubMessage", Connection = "EventHubConnection")]
public static string Run([TimerTrigger("0 */5 * * * *")] TimerInfo myTimer, TraceWriter log)
{
    ...
}

For a complete example, see Output - precompiled C# example.

Output - configuration

The following table explains the binding configuration properties that you set in the function.json file and the EventHub attribute.

function.json property Attribute property Description
type n/a Must be set to "eventHub".
direction n/a Must be set to "out". This parameter is set automatically when you create the binding in the Azure portal.
name n/a The variable name used in function code that represents the event.
path EventHubName The name of the event hub.
connection Connection The name of an app setting that contains the connection string to the event hub's namespace. Copy this connection string by clicking the Connection Information button for the namespace, not the event hub itself. This connection string must have send permissions to send the message to the event stream.

When you're developing locally, app settings go into the local.settings.json file.

Output - usage

In C# and C# script, send messages by using a method parameter such as out string paramName. In C# script, paramName is the value specified in the name property of function.json. To write multiple messages, you can use ICollector<string> or IAsyncCollector<string> in place of out string.

In JavaScript, access the output event by using context.bindings.<name>. <name> is the value specified in the name property of function.json.

Next steps