Automatically scale up Azure Event Hubs throughput units (standard tier)

Azure Event Hubs is a highly scalable data streaming platform. As such, Event Hubs usage often increases after starting to use the service. Such usage requires increasing the predetermined throughput units (TUs) to scale Event Hubs and handle larger transfer rates. The Auto-inflate feature of Event Hubs automatically scales up by increasing the number of TUs, to meet usage needs. Increasing TUs prevents throttling scenarios, in which:

  • Data ingress rates exceed set TUs
  • Data egress request rates exceed set TUs

The Event Hubs service increases the throughput when load increases beyond the minimum threshold, without any requests failing with ServerBusy errors.

Note

To learn more about the premium tier, see Event Hubs Premium.

How Auto-inflate works in standard tier

Event Hubs traffic is controlled by TUs (standard tier). For the limits such as ingress and egress rates per TU, see Event Hubs quotas and limits. Auto-inflate enables you to start small with the minimum required TUs you choose. The feature then scales automatically to the maximum limit of TUs you need, depending on the increase in your traffic. Auto-inflate provides the following benefits:

  • An efficient scaling mechanism to start small and scale up as you grow.
  • Automatically scale to the specified upper limit without throttling issues.
  • More control over scaling, because you control when and how much to scale.

Note

Auto-inflate does not automatically scale down the number of TUs when ingress or egress rates drop below the limits.

Enable Auto-inflate on a namespace

You can enable or disable Auto-inflate on a standard tier Event Hubs namespace by using either Azure portal or an Azure Resource Manager template.

For a premium Event Hubs namespace, the feature is automatically enabled. You can't disable it.

Note

Basic tier Event Hubs namespaces do not support Auto-inflate.

Use Azure portal

In the Azure portal, you can enable the feature when creating a standard Event Hubs namespace or after the namespace is created. You can also set TUs for the namespace and specify maximum limit of TUs

You can enable the Auto-inflate feature when creating an Event Hubs namespace. The follow image shows you how to enable the auto-inflate feature for a standard tier namespace and configure TUs to start with and the maximum number of TUs.

Screenshot of enabling auto inflate at the time event hub creation for a standard tier namespace

With this option enabled, you can start small with your TUs and scale up as your usage needs increase. The upper limit for inflation doesn't immediately affect pricing, which depends on the number of TUs used per hour.

To enable the Auto-inflate feature and modify its settings for an existing, follow these steps:

  1. On the Event Hubs Namespace page, select Scale under Settings on the left menu.

  2. In the Scale Settings page, select the checkbox for Enable (if the autoscale feature wasn't enabled).

    Screenshot of enabling auto-inflate for an existing standard namespace

  3. Enter the maximum number of throughput units or use the scrollbar to set the value.

  4. (optional) Update the minimum number of throughput units at the top of this page.

Note

When you apply the auto-inflate configuration to increase throughput units, the Event Hubs service emits diagnostic logs that give you information about why and when the throughput increased. To enable diagnostic logging for an event hub, select Diagnostic settings on the left menu on the Event Hub page in the Azure portal. For more information, see Set up diagnostic logs for an Azure event hub.

Use an Azure Resource Manager template

You can enable Auto-inflate during an Azure Resource Manager template deployment. For example, set the isAutoInflateEnabled property to true and set maximumThroughputUnits to 10. For example:

"resources": [
        {
            "apiVersion": "2017-04-01",
            "name": "[parameters('namespaceName')]",
            "type": "Microsoft.EventHub/Namespaces",
            "location": "[variables('location')]",
            "sku": {
                "name": "Standard",
                "tier": "Standard"
            },
            "properties": {
                "isAutoInflateEnabled": true,
                "maximumThroughputUnits": 10
            },
            "resources": [
                {
                    "apiVersion": "2017-04-01",
                    "name": "[parameters('eventHubName')]",
                    "type": "EventHubs",
                    "dependsOn": [
                        "[concat('Microsoft.EventHub/namespaces/', parameters('namespaceName'))]"
                    ],
                    "properties": {},
                    "resources": [
                        {
                            "apiVersion": "2017-04-01",
                            "name": "[parameters('consumerGroupName')]",
                            "type": "ConsumerGroups",
                            "dependsOn": [
                                "[parameters('eventHubName')]"
                            ],
                            "properties": {}
                        }
                    ]
                }
            ]
        }
    ]

For the complete template, see the Create Event Hubs namespace and enable inflate template on GitHub.

Next steps

You can learn more about Event Hubs by visiting the following links: