How to use the Spring Boot Starter for Apache Kafka with Azure Event Hubs
This article demonstrates how to configure a Java-based Spring Cloud Stream Binder created with the Spring Boot Initializer to use Apache Kafka with Azure Event Hubs.
Prerequisites
The following prerequisites are required in order to follow the steps in this article:
- An Azure subscription; if you don't already have an Azure subscription, you can activate your MSDN subscriber benefits or sign up for a free Azure account.
- A supported Java Development Kit (JDK). For more information about the JDKs available for use when developing on Azure, see Java support on Azure and Azure Stack.
- Apache Maven, version 3.0 or later.
Note
- Spring Boot version 2.0 or greater is required to complete the steps in this article.
Create an Azure Event Hub using the Azure portal
Create an Azure Event Hub Namespace
Browse to the Azure portal at https://portal.azure.com/ and sign in.
Select Create a resource, then Search the Marketplace, then search for Event Hubs.
Select Create.

On the Create Namespace page, enter the following information:
- Choose the Subscription you want to use for your namespace.
- Specify whether to create a new Resource group for your namespace, or choose an existing resource group.
- Enter a unique Namespace name, which will become part of the URI for your event hub namespace. For example: if you entered wingtiptoys-space for the Name, the URI would be
wingtiptoys-space.servicebus.windows.net. - Specify the Location for your event hub namespace.
- Specify the Pricing tier, which will limit your usage scenarios .
- You can also specify the Throughput units for the namespace.

When you have specified the options listed above, select Review + Create.
Review the specification and select Create to create your namespace.
Create an Azure Event Hub in your namespace
After your namespace is deployed, you can create an event hub in the namespace.
Navigate to the namespace created in the previous step.
Select Event Hub in top menu bar.
Name the event hub.
Select Create.

Create a simple Spring Boot application with the Spring Initializr
Browse to https://start.spring.io/.
Specify the following options:
Generate a Maven project with Java.
Specify a Spring Boot version that is equal to or greater than 2.0.
Specify the Group and Artifact names for your application.
Add the Web dependency.

Note
- The Spring Initializr uses the Group and Artifact names to create the package name; for example: com.wingtiptoys.kafka.
When you have specified the options listed above, click Generate Project.
When prompted, download the project to a path on your local computer.
After you have extracted the files on your local system, your simple Spring Boot application will be ready for editing.
Configure your Spring Boot app to use the Spring Cloud Kafka Stream and Azure Event Hub starters
Locate the pom.xml file in the root directory of your app; for example:
C:\SpringBoot\kafka\pom.xml
-or-
/users/example/home/kafka/pom.xml
Open the pom.xml file in a text editor, and add the Event Hubs Kafka starters to the list of
<dependencies>:<dependency> <groupId>com.azure.spring</groupId> <artifactId>azure-spring-cloud-starter-eventhubs-kafka</artifactId> <version>2.13.0</version> </dependency>Save and close the pom.xml file.
Sign into Azure and set your subscription
First, use the following steps to authenticate using the Azure CLI.
Optionally, sign out and delete some authentication files to remove any lingering credentials:
az logout rm ~/.azure/accessTokens.json rm ~/.azure/azureProfile.jsonSign into your Azure account by using the Azure CLI:
az loginFollow the instructions to complete the sign-in process.
List your subscriptions:
az account listAzure will return a list of your subscriptions. Copy the
idvalue for the subscription that you want to use; for example:[ { "cloudName": "AzureCloud", "id": "ssssssss-ssss-ssss-ssss-ssssssssssss", "name": "Converted Windows Azure MSDN - Visual Studio Ultimate", "state": "Enabled", "tenantId": "tttttttt-tttt-tttt-tttt-tttttttttttt", "user": { "name": "contoso@microsoft.com", "type": "user" } } ]Specify the GUID for the subscription you want to use with Azure; for example:
az account set -s ssssssss-ssss-ssss-ssss-ssssssssssss
Create a service principal
Azure AD service principals provide access to Azure resources within your subscription. You can think of a service principal as a user identity for a service. "Service" is any application, service, or platform that needs to access Azure resources. You can configure a service principal with access rights scoped only to those resources you specify. Then, configure your application or service to use the service principal's credentials to access those resources.
To create a service principal, use the following command.
az ad sp create-for-rbac --name contososp --role Contributor
The value of the name option must be unique within your subscription. Save aside the values returned from the command for use later in the tutorial. The return JSON will look similar to the following output:
{
"appId": "sample-app-id",
"displayName": "contososp",
"name": "http://contososp",
"password": "sample-password",
"tenant": "sample-tenant"
}
Configure your Spring Boot app to use your Azure Event Hub
Add an application.yaml in the resources directory of your app; for example:
C:\SpringBoot\kafka\src\main\resources\application.yaml
-or-
/users/example/home/kafka/src/main/resources/application.yaml
Open the application.yaml file in a text editor, add the following lines, and then replace the sample values with the appropriate properties for your event hub:
spring: cloud: azure: client-id: <your client ID> client-secret: <your client secret> tenant-id: <your tenant ID> resource-group: <your resource group> subscription-id: <your subscription ID> eventhub: namespace: wingtiptoys stream: function: definition: consume;supply bindings: consume-in-0: destination: wingtiptoyshub group: $Default supply-out-0: destination: wingtiptoyshubWhere:
Field Description spring.cloud.azure.client-idThe appIdfrom the return JSON fromaz ad sp create-for-rbac.spring.cloud.azure.client-secretThe passwordfrom the return JSON fromaz ad sp create-for-rbac.spring.cloud.azure.tenant-idThe tenantfrom the return JSON fromaz ad sp create-for-rbac.spring.cloud.azure.resource-groupSpecifies the Azure Resource Group that contains your Azure Event Hub. spring.cloud.azure.subscription-idSpecifies the Azure Subscription that contains your Azure Event Hub. spring.cloud.azure.regionSpecifies the geographical region that you specified when you created your Azure Event Hub. spring.cloud.azure.auto-create-resourcesSpecifies true to enable automatic creation of related resources if they don't exist. spring.cloud.azure.eventhub.namespaceSpecifies the unique name that you specified when you created your Azure Event Hub Namespace. spring.cloud.stream.bindings.input.destinationSpecifies the input destination Azure Event Hub, which for this tutorial is the hub you created earlier in this tutorial. spring.cloud.stream.bindings.input.groupSpecifies a Consumer Group from Azure Event Hub, which can be set to '$Default' in order to use the basic consumer group that was created when you created your Azure Event Hub. spring.cloud.stream.bindings.output.destinationSpecifies the output destination Azure Event Hub, which for this tutorial will be the same as the input destination. Note
If you enable automatic topic creation, be sure to add the configuration item
spring.cloud.stream.kafka.binder.replicationFactor, with the value set to at least 1. For more information, see Spring Cloud Stream Kafka Binder Reference Guide.Save and close the application.yaml file.
Add sample code to implement basic event hub functionality
In this section, you create the necessary Java classes for sending events to your event hub.
Modify the main application class
Locate the main application Java file in the package directory of your app; for example:
C:\SpringBoot\kafka\src\main\java\com\wingtiptoys\kafka\EventhubApplication.java
-or-
/users/example/home/kafka/src/main/java/com/wingtiptoys/kafka/EventhubApplication.java
Open the main application Java file in a text editor, and add the following lines to the file:
package com.wingtiptoys.kafka; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.context.annotation.Bean; import org.springframework.messaging.Message; import reactor.core.publisher.Flux; import reactor.core.publisher.Sinks; import java.util.function.Consumer; import java.util.function.Supplier; @SpringBootApplication public class KafkaApplication { private static final Logger LOGGER = LoggerFactory.getLogger(KafkaApplication.class); public static void main(String[] args) { SpringApplication.run(KafkaApplication.class, args); } @Bean public Sinks.Many<Message<String>> many() { return Sinks.many().unicast().onBackpressureBuffer(); } @Bean public Supplier<Flux<Message<String>>> supply(Sinks.Many<Message<String>> many) { return () -> many.asFlux() .doOnNext(m -> LOGGER.info("Manually sending message {}", m)) .doOnError(t -> LOGGER.error("Error encountered", t)); } @Bean public Consumer<Message<String>> consume() { return message -> LOGGER.info("New message received: '{}'", message.getPayload()); } }Save and close the main application Java file.
Create a new class for the source connector
Create a new Java file named KafkaSource.java in the package directory of your app, then open the file in a text editor and add the following lines:
package com.wingtiptoys.kafka; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.messaging.Message; import org.springframework.messaging.support.GenericMessage; import org.springframework.web.bind.annotation.PostMapping; import org.springframework.web.bind.annotation.RequestParam; import org.springframework.web.bind.annotation.RestController; import reactor.core.publisher.Sinks; @RestController public class KafkaSource { @Autowired private Sinks.Many<Message<String>> many; @PostMapping("/messages") public String sendMessage(@RequestParam String message) { many.emitNext(new GenericMessage<>(message), Sinks.EmitFailureHandler.FAIL_FAST); return message; } }Save and close the KafkaSource.java file.
Build and test your application
Open a command prompt and change directory to the folder where your pom.xml file is located; for example:
cd C:\SpringBoot\kafka-or-
cd /users/example/home/kafkaBuild your Spring Boot application with Maven and run it; for example:
mvn clean package -Dmaven.test.skip=true mvn spring-boot:runOnce your application is running, you can use curl to test your application; for example:
curl -X POST http://localhost:8080/messages?message=helloYou should see "hello" posted to your application's logs. For example:
2021-06-02 14:47:13.956 INFO 23984 --- [oundedElastic-1] o.a.kafka.common.utils.AppInfoParser : Kafka version: 2.6.0 2021-06-02 14:47:13.957 INFO 23984 --- [oundedElastic-1] o.a.kafka.common.utils.AppInfoParser : Kafka commitId: 62abe01bee039651 2021-06-02 14:47:13.957 INFO 23984 --- [oundedElastic-1] o.a.kafka.common.utils.AppInfoParser : Kafka startTimeMs: 1622616433956 2021-06-02 14:47:16.668 INFO 23984 --- [container-0-C-1] com.wingtiptoys.kafka.KafkaApplication : New message received: 'hello'
Clean up resources
When no longer needed, use the Azure portal to delete the resources created in this article to avoid unexpected charges.
Next steps
To learn more about Spring and Azure, continue to the Spring on Azure documentation center.
Additional Resources
For more information about Azure support for Event Hub Stream Binder and Apache Kafka, see the following articles:
For more information about using Azure with Java, see the [Azure for Java Developers] and the Working with Azure DevOps and Java.
The Spring Framework is an open-source solution that helps Java developers create enterprise-level applications. One of the more-popular projects that is built on top of that platform is Spring Boot, which provides a simplified approach for creating stand-alone Java applications. To help developers get started with Spring Boot, several sample Spring Boot packages are available at https://github.com/spring-guides/. In addition to choosing from the list of basic Spring Boot projects, the Spring Initializr helps developers get started with creating custom Spring Boot applications.