How to use Service Bus topics and subscriptions

13 min to read Contributors
  • Seth Manheim
  • Kim Whitlatch
  • Tyson Nevil
  • John Taubensee

This article describes how to use Service Bus topics and subscriptions. The samples are written in C# and use the .NET APIs. The scenarios covered include creating topics and subscriptions, creating subscription filters, sending messages to a topic, receiving messages from a subscription, and deleting topics and subscriptions. For more information about topics and subscriptions, see the Next steps section.

Note

To complete this tutorial, you need an Azure account. You can activate your MSDN subscriber benefits or sign up for a free account.

What are Service Bus topics and subscriptions?

Service Bus topics and subscriptions support a publish/subscribe messaging communication model. When using topics and subscriptions, components of a distributed application do not communicate directly with each other; instead they exchange messages via a topic, which acts as an intermediary.

TopicConcepts

In contrast with Service Bus queues, in which each message is processed by a single consumer, topics and subscriptions provide a "one-to-many" form of communication, using a publish/subscribe pattern. It is possible to register multiple subscriptions to a topic. When a message is sent to a topic, it is then made available to each subscription to handle/process independently.

A subscription to a topic resembles a virtual queue that receives copies of the messages that were sent to the topic. You can optionally register filter rules for a topic on a per-subscription basis, which enables you to filter or restrict which messages to a topic are received by which topic subscriptions.

Service Bus topics and subscriptions enable you to scale and process a very large number of messages across many users and applications.

Create a namespace

To begin using Service Bus topics and subscriptions in Azure, you must first create a service namespace. A namespace provides a scoping container for addressing Service Bus resources within your application.

To create a namespace:

  1. Log on to the Azure portal.
  2. In the left navigation pane of the portal, click New, then click Enterprise Integration, and then click Service Bus.
  3. In the Create namespace dialog, enter a namespace name. The system immediately checks to see if the name is available.
  4. After making sure the namespace name is available, choose the pricing tier (Basic, Standard, or Premium).
  5. In the Subscription field, choose an Azure subscription in which to create the namespace.
  6. In the Resource group field, choose an existing resource group in which the namespace will live, or create a new one.
  7. In Location, choose the country or region in which your namespace should be hosted.

    Create namespace

  8. Click the Create button. The system now creates your namespace and enables it. You might have to wait several minutes as the system provisions resources for your account.

Obtain the credentials

  1. In the list of namespaces, click the newly created namespace name.
  2. In the Service Bus namespace blade, click Shared access policies.
  3. In the Shared access policies blade, click RootManageSharedAccessKey.

    connection-info

  4. In the Policy: RootManageSharedAccessKey blade, click the copy button next to Connection string–primary key, to copy the connection string to your clipboard for later use.

    connection-string

Configure the application to use Service Bus

When you create an application that uses Service Bus, you must add a reference to the Service Bus assembly and include the corresponding namespaces. The easiest way to do this is to download the appropriate NuGet package.

Get the Service Bus NuGet package

The Service Bus NuGet package is the easiest way to get the Service Bus API and to configure your application with all the necessary Service Bus dependencies. To install the Service Bus NuGet package in your project, do the following:

  1. In Solution Explorer, right-click References, then click Manage NuGet Packages.
  2. Search for "Service Bus" and select the Microsoft Azure Service Bus item. Click Install to complete the installation, then close the following dialog box:

You are now ready to write code for Service Bus.

Create a Service Bus connection string

Service Bus uses a connection string to store endpoints and credentials. You can put your connection string in a configuration file, rather than hard-coding it:

  • When using Azure services, it is recommended that you store your connection string using the Azure service configuration system (.csdef and .cscfg files).
  • When using Azure websites or Azure Virtual Machines, it is recommended that you store your connection string using the .NET configuration system (for example, the Web.config file).

In both cases, you can retrieve your connection string using the CloudConfigurationManager.GetSetting method, as shown later in this article.

Configure your connection string

The service configuration mechanism enables you to dynamically change configuration settings from the Azure portal without redeploying your application. For example, add a Setting label to your service definition (.csdef) file, as shown in the next example.

<ServiceDefinition name="Azure1">
...
    <WebRole name="MyRole" vmsize="Small">
        <ConfigurationSettings>
            <Setting name="Microsoft.ServiceBus.ConnectionString" />
        </ConfigurationSettings>
    </WebRole>
...
</ServiceDefinition>

You then specify values in the service configuration (.cscfg) file.

<ServiceConfiguration serviceName="Azure1">
...
    <Role name="MyRole">
        <ConfigurationSettings>
            <Setting name="Microsoft.ServiceBus.ConnectionString"
                     value="Endpoint=sb://yourServiceNamespace.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=yourKey" />
        </ConfigurationSettings>
    </Role>
...
</ServiceConfiguration>

Use the Shared Access Signature (SAS) key name and key values retrieved from the portal as described previously.

Configure your connection string when using Azure websites or Azure Virtual Machines

When using websites or Virtual Machines, it is recommended that you use the .NET configuration system (for example, Web.config). You store the connection string using the <appSettings> element.

<configuration>
    <appSettings>
        <add key="Microsoft.ServiceBus.ConnectionString"
             value="Endpoint=sb://yourServiceNamespace.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=yourKey" />
    </appSettings>
</configuration>

Use the SAS name and key values that you retrieved from the Azure portal, as described previously.

Create a topic

You can perform management operations for Service Bus topics and subscriptions using the NamespaceManager class. This class provides methods to create, enumerate, and delete topics.

The following example constructs a NamespaceManager object using the Azure CloudConfigurationManager class with a connection string consisting of the base address of a Service Bus namespace and the appropriate SAS credentials with permissions to manage it. This connection string is of the following form:

Endpoint=sb://<yourNamespace>.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=<yourKey>

Use the following example, given the configuration settings in the previous section.

// Create the topic if it does not exist already.
string connectionString =
    CloudConfigurationManager.GetSetting("Microsoft.ServiceBus.ConnectionString");

var namespaceManager =
    NamespaceManager.CreateFromConnectionString(connectionString);

if (!namespaceManager.TopicExists("TestTopic"))
{
    namespaceManager.CreateTopic("TestTopic");
}

There are overloads of the CreateTopic method that enable you to set properties of the topic; for example, to set the default time-to-live (TTL) value to be applied to messages sent to the topic. These settings are applied by using the TopicDescription class. The following example shows how to create a topic named TestTopic with a maximum size of 5 GB and a default message TTL of 1 minute.

// Configure Topic Settings.
TopicDescription td = new TopicDescription("TestTopic");
td.MaxSizeInMegabytes = 5120;
td.DefaultMessageTimeToLive = new TimeSpan(0, 1, 0);

// Create a new Topic with custom settings.
string connectionString =
    CloudConfigurationManager.GetSetting("Microsoft.ServiceBus.ConnectionString");

var namespaceManager =
    NamespaceManager.CreateFromConnectionString(connectionString);

if (!namespaceManager.TopicExists("TestTopic"))
{
    namespaceManager.CreateTopic(td);
}
Note

You can use the TopicExists method on NamespaceManager objects to check whether a topic with a specified name already exists within a namespace.

Create a subscription

You can also create topic subscriptions using the NamespaceManager class. Subscriptions are named and can have an optional filter that restricts the set of messages passed to the subscription's virtual queue.

Important

In order for messages to be received by a subscription, you must create that subscription before sending any messages to the topic. If there are no subscriptions to a topic, the topic discards those messages.

Create a subscription with the default (MatchAll) filter

If no filter is specified when a new subscription is created, the MatchAll filter is the default filter that is used. When you use the MatchAll filter, all messages published to the topic are placed in the subscription's virtual queue. The following example creates a subscription named "AllMessages" and uses the default MatchAll filter.

string connectionString =
    CloudConfigurationManager.GetSetting("Microsoft.ServiceBus.ConnectionString");

var namespaceManager =
    NamespaceManager.CreateFromConnectionString(connectionString);

if (!namespaceManager.SubscriptionExists("TestTopic", "AllMessages"))
{
    namespaceManager.CreateSubscription("TestTopic", "AllMessages");
}

Create subscriptions with filters

You can also set up filters that enable you to specify which messages sent to a topic should appear within a specific topic subscription.

The most flexible type of filter supported by subscriptions is the SqlFilter class, which implements a subset of SQL92. SQL filters operate on the properties of the messages that are published to the topic. For more information about the expressions that can be used with a SQL filter, see the SqlFilter.SqlExpression syntax.

The following example creates a subscription named HighMessages with a SqlFilter object that only selects messages that have a custom MessageNumber property greater than 3.

// Create a "HighMessages" filtered subscription.
SqlFilter highMessagesFilter =
   new SqlFilter("MessageNumber > 3");

namespaceManager.CreateSubscription("TestTopic",
   "HighMessages",
   highMessagesFilter);

Similarly, the following example creates a subscription named LowMessages with a SqlFilter that only selects messages that have a MessageNumber property less than or equal to 3.

// Create a "LowMessages" filtered subscription.
SqlFilter lowMessagesFilter =
   new SqlFilter("MessageNumber <= 3");

namespaceManager.CreateSubscription("TestTopic",
   "LowMessages",
   lowMessagesFilter);

Now when a message is sent to TestTopic, it is always delivered to receivers subscribed to the AllMessages topic subscription, and selectively delivered to receivers subscribed to the HighMessages and LowMessages topic subscriptions (depending on the message content).

Send messages to a topic

To send a message to a Service Bus topic, your application creates a TopicClient object using the connection string.

The following code demonstrates how to create a TopicClient object for the TestTopic topic created earlier using the CreateFromConnectionString API call.

string connectionString =
    CloudConfigurationManager.GetSetting("Microsoft.ServiceBus.ConnectionString");

TopicClient Client =
    TopicClient.CreateFromConnectionString(connectionString, "TestTopic");

Client.Send(new BrokeredMessage());

Messages sent to Service Bus topics are instances of the BrokeredMessage class. BrokeredMessage objects have a set of standard properties (such as Label and TimeToLive), a dictionary that is used to hold custom application-specific properties, and a body of arbitrary application data. An application can set the body of the message by passing any serializable object to the constructor of the BrokeredMessage object, and the appropriate DataContractSerializer is then used to serialize the object. Alternatively, a System.IO.Stream can be provided.

The following example demonstrates how to send five test messages to the TestTopic TopicClient object obtained in the previous code example. Note that the MessageNumber property value of each message varies depending on the iteration of the loop (this determines which subscriptions receive it).

for (int i=0; i<5; i++)
{
  // Create message, passing a string message for the body.
  BrokeredMessage message = new BrokeredMessage("Test message " + i);

  // Set additional custom app-specific property.
  message.Properties["MessageNumber"] = i;

  // Send message to the topic.
  Client.Send(message);
}

Service Bus topics support a maximum message size of 256 KB in the Standard tier and 1 MB in the Premium tier. The header, which includes the standard and custom application properties, can have a maximum size of 64 KB. There is no limit on the number of messages held in a topic but there is a cap on the total size of the messages held by a topic. This topic size is defined at creation time, with an upper limit of 5 GB. If partitioning is enabled, the upper limit is higher. For more information, see Partitioned messaging entities.

How to receive messages from a subscription

The recommended way to receive messages from a subscription is to use a SubscriptionClient object. SubscriptionClient objects can work in two different modes: ReceiveAndDelete and PeekLock.

When using the ReceiveAndDelete mode, receive is a single-shot operation; that is, when Service Bus receives a read request for a message in a subscription, it marks the message as being consumed and returns it to the application. ReceiveAndDelete mode is the simplest model and works best for scenarios in which an application can tolerate not processing a message in the event of a failure. To understand this, consider a scenario in which the consumer issues the receive request and then crashes before processing it. Because Service Bus has marked the message as consumed, when the application restarts and begins consuming messages again, it will have missed the message that was consumed prior to the crash.

In PeekLock mode (which is the default mode), the receive process becomes a two-stage operation, which makes it possible to support applications that cannot tolerate missing messages. When Service Bus receives a request, it finds the next message to be consumed, locks it to prevent other consumers receiving it, and then returns it to the application. After the application finishes processing the message (or stores it reliably for future processing), it completes the second stage of the receive process by calling Complete on the received message. When Service Bus sees the Complete call, it marks the message as being consumed and removes it from the subscription.

The following example demonstrates how messages can be received and processed using the default PeekLock mode. To specify a different ReceiveMode value, you can use another overload for CreateFromConnectionString. This example uses the OnMessage callback to process messages as they arrive into the HighMessages subscription.

string connectionString =
    CloudConfigurationManager.GetSetting("Microsoft.ServiceBus.ConnectionString");

SubscriptionClient Client =
    SubscriptionClient.CreateFromConnectionString
            (connectionString, "TestTopic", "HighMessages");

// Configure the callback options.
OnMessageOptions options = new OnMessageOptions();
options.AutoComplete = false;
options.AutoRenewTimeout = TimeSpan.FromMinutes(1);

Client.OnMessage((message) =>
{
    try
    {
        // Process message from subscription.
        Console.WriteLine("\n**High Messages**");
        Console.WriteLine("Body: " + message.GetBody<string>());
        Console.WriteLine("MessageID: " + message.MessageId);
        Console.WriteLine("Message Number: " +
            message.Properties["MessageNumber"]);

        // Remove message from subscription.
        message.Complete();
    }
    catch (Exception)
    {
        // Indicates a problem, unlock message in subscription.
        message.Abandon();
    }
}, options);

This example configures the OnMessage callback using an OnMessageOptions object. AutoComplete is set to false to enable manual control of when to call Complete on the received message. AutoRenewTimeout is set to 1 minute, which causes the client to wait for up to one minute before terminating the auto-renewal feature and the client makes a new call to check for messages. This property value reduces the number of times the client makes chargeable calls that do not retrieve messages.

How to handle application crashes and unreadable messages

Service Bus provides functionality to help you gracefully recover from errors in your application or difficulties processing a message. If a receiving application is unable to process the message for some reason, then it can call the Abandon method on the received message (instead of the Complete method). This causes Service Bus to unlock the message within the subscription and make it available to be received again, either by the same consuming application or by another consuming application.

There is also a time-out associated with a message locked within the subscription, and if the application fails to process the message before the lock time-out expires (for example, if the application crashes), then Service Bus unlocks the message automatically and makes it available to be received again.

In the event that the application crashes after processing the message but before the Complete request is issued, the message will be redelivered to the application when it restarts. This is often called At Least Once processing; that is, each message is processed at least once but in certain situations the same message may be redelivered. If the scenario cannot tolerate duplicate processing, then application developers should add additional logic to their application to handle duplicate message delivery. This is often achieved using the MessageId property of the message, which remains constant across delivery attempts.

Delete topics and subscriptions

The following example demonstrates how to delete the topic TestTopic from the HowToSample service namespace.

// Delete Topic.
namespaceManager.DeleteTopic("TestTopic");

Deleting a topic also deletes any subscriptions that are registered with the topic. Subscriptions can also be deleted independently. The following code demonstrates how to delete a subscription named HighMessages from the TestTopic topic.

namespaceManager.DeleteSubscription("TestTopic", "HighMessages");

Next steps

Now that you've learned the basics of Service Bus topics and subscriptions, follow these links to learn more.