Azure Web PubSub service internals

Azure Web PubSub Service provides an easy way to publish/subscribe messages using simple WebSocket connections.

  • Clients can be written in any language that has Websocket support.
  • Both text and binary messages are supported within one connection.
  • There's a simple protocol for clients to do direct client-to-client message publishing.
  • The service manages the WebSocket connections for you.

Terms

  • Service: Azure Web PubSub Service.
  • Connection: A connection, also known as a client or a client connection, represents an individual WebSocket connection connected to the Web PubSub service. When successfully connected, a unique connection ID is assigned to this connection by the Web PubSub service.

  • Hub: A hub is a logical concept for a set of client connections. Usually you use one hub for one purpose, for example, a chat hub, or a notification hub. When a client connection connects, it connects to a hub, and during its lifetime it belongs to that hub. Once a client connection connects to the hub, the hub exists. Different applications can share one Azure Web PubSub service by using different hub names.

  • Group: A group is a subset of connections to the hub. You can add a client connection to a group, or remove the client connection from the group, anytime you want. For example, when a client joins a chat room, or when a client leaves the chat room, this chat room can be considered to be a group. A client can join multiple groups, and a group can contain multiple clients. The group is like a group "session", the group session is created once someone joins the group, and the session is gone when no one is in the group. Messages sent to the group are delivered to all of the clients connected to the group.

  • User: Connections to Web PubSub can belong to one user. A user might have multiple connections, for example when a single user is connected across multiple devices or multiple browser tabs.

  • Message: When the client is connected, it can send messages to the upstream application, or receive messages from the upstream application, through the WebSocket connection. Messages can be in plain text, binary, or JSON format and have a maximum size of 1 MB.

  • Client Connection and ConnectionId: A client connects to the /client endpoint, when connected, a unique connectionId is generated by the service as the unique identity of the client connection. Users can then manage the client connection using this connectionId. Details are described in Client protocol section.

  • Client Events: Events are created during the lifecycle of a client connection. For example, a simple WebSocket client connection creates a connect event when it tries to connect to the service, a connected event when it successfully connected to the service, a message event when it sends messages to the service and a disconnected event when it disconnects from the service. Details about client events are illustrated in Client protocol section.

  • Event Handler: The event handler contains the logic to handle the client events. Register and configure event handlers in the service through the portal or Azure CLI beforehand. Details are described in Event handler section. The place to host the event handler logic is considered as the server-side.

  • Server: The server can handle client events, manage client connections, and publish messages to groups. Details about server are described in the Server protocol section.

Workflow

Diagram showing the Web PubSub service workflow.

Workflow as shown in the above graph:

  1. A client connects to the service /client endpoint using WebSocket transport. Service forward every WebSocket frame to the configured upstream(server). The WebSocket connection can connect with any custom subprotocol for the server to handle, or it can connect with the service-supported subprotocol json.webpubsub.azure.v1, which empowers the clients to do pub/sub directly. Details are described in client protocol.
  2. The service invokes the server using CloudEvents HTTP protocol on different client events. CloudEvents is a standardized and protocol-agnostic definition of the structure and metadata description of events hosted by the Cloud Native Computing Foundation (CNCF). Details are described in server protocol.
  3. The Web PubSub server can invoke the service using the REST API to send messages to clients or to manage the connected clients. Details are described in server protocol

Client protocol

A client connection connects to the /client endpoint of the service using WebSocket protocol. The WebSocket protocol provides full-duplex communication channels over a single TCP connection and was standardized by the IETF as RFC 6455 in 2011. Most languages have native support to start WebSocket connections.

Our service supports two kinds of clients:

The simple WebSocket client

A simple WebSocket client, as the naming indicates, is a simple WebSocket connection. It can also have its custom subprotocol.

For example, in JS, a simple WebSocket client can be created using the following code.

// simple WebSocket client1
var client1 = new WebSocket('wss://test.webpubsub.azure.com/client/hubs/hub1');

// simple WebSocket client2 with some custom subprotocol
var client2 = new WebSocket('wss://test.webpubsub.azure.com/client/hubs/hub1', 'custom.subprotocol')

A simple WebSocket client follows a client<->server architecture, as the below sequence diagram shows: Diagram showing the sequence for a client connection.

  1. When the client starts a WebSocket handshake, the service tries to invoke the connect event handler (the server) for WebSocket handshake. Developers can use this handler to handle the WebSocket handshake, determine the subprotocol to use, authenticate the client, and join the client to groups.
  2. When the client is successfully connected, the service invokes a connected event handler. It works as a notification and doesn't block the client from sending messages. Developers can use this handler to do data storage and can respond with messages to the client.
  3. When the client sends messages, the service triggers the message event to the event handler (the server) to handle the messages sent. This event is a general event containing the messages sent in a WebSocket frame. Your code needs to dispatch the messages inside this event handler.
  4. When the client disconnects, the service tries to trigger the disconnected event to the event handler (the server) once it detects the disconnect.

The events fall into two categories:

  • Synchronous events (blocking) Synchronous events block the client workflow. When such an event trigger fails, the service drops the client connection.
    • connect
    • message
  • Asynchronous events (non-blocking) Asynchronous events don't block the client workflow, it acts as some notification to the upstream event handler. When such an event trigger fails, the service logs the error detail.
    • connected
    • disconnected

Scenarios

These connections can be used in a typical client-server architecture where the client sends messages to the server and the server handles incoming messages using Event Handlers. It can also be used when customers apply existing subprotocols in their application logic.

The PubSub WebSocket client

The service also supports a specific subprotocol called json.webpubsub.azure.v1, which empowers the clients to do publish/subscribe directly instead of a round trip to the upstream server. We call the WebSocket connection with json.webpubsub.azure.v1 subprotocol a PubSub WebSocket client. For more information, see the Web PubSub client specification on GitHub.

For example, in JS, a PubSub WebSocket client can be created using the following code.

// PubSub WebSocket client
var pubsub = new WebSocket('wss://test.webpubsub.azure.com/client/hubs/hub1', 'json.webpubsub.azure.v1');

A PubSub WebSocket client can:

  • Join a group, for example:

    {
        "type": "joinGroup",
        "group": "<group_name>"
    }
    
  • Leave a group, for example:

    {
        "type": "leaveGroup",
        "group": "<group_name>"
    }
    
  • Publish messages to a group, for example:

    {
        "type": "sendToGroup",
        "group": "<group_name>",
        "data": { "hello": "world" }
    }
    
  • Send custom events to the upstream server, for example:

    {
        "type": "event",
        "event": "<event_name>",
        "data": { "hello": "world" }
    }
    

PubSub WebSocket Subprotocol contains the details of the json.webpubsub.azure.v1 subprotocol.

You may have noticed that for a simple WebSocket client, the server is a must have role to handle the events from clients. A simple WebSocket connection always triggers a message event when it sends messages, and always relies on the server-side to process messages and do other operations. With the help of the json.webpubsub.azure.v1 subprotocol, an authorized client can join a group and publish messages to a group directly. It can also route messages to different upstream (event handlers) by customizing the event the message belongs.

Scenarios:

Such clients can be used when clients want to talk to each other. Messages are sent from client2 to the service and the service delivers the message directly to client1 if the clients are authorized to do so.

Client1:

var client1 = new WebSocket("wss://xxx.webpubsub.azure.com/client/hubs/hub1", "json.webpubsub.azure.v1");
client1.onmessage = e => {
    if (e.data) {
        var message = JSON.parse(e.data);
        if (message.type === "message" 
        && message.group === "Group1"){
            // Only print messages from Group1
            console.log(message.data);
        }
    }
};

client1.onopen = e => {
    client1.send(JSON.stringify({
        type: "joinGroup",
        group: "Group1"
    }));
};

Client2:

var client2 = new WebSocket("wss://xxx.webpubsub.azure.com/client/hubs/hub1", "json.webpubsub.azure.v1");
client2.onopen = e => {
    client2.send(JSON.stringify({
        type: "sendToGroup",
        group: "Group1",
        data: "Hello Client1"
    });
};

As the above example shows, client2 sends data directly to client1 by publishing messages to Group1 which client1 is in.

Client message limit

The maximum allowed message size for one WebSocket frame is 1MB.

Client authentication

Authentication workflow

Client uses a signed JWT token to connect to the service. The upstream can also reject the client when it's connect event handler of the incoming client. The event handler authenticates the client by specifying the userId and the roles the client has in the webhook response, or decline the client with 401. Event handler section describes it in detail.

The following graph describes the workflow.

Diagram showing the client authentication workflow.

As you may have noticed when we describe the PubSub WebSocket clients, that a client can publish to other clients only when it's authorized to. The roles of the client determines the initial permissions the client have:

Role Permission
Not specified The client can send events.
webpubsub.joinLeaveGroup The client can join/leave any group.
webpubsub.sendToGroup The client can publish messages to any group.
webpubsub.joinLeaveGroup.<group> The client can join/leave group <group>.
webpubsub.sendToGroup.<group> The client can publish messages to group <group>.

The server-side can also grant or revoke permissions of the client dynamically through server protocol as to be illustrated in a later section.

Server protocol

Server protocol provides the functionality for the server to manage the client connections and the groups.

In general, server protocol contains two roles:

  1. Event handler
  2. Connection manager

Event handler

The event handler handles the incoming client events. Event handlers are registered and configured in the service through the portal or Azure CLI. When a client event is triggered, the service can identify if the event is to be handled or not. Now we use PUSH mode to invoke the event handler. The event handler on the server side exposes a publicly accessible endpoint for the service to invoke when the event is triggered. It acts as a webhook.

Web PubSub service delivers client events to the upstream webhook using the CloudEvents HTTP protocol.

For every event, the service formulates an HTTP POST request to the registered upstream and expects an HTTP response.

The data sent from the service to the server is always in CloudEvents binary format.

Diagram showing the Web PubSub service event push mode.

Upstream and Validation

Event handlers need to be registered and configured in the service through the portal or Azure CLI before first use. When a client event is triggered, the service can identify if the event must be handled or not. For public preview, we use PUSH mode to invoke the event handler. The event handler on the server side exposes publicly accessible endpoint for the service to invoke when the event is triggered. It acts as a webhook upstream.

The URL can use {event} parameter to define a URL template for the webhook handler. The service calculates the value of the webhook URL dynamically when the client request comes in. For example, when a request /client/hubs/chat comes in, with a configured event handler URL pattern http://host.com/api/{event} for hub chat, when the client connects, it will first POST to this URL: http://host.com/api/connect. This behavior can be useful when a PubSub WebSocket client sends custom events, that the event handler helps dispatch different events to different upstream. The {event} parameter isn't allowed in the URL domain name.

When setting up the event handler upstream through Azure portal or CLI, the service follows the CloudEvents abuse protection to validate the upstream webhook. The WebHook-Request-Origin request header is set to the service domain name xxx.webpubsub.azure.com, and it expects the response having header WebHook-Allowed-Origin to contain this domain name.

When doing the validation, the {event} parameter is resolved to validate. For example, when trying to set the URL to http://host.com/api/{event}, the service tries to OPTIONS a request to http://host.com/api/validate and only when the response is valid the configuration can be set successfully.

For now, we don't support WebHook-Request-Rate and WebHook-Request-Callback.

Authentication between service and webhook

  • Anonymous mode
  • Simple authentication that code is provided through the configured Webhook URL.
  • Use Azure Active Directory (Azure AD) authentication. For more information, see how to use managed identity for details.
    • Step1: Enable Identity for the Web PubSub service
    • Step2: Select from existing Azure AD application that stands for your webhook web app

Connection manager

The server is by nature an authorized user. With the help of the event handler role, the server knows the metadata of the clients, for example, connectionId and userId, so it can:

  • Close a client connection
  • Send messages to a client
  • Send messages to clients that belong to the same user
  • Add a client to a group
  • Add clients authenticated as the same user to a group
  • Remove a client from a group
  • Remove clients authenticated as the same user from a group
  • Publish messages to a group

It can also grant or revoke publish/join permissions for a PubSub client:

  • Grant publish/join permissions to some specific group or to all groups
  • Revoke publish/joinh permissions for some specific group or for all groups
  • Check if the client has permission to join or publish to some specific group or to all groups

The service provides REST APIs for the server to do connection management.

Diagram showing the Web PubSub service connection manager workflow.

The detailed REST API protocol is defined here.

Summary

You may have noticed that the event handler role handles communication from the service to the server while the manager role handles communication from the server to the service. After combining the two roles, the data flow between service and server looks similar to the following diagram using HTTP protocol.

Diagram showing the Web PubSub service bi-directional workflow.

Next steps

Follow the tutorials listed below to start building your own application.