Map custom fields to Event Grid schema

If your event data doesn't match the expected Event Grid schema, you can still use Event Grid to route event to subscribers. This article describes how to map your schema to the Event Grid schema.

Important

Using this Azure feature from PowerShell requires the AzureRM module installed. This is an older module only available for Windows PowerShell 5.1 that no longer receives new features. The Az and AzureRM modules are not compatible when installed for the same versions of PowerShell. If you need both versions:

  1. Uninstall the Az module from a PowerShell 5.1 session.
  2. Install the AzureRM module from a PowerShell 5.1 session.
  3. Download and install PowerShell Core 6.x or later.
  4. Install the Az module in a PowerShell Core session.

Install preview feature

This feature is in preview. To use it, you must install a preview extension or module.

Install extension for Azure CLI

For Azure CLI, you need the Event Grid extension.

In CloudShell:

  • If you've installed the extension previously, update it az extension update -n eventgrid
  • If you haven't installed the extension previously, install it az extension add -n eventgrid

For a local installation:

  1. Install the Azure CLI. Make sure that you have the latest version, by checking with az --version.
  2. Uninstall previous versions of the extension az extension remove -n eventgrid
  3. Install the eventgrid extension with az extension add -n eventgrid

Install module for PowerShell

For PowerShell, you need the AzureRM.EventGrid module.

In CloudShell:

  • Install the module Install-Module -Name AzureRM.EventGrid -AllowPrerelease -Force -Repository PSGallery

For a local installation:

  1. Open PowerShell console as administrator
  2. Install the module Install-Module -Name AzureRM.EventGrid -AllowPrerelease -Force -Repository PSGallery

If the -AllowPrerelease parameter isn't available, use the following steps:

  1. Run Install-Module PowerShellGet -Force
  2. Run Update-Module PowerShellGet
  3. Close the PowerShell console
  4. Restart PowerShell as administrator
  5. Install the module Install-Module -Name AzureRM.EventGrid -AllowPrerelease -Force -Repository PSGallery

Original event schema

Let's suppose you have an application that sends events in the following format:

[
  {
    "myEventTypeField":"Created",
    "resource":"Users/example/Messages/1000",
    "resourceData":{"someDataField1":"SomeDataFieldValue"}
  }
]

Although that format doesn't match the required schema, Event Grid enables you to map your fields to the schema. Or, you can receive the values in the original schema.

Create custom topic with mapped fields

When creating a custom topic, specify how to map fields from your original event to the event grid schema. There are three values you use to customize the mapping:

  • The input schema value specifies the type of schema. The available options are CloudEvents schema, custom event schema, or Event Grid schema. The default value is Event Grid schema. When creating custom mapping between your schema and the event grid schema, use custom event schema. When events are in the CloudEvents schema, use Cloudevents schema.

  • The mapping default values property specifies default values for fields in the Event Grid schema. You can set default values for subject, eventtype, and dataversion. Typically, you use this parameter when your custom schema doesn't include a field that corresponds to one of those three fields. For example, you can specify that data version is always set to 1.0.

  • The mapping fields value maps fields from your schema to the event grid schema. You specify values in space-separated key/value pairs. For the key name, use the name of the event grid field. For the value, use the name of your field. You can use key names for id, topic, eventtime, subject, eventtype, and dataversion.

To create a custom topic with Azure CLI, use:

# If you have not already installed the extension, do it now.
# This extension is required for preview features.
az extension add --name eventgrid

az eventgrid topic create \
  -n demotopic \
  -l eastus2 \
  -g myResourceGroup \
  --input-schema customeventschema \
  --input-mapping-fields eventType=myEventTypeField \
  --input-mapping-default-values subject=DefaultSubject dataVersion=1.0

For PowerShell, use:

# If you have not already installed the module, do it now.
# This module is required for preview features.
Install-Module -Name AzureRM.EventGrid -AllowPrerelease -Force -Repository PSGallery

New-AzureRmEventGridTopic `
  -ResourceGroupName myResourceGroup `
  -Name demotopic `
  -Location eastus2 `
  -InputSchema CustomEventSchema `
  -InputMappingField @{eventType="myEventTypeField"} `
  -InputMappingDefaultValue @{subject="DefaultSubject"; dataVersion="1.0" }

Subscribe to event grid topic

When subscribing to the custom topic, you specify the schema you would like to use for receiving the events. You specify the CloudEvents schema, custom event schema, or Event Grid schema. The default value is Event Grid schema.

The following example subscribes to an event grid topic and uses the Event Grid schema. For Azure CLI, use:

topicid=$(az eventgrid topic show --name demoTopic -g myResourceGroup --query id --output tsv)

az eventgrid event-subscription create \
  --source-resource-id $topicid \
  --name eventsub1 \
  --event-delivery-schema eventgridschema \
  --endpoint <endpoint_URL>

The next example uses the input schema of the event:

az eventgrid event-subscription create \
  --source-resource-id $topicid \
  --name eventsub2 \
  --event-delivery-schema custominputschema \
  --endpoint <endpoint_URL>

The following example subscribes to an event grid topic and uses the Event Grid schema. For PowerShell, use:

$topicid = (Get-AzureRmEventGridTopic -ResourceGroupName myResourceGroup -Name demoTopic).Id

New-AzureRmEventGridSubscription `
  -ResourceId $topicid `
  -EventSubscriptionName eventsub1 `
  -EndpointType webhook `
  -Endpoint <endpoint-url> `
  -DeliverySchema EventGridSchema

The next example uses the input schema of the event:

New-AzureRmEventGridSubscription `
  -ResourceId $topicid `
  -EventSubscriptionName eventsub2 `
  -EndpointType webhook `
  -Endpoint <endpoint-url> `
  -DeliverySchema CustomInputSchema

Publish event to topic

You're now ready to send an event to the custom topic, and see the result of the mapping. The following script to post an event in the example schema:

For Azure CLI, use:

endpoint=$(az eventgrid topic show --name demotopic -g myResourceGroup --query "endpoint" --output tsv)
key=$(az eventgrid topic key list --name demotopic -g myResourceGroup --query "key1" --output tsv)

event='[ { "myEventTypeField":"Created", "resource":"Users/example/Messages/1000", "resourceData":{"someDataField1":"SomeDataFieldValue"} } ]'

curl -X POST -H "aeg-sas-key: $key" -d "$event" $endpoint

For PowerShell, use:

$endpoint = (Get-AzureRmEventGridTopic -ResourceGroupName myResourceGroup -Name demotopic).Endpoint
$keys = Get-AzureRmEventGridTopicKey -ResourceGroupName myResourceGroup -Name demotopic

$htbody = @{
    myEventTypeField="Created"
    resource="Users/example/Messages/1000"
    resourceData= @{
        someDataField1="SomeDataFieldValue"
    }
}

$body = "["+(ConvertTo-Json $htbody)+"]"
Invoke-WebRequest -Uri $endpoint -Method POST -Body $body -Headers @{"aeg-sas-key" = $keys.Key1}

Now, look at your WebHook endpoint. The two subscriptions delivered events in different schemas.

The first subscription used event grid schema. The format of the delivered event is:

{
  "id": "aa5b8e2a-1235-4032-be8f-5223395b9eae",
  "eventTime": "2018-11-07T23:59:14.7997564Z",
  "eventType": "Created",
  "dataVersion": "1.0",
  "metadataVersion": "1",
  "topic": "/subscriptions/<subscription-id>/resourceGroups/myResourceGroup/providers/Microsoft.EventGrid/topics/demotopic",
  "subject": "DefaultSubject",
  "data": {
    "myEventTypeField": "Created",
    "resource": "Users/example/Messages/1000",
    "resourceData": {
      "someDataField1": "SomeDataFieldValue"
    }
  }
}

These fields contain the mappings from the custom topic. myEventTypeField is mapped to EventType. The default values for DataVersion and Subject are used. The Data object contains the original event schema fields.

The second subscription used the input event schema. The format of the delivered event is:

{
  "myEventTypeField": "Created",
  "resource": "Users/example/Messages/1000",
  "resourceData": {
    "someDataField1": "SomeDataFieldValue"
  }
}

Notice that the original fields were delivered.

Next steps