Edit

Share via

Call out to a gRPC endpoint from an Azure IoT Data Processor Preview pipeline

  • Article

Important

Azure IoT Operations Preview – enabled by Azure Arc is currently in PREVIEW. You shouldn't use this preview software in production environments.

See the Supplemental Terms of Use for Microsoft Azure Previews for legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

Use the gRPC call out stage to call an external gRPC with an optional request body and receive an optional response. The call out stage lets you incorporate custom data processing logic, such as running machine learning models, into the pipeline processing.

  • Each partition in a pipeline independently executes the API calls in parallel.
  • API calls are synchronous, the stage waits for the call to return before continuing with further pipeline processing.
  • Currently, the stage only supports the Unary RPC type.
  • gRPC call out can only be used with the Protobuf format. You must use the Protobuf with the gRPC call out stage.

Prerequisites

To configure and use a gRPC cal lout pipeline stage, you need:

  • A deployed instance of Azure IoT Data Processor Preview.
  • A gRPC server that's accessible from the Data Processor instance.
  • The protoc tool to generate the descriptor.

Configure a gRPC call out stage

The gRPC call out stage JSON configuration defines the details of the stage. To author the stage, you can either interact with the form-based UI, or provide the JSON configuration on the Advanced tab.

Name Type Description Required Default Example
Name string A name to show in the Data Processor UI. Yes - MLCall2
Description string A user-friendly description of what the call out stage does. No Call ML endpoint 2
Server address string The gRPC server address. Yes - https://localhost:1313
RPC name string The RPC name to call Yes - GetInsights
Descriptor1 string The base 64 encoded descriptor. Yes - CuIFChxnb29nb
Authentication string The authentication type to use. None/Metadata. Yes None None
Metadata key string The metadata key to use when Authentication is set to Metadata. No authorization authorization
Secret string The secret reference to use when Authentication is set to Metadata. No - mysecret
Enable TLS boolean Whether to enable TLS. Data Processor currently supports TLS based authentication with public certificate. No false true
API request > Body path Path The path to the portion of the Data Processor message that should be serialized and set as the request body. Leave empty if you don't need to send a request body. No - .payload.gRPCRequest
API request > Metadata > Key2 Static/Dynamic field The metadata key to set in the request. No Static/Dynamic field
API request > Metadata > Value2 Static/Dynamic field The metadata value to set in the request. No Static/Dynamic field
API response > Body path Path The Path to the property in the outgoing message to store the response in. Leave empty if you don't need the response body. No - .payload.gRPCResponse
API Response > Metadata Path The Path to the property in the outgoing message to store the response metadata in. Leave empty if you don't need the response metadata. No - .payload.gRPCResponseHeader
API Response > Status Path The Path to the property in the outgoing message to store the response status in. Leave empty if you don't need the response status. No - .payload.gRPCResponseStatus

1Descriptor: Because the gRPC call out stage only supports the protobuf format, you use the same format definitions for both request and response. To serialize the request body and deserialize the response body, you need a base 64 encoded descriptor of the .proto file.

Use the following command to generate the descriptor, replace <proto-file> with the name of your .proto file:

protoc --descriptor_set_out=/dev/stdout --include_imports <proto-file> | base64 | tr '\n' ' ' | sed 's/[[:space:]]//g'

Use the output from the previous command as the descriptor in the configuration.

2API request > Metadata: Each element in the metadata array is a key value pair. You can set the key or value dynamically based on the content of the incoming message or as a static string.

Related content