Assembling a Batched EDI Interchange
To assemble individual transaction-set batch elements into an EDI interchange, BizTalk Server EDI and AS2 does the following:
Identifies batch elements for batching
Validates and buffers individual batch elements upon receipt
Retrieves specific batch elements and assembles a batched interchange when the release criteria is met
The start time for collection of individual messages to go into a batch is determined by the batch activation criteria. The time at which the batch is released is determined by the batch release criteria. For more information about these two sets of criteria, see Configuring an Outgoing Batch.
Message Flow for Outgoing Batched Messages
When BizTalk Server is configured to batch an outgoing message, BizTalk Server components will perform the following series of steps to prepare the batched message for sending. This series of steps describes the case in which the EDIReceive pipeline with the BatchMarker pipeline component processes the received interchanges that contain transaction sets to be batched for sending.
The BatchMarker pipeline component in the EDIReceive pipeline determines which messages need to be batched from the EDI batch filter settings in the party properties. (This is the only batching component that looks at the batch filter settings and acts upon them.)
If the filter settings of only one batch configuration subscribes to a message, the BatchMarker component will promote the property
EDI.ToBeBatched = True. This ensures that the batching orchestration will pick up the message.
If the filter settings of more than one batch configuration match the context of a message, the BatchMarker component promotes the properties
EDI.ToBeRouted = Trueand sets the
EDI.BatchIdsproperty to a space delimited list containing the matching batch IDs. This ensures that the routing orchestration will subscribe to the message.
You can promote the appropriate context properties in a custom receive pipeline or a custom orchestration. The custom receive pipeline can use the BatchMarker pipeline component, or can promote the properties without using the BatchMarker pipeline component.
The routing orchestration picks up any transaction set for which
EDI.ToBeRouted = Trueand
EDI.BatchIdsis promoted, and then creates copies of the transaction set, ensuring that there is a copy for each batch ID contained in
EDI.BatchIds. The routing orchestration sets
EDI.ToBeBatched = Trueand
EDI.BatchIdis set to the batch ID of the matching batch configuration for each copy of the transaction set. This ensures that the transaction sets will be picked up by the batching orchestration for batching.
The batching orchestration picks up all messages for which the following properties have been promoted:
EDI.ToBeBatched = Trueand EDI.BatchId = the batch id of the batch associated with this instance of the batching orchestration.
EDI.ToBeBatched = Trueand EDI.BatchName = the name of the configured batch and EDI.DestinationPartyName = the party name that contains the batch configuration.
When the incoming messages are processed by the EDIReceive pipeline (with the BatchMarker pipeline component), the batching orchestration will batch only X12- or EDIFACT-encoded transaction sets.
There will be one instance of the batching orchestration for each active batch configuration, each subscribing to a specific batch ID. The batch ID value is set automatically when creating a new batch configuration in the Identification section of the Batching Configuration page of the one-way agreement tab of the Agreement Properties dialog box.
The batching orchestration validates each transaction set to be batched. If the transaction set fails validation, it sets the
EDI.BatchItemValidationFailurecontext property to "True". The BatchSuspend orchestration picks up the message based upon that context property, posts error information, and then is suspended.
When the batch release criteria has been met, the batching orchestration assembles the batch elements into a batch and creates an envelope.
After the batching orchestration completes batching an interchange, it promotes the following properties on that interchange: EDI.DestinationPartyName = %PartyName%, EDI.BatchEncodingType = X12 or EDIFACT, and EDI.ToBeBatched = False.
A send port picks up the batched transaction sets based on EDI.DestinationPartyName = <PartyName>, EDI.BatchEncodingType = EDIFACT or X12, and EDI.ToBeBatched = False.
Batching Orchestration Control Messages
The batching orchestration is activated, terminated, or overridden by the following control messages:
BatchActivation: When the orchestration receives this message, an instance of the batching orchestration is created and the orchestration is active to receive batch elements (if it meets the batch activation criteria). This control message is sent by clicking the Start button of a batch configuration on the Batching Configuration page of the one-way agreement tab of the Agreement Properties dialog box.
BatchTermination: When the orchestration receives this message, it creates a batch from existing batch elements, publishes the message to the MessageBox, and terminates. This control message is sent by clicking the Stop button of a batch configuration on the Batching Configuration page of the one-way agreement tab of the Agreement Properties dialog box.
The orchestration is also terminated if it reaches the time specified for the End-by property in the Termination section of the Batching Configuration page of the one-way agreement tab of the Agreement Properties dialog box.
BatchOverride: When the orchestration receives this message, it creates a batch from existing elements, publishes the message to the MessageBox, and then waits for messages for the next batch. This control message is sent by clicking the Override button of a batch configuration on the Batching Configuration page of the one-way agreement tab of the Agreement Properties dialog box.
The batching orchestration receives the control messages via the BatchControlMessageRecvLoc receive location. The polling interval for this SQL receive location is set by default to 30 seconds, but can be changed in the SQL Transport Properties dialog box for the receive location. Decreasing the polling interval will ensure that the BatchControlMessageRecvLoc receive location will receive a control message soon after you perform the action that sent the control message (such as when you start the batching orchestration).
The following steps occur when you start a batch orchestration:
When you click the Start button, BizTalk Server creates a record in a table indicating which party and batch id you are activating the batch orchestration for.
The SQL adapter associated with the BatchControlMessageRecvLoc receive location polls to see if the record exists in the database.
If the record exists, the SQL adapter builds a control message, using information in the record.
Building the control message in this way ensures that the orchestration cannot be started by an invalid control message.
The BatchControlMessageRecvLoc receive location receives the control message, and BizTalk Server activates a batching orchestration instance.
BizTalk Server EDI batches XML transaction sets into EDI interchanges using the following components:
BatchMarkerReceivePipelineComponent in the EDI receive pipeline
Upgrade Batching Orchestration
EDI Send Pipeline
These components are installed as DLLs when you install and configure BizTalk Server EDI and AS2.
The batching components in BizTalk Server EDI and AS2 do not guarantee the ordering of the transaction sets in a batch.
BatchMarkerReceivePipelineComponent in the EDI receive pipeline enables the batching orchestration to pick up messages to be batched. This pipeline component is applied after the Disassembler in the EDI Receive Pipeline. The component evaluates the filter criteria set in the Filter section of the Batching Configuration page of the one-way agreement tab of the Agreement Properties dialog box, and marks the transaction sets with the following context properties for processing by the routing and batching orchestrations
If a single party subscribes to a message to be batched, it promotes
ToBeBatched = Trueand BatchId is set to the value of the batch ID of the matching batch configuration. This enables pickup by the batching orchestration.
If multiple batches subscribe to a message to be batched, it promotes
ToBeRouted = True, and sets the
EDI.BatchIdsproperty set to a space-delimited list of batch IDs. This enables pickup by the routing orchestration.
If no subscriptions exist, it does not promote a context property. This indicates that the transaction set is not to be batched.
The pipeline component ignores messages other than XML and messages with the
ReuseEnvelopeproperty (preserved batches). If acknowledgments are not to be batched, the pipeline component will ignore the ACK message types (CONTRL, TA1, and 997). To optimize processing of the routing and batching orchestrations, the BatchMarkerPipelineComponent passes the message through to the MessageBox if the message context property
MessageDestinationis set to "SuspendedQueue" by the disassembler.
If you are using a custom pipeline, rather than the EDIReceive pipeline, you can use the BatchMarker component in your custom pipeline. If you are not using the EDIReceive pipeline, and are publishing messages from an orchestration, you will have to promote
ToBeBatched = Trueand
BatchIDto the ID of an active batch in one of your components.
The routing orchestration subscribes to any message with the context property
ToBeRouted = True and the context property
EDI.BatchIds set to a space-delimited list of batch IDs. This occurs when multiple batch filters subscribe to a message to be batched. The routing orchestration will make a copy of the message for each batch ID contained in
EDI.BatchIds. It stamps each copy with two new context properties:
EDI.BatchID, which is set to the ID of the batch that this message is intended for.
EDI.ToBeBatched, which is set to True.
It then routes the copies to the MessageBox to be picked up by the batching orchestration. Each destination batch ID use a singleton instance of the same orchestration, with a filter on the specific batch ID.
The batching orchestration is a stateful service that buffers batch elements (transaction sets) over a period of time, assembles them into an interchange, and then releases the interchange to the send pipeline based upon the release criteria.
After being activated, an instance of the batching orchestration can start batching messages of a particular encoding type to a given party (if the Start datetime has passed). At any one time there can be many instances of the batching orchestration for each party, one per active batch configuration. A single instance of the batching orchestration can release multiple batches for a single batch configuration. After the end criteria is met, the batching orchestration instance will terminate. A new instance of the batching orchestration will need to be created manually from the Trading Partner Management (TPM) using the Start button.
If the batch orchestration starts before the start date time shown in the Activation section on the of the Batching Configuration page of the one-way agreement tab of the Agreement Properties dialog box, it will only receive messages that are specified in the activation range. It will not receive messages sent before the start date time.
The batching orchestration does the following:
Subscribes to XML batch elements with the context properties
EDI.ToBeBatched = Trueand
EDI.BatchIdthe ID of the batch configuration, or
EDI.ToBeBatched = Trueand EDI.BatchName = the name of the configured batch and EDI.DestinationPartyName = the party name that contains the batch configuration. It receives batch elements using a Receive action operation in a loop.
The batching orchestration does not batch transaction sets based upon the filter criteria set in the Filter section of the Batching Configuration page of the one-way agreement tab of the Agreement Properties dialog box. It subscribes to transaction sets that have the above context properties set on them. The BatchMarker pipeline component sets and promotes those context properties based upon the filter settings in the party properties.
Retrieves the batch configuration settings for the party identified in the
Validates the batch element (transaction set) based on party settings.
If there is an error in a batch element, the batching orchestration will promoted the following property on that transaction set:
EDI.BatchItemValidationFailure = True. The BatchElementSuspend orchestration subscribes to any transaction set for which this property has been promoted. This orchestration will provide detailed error information for the first error encountered in batching the interchange.
If there is no error in a batch element, holds a reference to that batch element.
When the appropriate control message is received or the batching release criteria is met, breaks out of the Receive action loop, retrieves all batch elements from the MessageBox, and assembles the interchange.
Sets the context property
ToBeBatched = Falsefor the interchange and the context property DestinationPartyName = %PartyName% where %PartyName% is the name of the party for which the message is intended.
If a send port subscribes to either or both of the properties
EDI.ToBeBatched = Falseand EDI.DestinationPartyName = %PartyName%, that send port can pick up the batched interchange. Make sure that a send port's filters are configured such that the send port picks up only those batched interchanges that it is intended to pick up.
Interchanges dropped by the batching orchestration into the MessageBox have only the properties
EDI.ToBeBatched = False, EDI.DestinationPartyName = %PartyName%, and EDI.BatchEncodingType = "X12" or "EDIFACT" promoted to the context. All context properties from the original transaction sets are lost.
For an X12-encoded interchange, applies the following properties to the envelope:
ISA6: Interchange Sender ID
ISA8: Interchange Receiver ID
ISA15: Usage indicator
ISA_Blob (written to context)
For an EDIFACT-encoded interchange, applies the following properties to the envelope:
UNB2.1: Interchange Sender ID
UNB3.1: Interchange Recipient ID
UNB2.3: Address for Reverse Routing
UNB11: Usage indicator
UNA_Blob (written to context)
UNB_Blob (written to context)
Delivers the batched interchange to the MessageBox for pickup by the EDI send pipeline.
The UpgradeBatching orchestration handles messages that are have the
EDI.ToBeBatched property set to true, but do not have the
EDI.BatchID property set.
In previous versions of BizTalk Server, each party could have only one batch configuration. When processing messages that had
EDI.ToBeBatched set to true, the
EDI.DestinationPartyId was used to determine the party and then the batch configuration was read from the agreement properties.
In BizTalk Server, each party can have multiple batch configurations associated with it, so the
EDI.DestinationPartyId does not provide enough information to determine which batch configuration should be used. When BizTalk Server receives messages, the
EDI.BatchId property is used to identify which specific batch configuration should be used when processing a message.
After upgrading to BizTalk Server, you may still have custom pipelines that use the
EDI.DestinationPartyId property to specify the party configuration. When a message is received that has
EDI.ToBeBatched set to true, and has
EDI.DestinationPartyID set instead of EDI.BatchID, the UpgradeBatching orchestration attempts to determine which batch configuration should be used.
The UpgradeBatching orchestration uses the following subscription filters to subscribe to documents that are marked for batching, but do not specify a batch ID:
When the orchestration receives a message, it will try to find a matching batch configuration for the message by using the party name and encoding type. The
EDI.DestinationPartyIDproperty is used to determine the party name, and then the orchestration looks for a batch name that matches <PartyName>+<EncodingType>+Default. For example if the party name is Contoso, and the value of
EDI.EncodingTypeis X12, then the orchestration will look for a batch named ContosoX12Default.
If a matching batch configuration is found, the message is placed back in the message box with the following properties:
EDI.ToBeBatched = True
EDI.ToBeRouted = False
EDI.BatchId = the batch ID for the matching batch
The batching orchestration will then process the message
If no matching batch is found, then the following will happen:
- The message will not be sent to the BatchSuspend orchestration.
- The UpgradeBatching orchestration instance and message will be suspended.
- An error will be logged to the event log stating that a batch was not found.
The BatchSuspend orchestration handles invalid messages received by the batching orchestration. The BatchSuspend orchestration is needed because there is no direct way to suspend a message from an orchestration (in this case, the batching orchestration) without stopping the execution of the instance of the orchestration.
When an instance of the batching orchestration receives a message, it attempts to validate it. If the message fails validation, the batching orchestration creates an instance of the BatchSuspend orchestration, and sets the
EDI.BatchItemValidationFailure context property to True. The BatchSuspend orchestration subscribes to all messages with that context property set to True. After the invalid transaction set is routed to the BatchSuspend orchestration, the BatchSuspend orchestration instance is suspended.
Detailed error information for the first error encountered is provided by the BatchSuspend orchestration.
You can create a custom orchestration to handle transaction sets that fail validation by the batching orchestration, by using the
EDI.BatchElementValidationFailure property in a filter.
EDI Send Pipeline
After receiving a batched interchange from the batching orchestration, the EDI send pipeline does the following:
For an X12-encoded interchange, the send pipeline applies the following properties to the envelope:
ISA2: Authorization Information
ISA4: Security Information
ISA9: Interchange Date
ISA10: Interchange Time
ISA13: Interchange Control Number
GS6: Group Control Number
ST2: Transaction Set Control Number
For an EDIFACT-encoded interchange, the send pipeline applies the following properties to the envelope:
UNB5: Interchange Control Reference
UNB6.1: Recipient Reference Password
UNG5: Functional Group Reference
UNG8: Application Password
Delivers the message via the associated adapter