How to use Duplex MEP to communicate with BizTalk from a .NET application or a WF workflow running inside AppFabric Part 1
Some time ago, I was asked by a customer whether or not BizTalk WCF Adapters support the Duplex Message Exchange Pattern. The latter is characterized by the ability of both the service and the client to send messages to each other independently either using one-way or request/reply messaging. This form of bi-directional communication is useful for services that must communicate directly to the client or for providing an asynchronous experience to either side of a message exchange, including event-like behavior. The BizTalk WCF Adapters support this message exchange pattern, but unfortunately this feature is undocumented. Therefore, I decided to create a full demo where a client .NET application exchange messages with an Orchestration via a two-way Request-Response WCF Receive Location that supports callbacks and duplex communication. Subsequently, I decided to extend this demo and introduce a WCF Workflow Service between the WinForm client application and the underlying Orchestration to implement a more complex scenario The final objective was to investigate how exploiting the Correlation mechanisms provided by WF 4.0 to get a workflow running within IIS/AppFabric to exchange messages with a downstream orchestration in an asynchronous mode.
This article will be composed of 3 parts:
- Part 1: How to exchange messages with an orchestration via a two-way WCF Receive Location using the Duplex Message Exchange Pattern.
- Part 2: How to implement an asynchronous communication between a client application and a WCF Workflow Service running within IIS\AppFabric Hosting Services using the Durable Duplex Correlation.
- Part 3: How to implement an asynchronous communication between a WCF Workflow Service and an Orchestration using WS-Addressing and Content-Based Correlation.
Why Duplex Message Exchange Pattern?
The Duplex Message Exchange Pattern is fully supported by WCF and is extensively documented on MSDN. As I already mentioned in the introduction, the Duplex pattern allows two applications to act both as service endpoints and send messages to each other independently in an asynchronous way. There are many reasons for using the Duplex Message Exchange Pattern when communicating with a BizTalk solution:
- If an orchestration invoked by the consumer application takes seconds to minutes to complete.
- If you are invoking an orchestration within an ASP.NET page, use asynchronous pages.
- If you are invoking an orchestration from a single threaded application, such as a Windows Forms or Windows Presentation Foundation (WPF) application. When using the event-based asynchronous calling model, the result event is raised on the UI thread, adding responsiveness to the application without requiring you to handle multiple threads yourself.
In general, if you have a choice between a synchronous and asynchronous call, you should choose the latter approach. In fact, a synchronous call typically blocks the client thread till the operation completes, whereas an asynchronous call is non-blocking and only initiates the operation. This way the consumer application can continue its execution without waiting for the call to complete and get notified by the service when the result of its request is ready.
For more information on how implementing the Duplex Message Exchange Pattern with WCF, you can read the following articles:
- " Synchronous and Asynchronous Operations " topic on MSDN.
- " Designing Service Contracts " topic on MSDN.
- “ How to: Create a Duplex Contract ” topic on MSDN.
- “ Duplex Services ” topic on MSDN.
- “Choosing a Message Exchange Pattern” topic on MSDN.
Using a Duplex Channel to communicate with a BizTalk Application
The following picture represents the architecture of the first use case. The idea behind the application is quite straightforward: a Windows Forms application submits a question to an orchestration called SyncMagic8Ball via a WCF-NetTcp Receive Location. The orchestration is a BizTalk version of the notorious Magic 8 Ball toy and it randomly returns one of 20 standardized answers.
- The Windows Forms Client Application enables a user to specify a question and a delay in seconds. When the user presses the Ask button, a new request message containing the question and the delay is created and sent to a Two-Way WCF-NetTcp Receive Location.
- The XmlReceive pipeline promotes the MessageType context property.
- The Message Agent submits the request message to the MessageBox (BizTalkMsgBoxDb).
- A new instance of the SyncMagic8Ball orchestration receives the request message via a two-way logical port and uses a custom helper component called XPathHelper to read the value of the Question and Delay elements from the inbound message.
- The SyncMagic8Ball orchestration invokes the SetResponse static method exposed by the ResponseHelper class to build the response message containing the answer to this question contained in the request message. The response message is then published to the MessageBox (BizTalkMsgBoxDb) by the Message Agent.
- The response message is retrieved by the WCF-NetTcp Receive Location.
- The PassThruTransmit send pipeline is executed by the WCF-NetTcp Receive Location.
- The response message is finally returned to the original caller.
WCF Receive Locations
In order to establish an asynchronous communication, the client application and service, in our case BizTalk Server, have to meet the following conditions:
- They need to be online simultaneously.
- They need to use the same WCF binding.
- The selected binding must support sessions and duplex contracts. WCF provides a rich set of built-in bindings that support the Duplex MEP. For more information, you can read “System-Provided Bindings” topic on MSDN.
- The client application needs to implement a Service Contract to submit the request message to BizTalk and expose a Callback Contract to receive the response message from BizTalk Server asynchronously.
To satisfy the second and third conditions, I have created 3 different Two-Way Request-Response Receive Locations with the following characteristics:
- The 3 Receive Locations are children of the same Receive Port called DuplexMEP.Sync.ReceivePort.
- They use the XmlReceive pipeline to process the inbound message and the PassThruTrasmit pipeline to process the outbound message.
- The DuplexMEP.Sync.WCF-NetTcp.ReceiveLocation uses the WCF-NetTcp Adapter and is hosted by the BizTalkServerReceiveHost64 in-process host.
- The DuplexMEP.Sync.WCF-NetNamedPipe.ReceiveLocation uses the WCF-NetNamedPipe Adapter and is hosted by the BizTalkServerReceiveHost64 in-process host.
- The DuplexMEP.Sync.WCF-Custom.WsDualHttpBinding.ReceiveLocation uses the WCF-CustomIsolated Adapter in conjunction with the WsDualHttpBinding and is hosted by the BizTalkServerIsolatedHost.
The following picture shows the 3 Receive Locations inside the BizTalk Administration Console:
| Note You should select the most appropriate WCF Adapter and binding based on your needs:
Bindings have different characteristics in terms of response time and throughput, so the general advice to increase performance is using the NetTcpBinding and NetNamedPipeBinding whenever possible.
The request messages sent by the Windows Forms Client Application have the following format:
The corresponding response messages returned by the SyncMagic8Ball orchestration have the following format:
Both message types are defined by an XML Schema contained in the Schemas project that you can find in the companion code for this article.
The following picture shows the structure of the SyncMagic8Ball orchestration.
As you can easily notice, the orchestration uses the Two-Way Request-Response Logical Port to receive the inbound request message and to return the corresponding response message. The Trace Request Expression Shape contains the following code to extract the information from the request message. The namespace of the LogHelper and XPathHelper static classes have been eliminated for ease of reading.
You can use DebugView, as shown in the picture below, to monitor the trace produced by the orchestration and helper components.
Note My LogHelper class traces messages to the standard output using the capability supplied by the Trace class. This component is primarily intended to be used for debugging a BizTalk application in a test environment, rather than to be used in a production environment. If you are looking for a tracing framework which combines the high performance and flexibility provided by the Event Tracing for Windows (ETW) infrastructure, you can read the following whitepaper by Valery Mizonov:
The value of the Delay Shape is defined as follows:
Therefore, the orchestration waits for the time interval in seconds specified in the request message before returning the response message to the caller.
Note To extract the value of individual fields from the inbound document, I could have used Distinguished Fields defined on the XML Schema for the request message. These are context properties whose name is defined by the XPath Expression used to retrieve data from an input XML document. If the XML Schema of the document is quite complex, the XPath Expression in question can be very long and therefore the corresponding Distinguished Field can occupy a significant amount of space in the message context. Therefore, I recommend to implement the following best practices in any BizTalk application:
When the document in question is extremely small, you can also load the message into an XmlDocument object and use the SelectSingleNode method to extract the information you need using an XPath Expression. However, the general advice is to minimize the usage of XmlDocument variables in orchestrations and in .NET code. Loading a message into an XmlDocument variable has significant overhead, especially for large messages. This overhead is in terms of memory usage and system resources required to build the in-memory structures. The use of an XmlDocument instance forces the message content to be entirely loaded into memory in order to build the object graph for the DOM. The total amount of memory used by an instance of this class can be around 10 times the actual message size. For more information and evidence, see the following articles I wrote on this topic:
The request message in my demo is relatively small; nevertheless, I decided to use an helper class that adopts a streaming approach to extract data from the XLANGMessage passed in as argument:
The Set Response Message Assignment shape invokes the SetResponse static method exposed by the ResponseHelper class to generate the response message:
As you can see, the ResponseHelper class uses a VirtualStream object an XmlWriter class instance to initialize the content of the response message. Once again, the response message in this case is so small that I could have used an XmlDocument object to load the content of the response message. However, my intention was to show you how to take advantage of the streaming approach to read and write the content of an XLANGMessage object within an orchestration.
Let’s take a look at the code used by the Windows Forms Client Application to send a request message and receive the corresponding response using a Callback contract.
Data and Message Contracts
I started defining the Data and Message Contracts for the request and response messages. These 2 types of contracts have different roles in WCF:
Data Contracts provide a mechanism to map .NET CLR types that are defined in code and XML Schemas (XSD) defined by the W3C organization (www.w3c.org). Data contracts are published in the service’s metadata, allowing clients to convert the neutral, technology-agnostic representation of the data types to their native representations.
- Message Contracts describe the structure of SOAP messages sent to and from a service and enable you to inspect and control most of the details in the SOAP header and body. Whereas data contracts enable interoperability through the XML Schema Definition (XSD) standard, message contracts enable you to interoperate with any system that communicates through SOAP. Using message contracts gives you complete control over the SOAP message sent to and from a service by providing access to the SOAP headers and bodies directly. This allows the use of simple or complex types to define the exact content of the SOAP parts.
For more information on Data and Message Contracts, you can read the following articles:
In my solution, I created 2 separate projects called DataContracts and MessageContracts respectively. For your convenience, I included below the code of the 4 classes used to define the Data and Message Contract of the request and response messages.
Indeed, I could have used just Data Contracts to model messages as Message Contracts add a degree of complexity. So why I decided to use Message Contracts? Well, the reason is quite straightforward. By assigning false to the IsWrapped property exposed by the MessageContractAttribute , you specify that the message body won’t be contained in a wrapper element. Typically, the wrapper element of a request message is the name of the operation invoked and it’s defined in the WSDL. Setting the value of the IsWrapped property to false, you can simply select the Body option in both the Inbound BizTalk message body and Outbound WCF message body sections on the Messages tab when configuring a WCF Receive Location. On the contrary, you should define a Path in the Inbound BizTalk message body section to extract the payload from the inbound message, and specify a template in the Outbound WCF message body section to include the outgoing response message within a wrapper element.
|Note The namespace of the Message and Data Contract classes match those defined by the XML Schemas that model the request and response messages in the BizTalk Server application.|
The next step was to define the Service Contracts used by the client application to exchange messages with BizTalk Server. To this purpose, I created a new project in my solution called ServiceContracts, and then I started off by defining the Service Contract Interface that models the server side of the duplex contract. I declared the signature of a One-Way method called AskQuestion. I specified a parameter of type BizTalkRequestMessage (see the Data and Message Contracts section) and void as return type. Then I decorated the method with the XmlSerializerFormatAttribute to indicate to the WCF Runtime to use the XmlSerializer instead of the DataContractSerializer. Please note that the WCF Adapters and BizTalk in general use the XmlSerializer, thus I had to explicitly set the correct serializer in the contract definition. Next, I decorated the method with the OperationContractAttribute to indicate that the method is one way and to specify the WS-Addressing Action of the request message. Finally I decorated the interface with the ServiceContractAttribute.
Then I defined the the callback interface as follows:
Finally, I linked the two interfaces into a Duplex Contract by assigning the type of the callback interface to the CallbackContract property in the ServiceContractAttribute that decorates the IMagic8BallBizTalk interface.
Callback Contract Implementation
The next step was to implement the IMagic8BallBizTalkCallback callback interface in order to receive response messages from BizTalk Server. To this purpose I created a new class in the Client project called Magic8BallBizTalkCallback that implements the IMagic8BallBizTalkCallback callback interface.
In particular, when the WCF Receive Location invokes the AskQuestionResponse method to return a response message, the callback is handled by a ThreadPool thread other than the main thread running the client application. Since the InternalWriteToLog method is used to write the answer on a ListBox control owned by the main thread, the WriteToLog method exposed by the MainForm class uses the InvokeRequired property to check whether it's necessary to use the Invoke method because the caller is on a different thread than the main thread.
Invoking BizTalk Server
The next step was to write the code to invoke one of the 3 WCF Receive Locations exposed by by BizTalk application.
As you may have noticed, the method btnAsk_Click performs the following actions in order:
- Creates a data contract object (BizTalkRequest)
- Creates a message contract object (BizTalkRequestMessage).
- Creates a new instance of the Magic8BallBizTalkCallback type to handle callbacks.
- Creates a new InstanceContext object. In particular, the Magic8BallBizTalkCallback object is passed in as argument to the constructor of the InstanceContext object.
- Instantiates a DuplexChannelFactory to create a duplex channel of type IMagic8BallBizTalk. In particular, the first argument specifies the InstanceContext that the client uses to listen for messages from the connected service, in our case BizTalk Server.
- Creates a new duplex channel.
- Invokes the AskQuestion method.
The final step was to define the client endpoints in the client configuration file in order to invoke the 3 WCF Receive Locations exposed by the BizTalk application.
We are now ready to test the solution.
Testing the Solution
To test the application, you can proceed as follows:
Configure and start the DuplexMEP BizTalk application.
Open a new instance of the Client Application, as indicated in the picture below.
Enter an existential question like "Why am I here?", "What's the meaning of like?" or "Will the world end in 2012?" in the Question textbox.
Select one of the following endpoints:
- NetTcpEndpointBizTalk to invoke the DuplexMEP.Sync.WCF-NetTcp.ReceiveLocation.
- NetNamedPipeEndpointBizTalk to invoke the DuplexMEP.Sync.WCF-NetNamedPipe.ReceiveLocation.
- WsDualHttpEndpointBiztalk to invoke the DuplexMEP.Sync.WCF-Custom.WsDualHttpBinding.ReceiveLocation.
Specify a Delay in seconds in the corresponding textbox.
Press the Ask button.
Now, if you press the Ask button multiple times in a row, you can easily notice that the client application is called back by BizTalk in an asynchronous way. Therefore, the client application doesn't need to wait for the response to the previous question before posing a new one.
In order for the demo to work, it’s important that the Delay in seconds that you specify in the client application is less than the ReceiveTimeout configured in the WCF Receive Location. When using a WCF Adapter other than the WCF-Custom and WCF-CustomIsolated, you cannot specify a custom ReceiveTimeout. The following picture shows the Binding tab of a WCF-NetTcp Receive Location.
As a consequence, the default 10 minutes timeout will be used. Therefore, if you need to increase the ReceiveTimeout beyond 10 minutes, you can replace the WCF Adapter in question with the WCF-Custom and WCF-CustomIsolated adapter and select the same binding. This way, you can specify a custom value for the ReceiveTimeout property exposed by the binding, as shown in the following picture:
In this article we have seen how to exchange messages with an orchestration via a two-way WCF Receive Location using the Duplex Message Exchange Pattern. In the next article of the series, we’ll see how to implement an asynchronous communication between our client application and a WCF Workflow Service running within IIS\AppFabric Hosting Services using the Durable Duplex Correlation. In the meantime, here you can download the companion code for this article. As always, you feedbacks are more than welcome!