BizTalk 2004 -> Aggregator Pattern Using a Map and Orchestration. Also a Content Based Router Pattern

Below is a method to aggregate many incoming messages into one message using a Map inside of a BizTalk Orchestration.

This Aggregator pattern is the third demo of a webcast I did on March 28 2005 called: BizTalk Server 2004 Implementing Enterprise Integration Patterns. It can be viewed HERE as a recorded Webcast. The full BizTalk solution can be downloaded at the end of this blog.

In this particular scenario, the messages split from the Splitter orchestration have been processed by a Production Order application. These processed messages
now have to be aggregated back into one complete message.

The three abbreviated incoming split messages to be aggregated look something like:

<Rolls UniqueOrderID="10049" IdMill="Mill_One" OrderCount="3" UniqueIdentifier="7e96e060-3685-48bb-8113-9ae9957d8c4b">
  <trk_unit_roll_ageable trk_unit_id="10003" pro_product_id="10031" sch_prod_order_id="10049"/>
  <trk_unit_roll_ageable trk_unit_id="10006" pro_product_id="10022" sch_prod_order_id="10049"/>   
</Rolls>

<Rolls UniqueOrderID="10043" IdMill="Mill_One" OrderCount="3" UniqueIdentifier="7e96e060-3685-48bb-8113-9ae9957d8c4b">
  <trk_unit_roll_ageable trk_unit_id="10004" pro_product_id="10024" sch_prod_order_id="10043"/>
  <trk_unit_roll_ageable trk_unit_id="10005" pro_product_id="10022" sch_prod_order_id="10043"/>
  <trk_unit_roll_ageable trk_unit_id="10008" pro_product_id="10022" sch_prod_order_id="10043"/>     
</Rolls>

<Rolls UniqueOrderID="10048" IdMill="Mill_One" OrderCount="3" UniqueIdentifier="7e96e060-3685-48bb-8113-9ae9957d8c4b">
   <trk_unit_roll_ageable trk_unit_id="10007" pro_product_id="10022" sch_prod_order_id="10048"/>     
</Rolls>


The final aggregated message to construct looks something like :

<Rolls IdMill="Mill_One">
  <trk_unit_roll_ageable trk_unit_id="10003" pro_product_id="10031" sch_prod_order_id="10049"/>
  <trk_unit_roll_ageable trk_unit_id="10006" pro_product_id="10022" sch_prod_order_id="10049"/>
  <trk_unit_roll_ageable trk_unit_id="10004" pro_product_id="10024" sch_prod_order_id="10043"/>
  <trk_unit_roll_ageable trk_unit_id="10005" pro_product_id="10022" sch_prod_order_id="10043"/>
  <trk_unit_roll_ageable trk_unit_id="10008" pro_product_id="10022" sch_prod_order_id="10043"/>
  <trk_unit_roll_ageable trk_unit_id="10007" pro_product_id="10022" sch_prod_order_id="10048"/>     
</Rolls> 


One solution to accomplish the above is discussed below :

1) Create an Orchestration to accept the incoming messages to be aggregated.

In the Orchestration:

a) The first Receive Shape initializes a Correlation Set to aid in the aggregation.
A looping sequential convoy pattern is used in the Orchestration to receive the incoming messages.The first message received by the orchestration is interrogated to determine the expected number of  messages to receive. This attribute can be seen on the message such as : OrderCount="3". Therefore in this case loop
three times in the orchestration, because three messages are expected. Another attribute on the first incoming message is also interrogated such as : 
UniqueIdentifier="7e96e060-3685-48bb-8113-9ae9957d8c4b". This value will be used to correlate the three incoming messages, into the correct running instance of the same orchestration. After the first message is received, only messages with  UniqueIdentifier="7e96e060-3685-48bb-8113-9ae9957d8c4b" will be accepted. If another messages comes in with a different identifier such as :  UniqueIdentifier="D9E9DF5E-0071-4ddd-962D-FD4478E4C6FE", then another instance of the Aggregator Orchestration will be created that just accepts messages with that identifier.
Note: The attributes OrderCount and UniqueIdentifier were added to the split messages by the Splitter Orchestration.


b) Three messages are declared in the Orchestration:

i)   msgInternalRollsAll (This messages will be the final aggregated message sent out). 
ii)  msgInternalRollsAllTemp (This message is of the same type as above, but is used to help in processing, discussed below)
iii) msgInternalRollsOneOrderID (This message represents each of the separate incoming messages to be aggregated)

c) In the orchestration create a valid instance of the msgInternalRollsAll message. This is so the mapping (discussed below) will execute correctly the first time around in the loop.
The below is done inside an expression shape.

varXMLDomForMsgInternalRolls.LoadXml(@"<ns0:Rolls IdMill=""IdMill_0"" xmlns:ns0=""http://RollsInternal"" />") ;
construct msgInternalRollsAll
{
   msgInternalRollsAll = varXMLDomForMsgInternalRolls;
}

d) Use a loop shape. The number of iterations for the loop is controlled by the attribute of the first incoming
message such as : OrderCount="3"  

e) In the loop, assign the msgInternalRollsAllTemp to the msgInternalRollsAll message.
This is done so we can keep appending to the final output message as each loop executes.
The is accomplished inside an expression shape of the orchestration.

construct msgInternalRollsAllTemp
{
   msgInternalRollsAllTemp = msgInternalRollsAll;
}    
 
f) In the loop invoke a map that will keep appending to message -> msgInternalRollsAll:

The Destination message for the map is msgInternalRollsAll. A map with two inputs (two xml messages) is used to create the output message.
The first source for the map is the msgInternalRollsOneOrderID message that contains the contents  of each incoming split message to be appended to the final outgoing message   -> msgInternalRollsAll. The second source to the map is the msgInternalRollsAllTemp message. This is an interim message that is used to keep a copy of the final output message (msgInternalRollsAll) for each iteration of the loop. The map will then take the contents of two messages (nodes) and combine them into one node.
A looping functoid is used to combine the contents of two nodes into the final source node.
A more detailed explanation of this method can be found HERE.
 
g) Use a decide shape to determine if more messages are expected.
If more messages are expected, use a receive shape to receive the next incoming split message, then go to the top of the loop to append this split message to the final output message. If no more messages are expected then the final output message (msgInternalRollsAll) is sent to the
correct destination using a Content Based Router Pattern as below.


Content Based Router Pattern
----------------------------------

Initially a message came into the system from a particular Party. In the demo, messages can be received from three different parties : Mill One, Mill Two, Mill Three.
These messages were then Normalized, Split, Aggregated and now the final message must be routed back to the Mill that originally sent the message. Therefore the outgoing message contains some information about where it is going. In this case there is a attribute in the outgoing message such as :
IdMill="Mill_One". This value in the attribute will be used to dynamically route the message to the correct party. Therefore to implement this in a BizTalk Solution, Role Links in an Orchestration and Parties created in BizTalk Explorer were used. In the orchestration the following code in a expression shape is used to route the
message to the correct party:

varStrMillToSendTo = msgInternalRollsAll.IdMill;
RoleLinkToCorrectMill(Microsoft.XLANGs.BaseTypes.DestinationParty) = new Microsoft.XLANGs.BaseTypes.Party(varStrMillToSendTo, "OrganizationName");

If you have not used Role Links and Parties before, go to HERE and HERE for a more detailed explanation.

Again the full sample can be downloaded HERE . Read the ReadMe.txt file in the zip file, before unzipping and installing.

More Aggregator Patterns
------------------------------

Note: There are multiple ways to implement an Aggregator Pattern in BizTalk.
Another method includes using XML Document type .Net objects to build up the message in the orchestration:
Please see HERE and HERE for other examples.