Dynamically Configuring Pipeline Component Properties in a BizTalk Orchestration

This entry discusses the use of the following to programmatically alter a receive or send pipeline component(s) properties:

BTS.SendPipelineConfig
BTS.SendPipelineResponseConfig
BTS.ReceivePipelineConfig
BTS.ReceivePipelineResponseConfig

There is a link at the end of this entry to download a couple of examples.

Before discussing the above properties, two BizTalk 2006 pipeline enhancements are:

1) The execution of send and receive pipelines in orchestrations. This feature allows a receive or send pipeline to be executed without the use of a physical receive or send port.
2) Per Instance Pipeline Configuration using the BizTalk 2006 Administration Console. This allows changing the properties of receive and send pipeline components that
are set in the various stages of receive and send pipelines. The receive and send pipelines are configured to be invoked in physical receive and send ports. For example, as below the MIME/SMIME decoder pipeline component configuration of a receive pipeline can be altered on a per instance basis using the BizTalk 2006 Administration tool.

Both 1) and 2) are extremely useful features new for BizTalk 2006.

An additional feature that would be useful (in some cases) would be to alter a pipelines component property values in an orchestration.

As below the following properties may be set for a message in an orchestration.

Property                                                            Type of Port 

BTS.SendPipelineConfig                                Send Pipeline Configuration for a Two Way or One Way Send Port 
BTS.SendPipelineResponseConfig                    Receive Pipeline Configuration for a Two Way Send Port

BTS.ReceivePipelineConfig                            Receive Pipeline For a Two or One Way Receive Port
BTS.ReceivePipelineResponseConfig                Send Pipeline for a Two way Receive Port

For the below receive pipeline configured with a MIME/SMIME decoder:

The below XML is used internally by BizTalk to store the above configuration:

The above XML configuration can then be modified using a statement as below in a message assignment shape in an orchestration:

When the above altered message is sent through a physical port that is set to use a receive pipeline configured  with the MIME/SMIME component (in the decode stage):
The new configuration for the MIME/SMIME component (programmatically set in the orchestration) will then temporarily override the hardcoded configuration or the per-instance configuration pipelines configuration.

Ideally this is useful to cut down on the number of pipelines or physical send or receive ports that have to be created.

In order to use this method you can do the following:

1) Create receive or send pipelines set with the appropriate pipeline components. Deploy the pipelines. 
Note: You do not have to configure the pipeline components properties.
Note: You can also use the out of the box XML Receive or Send pipelines.

2) Create physical ports that are set to use the pipelines from 1).

3) Create and deploy orchestrations that temporarily override the hardcoded or per-instance send or receive ports pipeline configuration as was illustrated in an above graphic.
Logical ports in the orchestration are needed to send or receive the messages from an orchestration. The logical ports are then bound to physical ports. The configuration set in the orchestration will then temporarily override the configuration of the physical ports.

Note: To get a sample of the configuration for a pipeline. Do the below:
Using a tool such as Microsoft Sql Server Management Studio, query/open and copy the following values for a port created in 2), from a row in the following table.column in the BizTalkMgmtDB database:

bts_sendport.nvcSendPipelineData
bts_sendport.nvcReceivePipelineData
adm_receivelocation.ReceivePipelineData
adm_receivelocation.SendPipelineData

Note: If the column/row for a port is null, using the BizTalk Console, edit the receive or send ports pipeline configuration so that it overrides the default configuration. The configuration for the pipeline component(s) should then be set in one of the above table.column combinations. You can then revert back to the default configuration, by choosing the passthrough pipeline and then again choosing the original pipeline.
 
Note: Do not modify the contents of the rows/columns directly in the BizTalkMgmtDB database.
Note. The BizTalk Object Model could also be used to retrieve the configurations.
Note: The configuration for the pipeline can be stored and then retrieved from a configuration store (i.e. config file, SSO, database etc) and then set in the orchestration.

4) Bind the logical ports in the deployed orchestrations from 3) to physical ports from 2).

Good Things
1) This is a technique to cut down on the number of pipeline components and/or physical ports.
For example if the same XML received message was to be delivered to the same file directory i.e. (C:\flatfiles) as a delimited and positional flat file,
the following send ports, pipelines and orchestrations would be required:

Executing Send Pipelines in an Orchestration
i) Send Pipeline with FlatFile Assembler component for delimited flat file.
ii) Send Pipeline with FlatFile Assembler component for positional flat file.
iii) Send Port configured with Passthrough pipeline.
iv) Orchestration

Per Instance Pipeline Configuration
i) Send Pipeline with FlatFile Assembler.
ii) Send Port with per instance configuration of above pipeline to create delimited flat file.
iii) Send Port with per instance configuration of above pipeline to create positional flat file.
iv) Optionally an orchestration.
 
Dynamically Configuring Pipeline Component Properties in a BizTalk Orchestration
i) Send Pipeline with FlatFile Assembler.
ii) Send Port configured to use above pipeline.
iii) Orchestration

Not So Good Things
1) To dynamically configure and execute a receive pipeline, a two way send port will have to be used. This might introduce the need for a loopback adapter. A loopback adapter also introduces messagebox hops that will degrade performance.
2) Maintaining a store of the Pipeline configuration XML. The configuration store replaces individual pipeline components or per instance configuration of pipelines in a physical send or receive port.

There are two examples included in the download :
1) Orchestration sets receive pipeline configuration and uses a loopback adapter that executes altered configuration.
2) Orchestration sets send pipeline configuration and uses send file adapter to invoke altered configuration.
Note: A loopback adapter is also included in the download. Please read the ReadMe before installing

Unfortunately
I wish you could do something like the below:

OutputMessage(BTS.ReceivePipelineConfig) = "Pipeline Config XML would go here";
OutputMessage = Microsoft.XLANGs.Pipeline.XLANGPipelineManager.ExecuteReceivePipeline(typeof(PipelinesAndSchemas.ReceivePipeline), InputMessage);

It did not produce a runtime error, but the configuration for the Receive Pipeline was not overridden.

Last of All
I have yet to use this technique in a production environment. I have not tried this technique with pipelines configured to use multiple or custom components, but it should work with these scenarios. Please consider all the pros and cons of using this technique. Again, this could be an alternative to replace an unmanageable number of receive/send ports and pipelines.