This project has moved. For the latest updates, please go here.

How to install the BRE Pipeline Framework

Download the latest version of the installer and run it on your PC which has BizTalk Server 2010 or 2013 installed on it. The installer will add the relevant assemblies to the .NET 4 GAC, will add the BREPipelineFrameworkComponent pipeline component to the pipeline components folder, and will create the BRE Pipeline Framework folder in your 32-bit program files folder (so "Program Files(x86)" on a 64-bit machine, or "Program Files" on a 32-bit machine). 

Note that if you are installing a new version of the framework on a machine which is running a version older than 1.4.0 (this won't be a problem in version 1.4.0 or later) then you might have to shut down host instances in order for the installation to complete.

Open the Business Rules Engine Deployment Wizard (found in the Microsoft BizTalk Server 2010 folder in your start menu) and choose to import each of the vocabularies contained within the Vocabularies subfolder in the BRE Pipeline Framework program files folder.  You only need to install the latest version of each vocabulary unless older ones are required due to pre-existing BizTalk artefacts which depend on those versions.

 

How to use the BREPipelineFrameworkComponent Pipeline Component in a BizTalk Pipeline

In order to use the BREPipelineFrameworkComponent in a pipeline you will first need to add it to the pipeline toolbox in Visual Studio. Open (or choose to create) a send or receive pipeline in Visual Studio, hover your mouse over the toolbox on the left hand side of the window, right click and click choose items. Navigate to the BizTalk Pipeline Components tab, tick the BREPipelineFrameworkComponent and click ok.

Now drag the BREPipelineFrameworkComponent from the toolbox into the relevant stage of your pipeline (any stage except disasseble or assemble). You will need to set the relevant values for the below configuration properties on the component.

  • ApplicationContext (Optional) - This can be any string which you can use in your business rules to selectively choose whether that rule should fire or not when the policy is called by a specific pipeline. Typically this should be the name of the pipeline, but you could be more prescriptive and use the name of the pipeline and the stage of the pipeline in case you were going to use the BREPipelineFrameworkComponent in multiple stages of your pipeline.
  • Enabled - A boolean value which instructs whether the pipeline component should be active or not. If set to false then the pipeline component will behave in a pass through manner.  The default value is true.
  • ExecutionPolicy (Optional) - The name of the BRE policy which contains all the logic used to assess and manipulate the message being processed by the pipeline.
  • InstructionLoaderPolicy (Optional) - The name of the BRE policy that allows you to conditionally load facts into your ExecutionPolicy in advanced used cases as below in addition to some other functionality.
    • Custom .NET based facts - The InstructionLoaderPolicy can be used to assert custom metainstruction classes into the ExecutionPolicy in case that you want to use custom .NET based vocabularies in your ExecutionPolicy in addition to out of the box vocabularies. 
    • XML based facts - The InstructionLoaderPolicy can also be used to assert TypedXmlDocument facts into the ExecutionPolicy so you can make use of XML based vocabularies.
    • SQL based facts - The InstructionLoaderPolicy can also be used to assert SQL DataConnection facts into the ExecutionPolicy so you can make use of SQL based vocabularies. 
    • Override ExecutionPolicy - The InstructionLoaderPolicy can be used to override which ExecutionPolicy gets called, which can help you to avoid building very large BRE policies for ExecutionPolicies, and potentially deliver you better performance by eliminating the evaluation of unnecessary conditions.
    • Override ApplicationContext - The InstructionLoaderPolicy can be used to override the value of the ApplicationContext which is passed into the ExecutionPolicy, providing for more flexibility in complex use cases.
  • RecoverableInterchangeProcessing - This is a boolean value that specifies whether the pipeline component should behave in a recoverable interchange fashion or not.  If a message has been debatched in a pipeline component prior to the BREPipelineFrameworkComponent and one of the debatched messages fail, if this parameter is set to true then only that one debatched message fails.  If the parameter value is set to false (default) then the entire message fails.
  • TrackingFolder - If a folder name is specified in this parameter then the rules engine debug tracing data will be written to this folder for both the InstructionLoaderPolicy and ExecutionPolicy, helping developers to debug their BRE policies.  The BizTalk host instance must have write access over the folder in question.

The pipeline can now be deployed to the BizTalk runtime.  You will need to publish and deploy BRE policies with names matching those you set for the InstructionLoaderPolicy and ExecutionPolicy in the pipeline component parameters.  Note that the pipeline component will always choose the latest version of said policies that is currently deployed (though note that BizTalk caches policies and renews the cache at regular intervals, so you might not see new rules take effect immediately unless you restart the rule engine update service).

 

Summary explanation of out of the box vocabularies

After completing the installation instructions you should three BRE Pipeline Framework related vocabularies (you might potentially have multiple versions of each one) published to the rules engine database that you can view in the Business Rules Composer as below.

  • BREPipelineFramework - This vocabulary for the most part can only be used in InstructionLoaderPolicy BRE policies.  The one exception to this rule is the ApplicationContext vocabulary definition which can be used in ExecutionPolicy BRE policies as well, and can be used in rules to evaluate the value of the ApplicationContext on the pipleine properties.  Other vocabulary definitions that can only be used in InstructionLoaderPolicy BRE policies can be used to inspect the BizTalk message (through regex matching, string finds, XPaths) for condition evaluation, and to assert .NET, XML, or SQL based facts, or to override the ExecutionPolicy to be called, or to override the ApplicationContext to be passed to the ExecutionPolicy.
  • BREPipelineFramework.SampleInstructions.ContextInstructions - This vocabulary can only be used in ExecutionPolicy BRE policies, and contains vocabulary definitions to get or set all BizTalk out of the box context properties through enumeration, to get or set custom context properties, and to remove context properties.
  • BREPipelineInstructions.SampleInstructions.HelperInstructions - This vocabulary provides helper type functionality to evaluate and manipulate message content and context, as well as common helper functionality such as string concatenation and formatting, date time rounding, guid generation, get the current date time, or throw an exception with a custom message.

 

How to setup an ExecutionPolicy

An ExecutionPolicy is used to assess rule conditions and then to carry out actions which were specified by the rules that evaluated succesfully. The execution policy can make use of any vocabulary definitions in the out of the box BRE vocabularies which should suffice in most simple message context/content assessment/manipulation scenarios.

In more advanced scenarios one can also make use of BRE policies with custom .NET, SQL, or XML based vocabulary definitions if their corresponding facts were asserted into the ExecutionPolicy from within the InstructionLoaderPolicy.  InstructionLoaderPolicy BRE policies are only required for advanced scenarios.

Assuming you only have simple requirements that only dicate usage of an ExecutionPolicy, create a new BRE policy using the Business Rules Composer and name it appropriately such that it matches the ExecutionPolicy parameter value on your pipeline. Create a rule, and in the conditions section of the rule you can setup predicates which evaluate conditions based on applicable BRE vocabulary definitions.

In the actions section of the rule you can specify what actions you want to carry out. In the below example the value BREPipelineFramework.csv is written to the ReceivedFileName context property (using the GetCustomContextProperty vocabulary definition contained within the BREPipelineFramework.SampleInstructions.ContextInstructions vocabulary).

An ExecutionPolicy policy can be made up of multiple rules, each one with different sets of conditions and actions. An ExecutionPolicy might also be shared between multiple pipelines, with rules that aren't common to all pipelines having conditions based on ApplicationContext.

 

Using the out of the box BREPipelineFramework vocabulary definitions

Below are some examples (this list is not all-inclusive) of usage of vocabulary definitions in the BREPipelineFramework BRE vocabulary (which can only be used in InstructionLoaderPolicy BRE policies except for ApplicationContext).

  • AddMetaInstruction - This definition allows you to instantiate and assert a custom .NET fact into the ExecutionPolicy so that you can use corresponding .NET based vocabulary definitions in the ExecutionPolicy.
  • AddSQLDataConnection / AddSQLDataConnectionFromSSO - These definitions allow you to assert a SQL DataConnection based fact into the ExecutionPolicy so that you can use corresponding SQL based vocabulary definitions in the ExecutionPolicy.  You can add as many SQL DataConnections as required.
  • SetTypedXmlDocument - This definition allows you to assert the current BizTalk message as an XML fact into the ExecutionPolicy so that you can use corresponding XML based vocabulary definitions in the ExecutionPolicy.
  • ApplicationContext - This definition fetches the value of the ApplicationContext parameter set on the calling pipeline component.
  • CheckIfRegexEvaluatesInMessage - This definition returns a boolean value indicating whether a specified regular expression finds a match in the BizTalk message.
  • CheckIfStringExistsInMessage - This definition returns a boolean value indicating whether a specified string is found in the BizTalk message.
  • GetMessageBodyLength - This definition returns the length of the message body.
  • GetMessageRootNodeName / GetMessageRootNodeNamespace - These definitions returns the root node name/namespace of the BizTalk message assuming that the message is an XML message.  If the BTS.MessageType context property is known then this definition will extract the value from that, otherwise it will run an XPATH statement against the message which is slower.
  • OverrideApplicationContext - This definition allows you to override the ApplicationContext value set on the pipeline component properties when evaluated within the ExecutionPolicy.  Note that the override will not apply for the remainder of the InstructionLoaderPolicy's execution.
  • OverrideExecutionPolicy - This definition allows you to override the ExecutionPolicy value set on the pipeline component properties thus allowing for other ExecutionPolicy BRE policies to be called on.
  • ReturnFirstRegexMatch - This definition will return a string that is the first match for a given regular expression against the BizTalk message.
  • ReturnRegexMatchByIndex - This definition will return a string that is the match at the given index for a given regular expression against the BizTalk message.

 

Using the out of the box BREPipelineFramework.SampleInstructions.ContextInstructions vocabulary definitions

Below are some examples (this list is not all-inclusive) of usage of vocabulary definitions in the BREPipelineFramework.SampleInstructions.ContextInstructions BRE vocabulary.

  • Get<XXX>ContextProperty - These are a collection of definitions that allow you to get the context property values from the message being processed, the context properties being from any of the major out of the box BizTalk property schemas.  There is a seperate definition for each of the following major property schema namespaces - BTF2, BTS, EDI, EdiIntAS, EdiOverride, ErrorReport, FILE, FTP, HTTP, LEGACY, MessageTracking, MIME, MSMQT, POP3, SMTP, SOAP, SQL, WCF, WSS, XLANGsBTXEngine, and XMLNORM.  These definition allow you to choose the relevant context property within the contained property schema namespace from a drop down box containing a list of all the supported context properties, thus removing the need for the implementer to know the names of the context property names and namespaces, and also removing the chance of typos.  If the context property is not found on the message you can specify if the return value should be blank (or the default for whatever .NET type the property corresponds to) or if an exception should be thrown.

  • GetCustomContextProperty - This definition allows you to access any context properties on the message being processed by the pipeline by specifying the property name and namespace of the property. If the context property is not found you can specify if the return value should be blank (or the default for whatever .NET type the property corresponds to) or if an exception should be thrown.

 

  • RemoveCustomContextProperty - This definition allows you to remove any supported context property (some out of the box context properties can't be removed) from the message being processed by the pipeline by specifying the name and namespace of the property to be removed.

  • Set<XXX>ContextProperty - These are a collection of definitions that allow you to set the context property values on the message being processed, the context properties being from any of the major out of the box BizTalk property schemas. There is a seperate definition for each of the following major property schema namespaces - BTF2, BTS, EDI, EdiIntAS, EdiOverride, ErrorReport, FILE, FTP, HTTP, LEGACY, MessageTracking, MIME, MSMQT, POP3, SMTP, SOAP, SQL, WCF, WSS, XLANGsBTXEngine, and XMLNORM.  These definition allow you to choose the relevant context property within the contained property schema namespace from a drop down box containing a list of all the supported context properties, thus removing the need for the implementer to know the names of the context property names and namespaces, and also removing the chance of typos. These definitions allow you to set any value (remember that this value doesn't have to be static, you can use vocabularies to dynamically set the value as well) to the relevant context properties.  You can also choose whether you want to promote the context property or just write the value to it.

  • SetCustomContextProperty - This definition allows you to set any value to any context property, the property being specified with the property name and namespace and by choosing the .NET type of the property with a drop down. You can also choose whether you want to promote the context property or just write the value to it.

  • SetCustomContextPropertyFromSSO - This definition allows you to set a value that is read in from a specified SSO key in a specified SSO application to any context property, the property being specified with the property name and namespace and by choosing the .NET type of the property with a drop down. You can also choose whether you want to promote the context property or just write the value to it.

  • SetCustomContextPropertyFromXPathResult - This definition allows you to set the resulting value / node name / node namespace of an XPath expression run against the message (you can choose whether to return an exception or not if the XPath expression does not evaluate) to any context property, the property being specified with the property name and namespace and by choosing the .NET type of the property with a drop down. You can also choose whether you want to promote the context property or just write the value to it. Using this definition against a message multiple times will still only result in the XML document being loaded into memory once.

 

Using the out of the box BREPipelineFramework.SampleInstructions.HelperInstructions vocabulary definitions

Below are some examples (this list is not all-inclusive) of usage of vocabulary definitions in the BREPipelineFramework.SampleInstructions.HelperInstructions BRE vocabulary. 

  • GenerateGUID - This definition generates a random GUID.
  • GetCurrentDateTime - This definition gets the current date and time, allowing you to specify the format it should be fetched in.
  • GetFileExtension - This definition returns the file extension for a specified filename.
  • GetLowercaseString - This definition converts the string parameter to lowercase.
  • GetStringLength - This definition returns the length of a string.
  • GetUppercaseString - This definition converts the string parameter to uppercase.
  • GetXPathResult - This definition returns the resulting value / node name / node namespace of an XPath Expression. It allows you to specify whether an exception should be thrown if the expression doesn't evaluate.
  • ReplaceSubstring - This definition allows you to replace a substring in a given string with another string.
  • RoundCurrentTime - This definition allows you to round the current time up to a given interval, also allowing you to specify the format the time should be returned in. This can be quite handy for non-sequential convoy type batching scenarios.
  • StringContains - This definition checks whether a string contains the specified substring.
  • ValidateStringLength - This definition throws an exception if the length of a string does not exactly match a specified value. Since this defintion returns a void it is really only intended for usage in the Action section of a rule.

 

Using XML based vocabulary definitions

XML based vocabulary definitions corresponding to the BizTalk message passing through the executing pipeline can be used in ExecutionPolicy BRE policies only, but must first be setup in an InstructionLoaderPolicy BRE policy.

Rules can be setup in the InstructionLoaderPolicy in which conditions to check the message type based on the root node name and namespace are evaluated, and based on results the appropriate TypedXmlDocument type can be asserted into the ExecutionPolicy. These conditions can be based on the GetMessageRootNodeName and GetMessageRootNodeNamespace vocabulary definitions in the BREPipelineFramework vocabulary.

The action in these rules would make use of the SetTypedXmlDocument vocabulary definition in the BREPipelineFramework vocabulary, in which you would need to pass in the fully qualified .NET type corresponding to the schema in question.

Once you've done this you can then make use of XML based vocabularies in your ExecutionPolicy just as you would with a BRE policy being called from within an orchestration.

 

Using SQL based vocabulary definitions

SQL based vocabulary definitions can be used in ExecutionPolicy BRE policies only, but first must be setup in an InstructionLoaderPolicy BRE policy, with one fact needing to be asserted per database table which will be evaluated or updated.

The AddSQLDataConnection or AddSQLDataConnectionFromSSO vocabulary definitions can be used in InstructionLoaderPolicy rule actions to setup a SQL DataConnection object. The AddSQLDataConnection vocabulary definition requires you to specify a connection string, the name of the database, and the name of the table in question. The AddSQLDataConnectionFromSSO vocabulary definition works in the same manner except it fetches the connection string from the nominated SSO config store (note that the BizTalk host instance executing the pipeline must have access to this config store) which is handy in cases where you can't use integrated security and don't want to store credentials in your BRE policies.

Once you've done this you can then make use of SQL based vocabularies in your ExecutionPolicy just as you would with a BRE policy being called from within an orchestration.

 

Overview on using custom .NET based vocabulary definitions

The BRE Pipeline Framework out of the box vocabuary definitions in the BREPipelineFramework.SampleInstructions.ContextInstructions and BREPipelineInstructions.SampleInstructions.HelperInstructions make use of an extensible framework which make us of MetaInstructions and Instructions. 

It is possible to extend the framework by writing your own MetaInstruction and Instruction classes in assemblies which should be deployed to the GAC, creating vocabulary definitions that correspond to public methods on your MetaInstructions, and then using the AddMetaInstruction vocabulary definition in the BREPipelineFramework vocabulary in InstructionLoaderPolicy BRE Policies which will allow you to use the your custom vocabularies in the ExecutionPolicy.

A MetaInstruction is a class that derives from the BREPipelineFramework.MetaInstructionBase base class in the BREPipelineFramework assembly which is provided as part of the framework. A MetaInstruction is composed of helper methods which are intended to be accessed from within a BRE policy. These methods could be used to access context information within a BizTalk message or could simply be Helper type methods such as one to check whether a string contains a given substring. A MetaInstruction also contains a collection of Instructions and optionally a set of methods that are used to instantiate an instruction and add it to the collection.

An Instruction is a class that implements the BREPipelineFramework.IInstruction interface in the BREPipelineFramework assembly which is provided as part of the framework. It must implement a method called Execute which performs the logic required by that Instruction. Such logic would typically be an action that would modify something about a BizTalk message such as it's context or content. An example would be to promote a value into the BTS.DestinationPartyName context property.

The MetaInstructionBase class also contains an overridable execute method which will perform the execute method on all contained instructions.  The execute method on MetaInstruction classes will always be automatically called on after the ExecutionPolicy completes.

The below sections explain how you can write your own custom Instructions and MetaInstructions.

 

Writing your own custom Instruction

Create a new .NET class library type project using .NET 4.0 and reference the BREPipelineFramework.dll in the BRE Pipeline Framework\Bin folder in your program files folder (you can copy it to a local libraries folder in your solution folder if you want). The project will need to be signed and added to the GAC once it has been completed. You will also find a lot of handy helper methods in BREPipelineFramework.Helpers.dll and BREPipelineFramework.SampleInstructions.dll also located in the BRE Pipeline Framework\Bin folder in your program files folder and might want to add references to these assemblies as well.

Create a new class and choose to implement the BREPipelineFramework.IBREPipelineInstruction interface. Create the appropriate properties that are required by your class and create as many constructors as you want. Implement the Execute(ref IBaseMessage inmsg, IPipelineContext pc) method which is dictated by the IBREPipelineInstruction interface such that it manipulates your message in whatever way you want.

 

Writing your own custom MetaInstruction

Create a new .NET class library type project using .NET 4.0 and reference the BREPipelineFramework.dll in the BRE Pipeline Framework\Bin folder in your program files folder (you can copy it to a local libraries folder in your solution folder if you want). Also reference the project that contains the instructions you want to work with, or optionally you can combine your Instructions and MetaInstructions into the same project. The project will need to be signed and added to the GAC once it has been completed.

Create a new class and choose to inherit from the BREPipelineFramework.BREPipelineMetaInstructionBase base class. If you are going to specify constructors for this class then ensure that you specify a no-parameter constructor since that is how this class will be instantiated by the InstructionLoaderPolicy.

Create as many methods as you want to use within the BRE. These can be simple methods such as those contained within the out of the box BREPipelineFramework.SampleInstructions.MetaInstructions.HelperMetaInstructions class which are fully self contained. Alternatively they can be methods which are used to instantiate Instructions and add them to the collection held by the MetaInstruction (do this by calling the base.InstructionCollection.Add(Instruction) method within your MetaInstruction) which will be executed after the ExecutionPolicy has completed like the majority of the methods contained within the out of the box BREPipelineFramework.SampleInstructions.MetaInstructions.ContextMetaInstructions class.

If you want to return an exception from any of your methods then it's preferable that you set the base.BREException property to your exception rather than throwing it, as throwing it would return a generic error to the pipeline saying there was an exception encountered exercising a rules policy whereas setting the base property will instruct the pipeline component to throw the error so that the error details are surfaced properly.

Once you have setup your MetaInstruction classes you will want to deploy the assembly to the GAC. You can now open the Business Rules Composer and create vocabularies for each of the methods exposed by your MetaInstruction. As long as you have specified that your class should be instantiated in your InstructionLoaderPolicy you can make use of the vocabulary definitions in your ExecutionPolicy.

 

Using .NET based vocabulary definitions based on custom MetaInstruction classes

Use the AddMetaInstruction vocabulary definition from the BREPipelineFramework vocabulary to instantiate instances of your custom MetaInstruction classes in actions in your InstructionLoaderPolicy, and to then assert them into your ExecutionPolicy.  You will need to specify the fully qualified class name and fully qualified assembly name that contains that class as parameter values into the AddMetaInstruction vocabulary definition.

All MetaInstructions that are used by the BRE Pipeline Framework must be in the GAC. In order to find out the fully qualified assembly name you can open a visual studio command prompt and type in - Gacutil -l <assembly name without the .dll extension>.

Last edited Apr 9, 2014 at 12:45 PM by jcooper1982, version 8