This project has moved. For the latest updates, please go here.

BRE and Maps

Apr 21, 2013 at 6:29 PM
I've come across your project and will be playing with it this week. I've come across a Export, Transform, Load conundrum between two big-data systems. For example, I have one property that needs to be validated by the means of regex and split by the means of regex, outputting three separate values to the target schema. I do want to use a rules engine that is business analyst friendly but I'm not sure that a BT map can accomplish this.

My initial thought was to add custom functoids that would be able to execute BRE rules during the mapping process with the help of helper methods, but I'm not sure that is the most efficient way. The other way I thought to do this is to use Automapper and BRE from an orchestration bypassing a BT map completely.

What are your thoughts on accomplishing complex validation and data splitting in order to successfully transform data between two systems?

-c
Coordinator
Apr 21, 2013 at 7:24 PM
Edited Apr 21, 2013 at 7:26 PM
Hi Chris,

Thanks for having a play with the BRE pipeline framework. I think there are a few options to achieve what you're trying to do, some of them will be better/worse/impossible based on your specific circumstances. I will provide less details on the non BRE pipeline framework options but at least you should get the idea.
  1. ) BRE pipeline framework + property demotion - With this option, in your receive pipeline you could run regexes against specific XML elements by specifying the xpath query to reach that element, using the result of the regex to evaluate your rule condition, and in your actions you could set the values of custom context properties to the result of your desired regexes against the xpath query to your specific element. Your map on the receive port should set blank values to the xml elements in which you want to store the results (unless the source system pre-initializes the elements with blank values). Your send pipeline on the outgoing side would need to include an XML disassembler to demote the property values into the result xml elements (the XML elements must be linked to these context properties in the schema designer).
Pros
•No orchestration required, pure messaging solution with regexes logic purely held in BRE.
•Very easily tested using a framework such as BizUnit.

Cons
•This solution is a lot more complex if the node you want to split/evaluate is a repeating element.
•Requires an advanced understanding of many BizTalk concepts such as path (can hide the expressions by using vocabulary constants) and property demotion (could potentially work around this by using the context access or pipeline/functoid and setting the value of the result elements from the context properties in a map).

2.) Call BRE from map using scripting functoids and C# helpers - With this option you will pass the source element value(s) to a scripting functoid which would call the BRE which would set the results to a .Net object. You could then set the properties on that .Net object to the result elements.

3.) Call BRE from orchestration asserting XML document and .Net facts - With this option the BRE can make use of XML facts to get/set values from the XML document, but this is a rather inefficient method of doing this, especially if you don't need orchestration for any other reason.

4.) Look at non BRE options to solving this problem.

I hope that helps.
Cheers
Johann
Apr 23, 2013 at 4:04 AM
Thanks for the tips. I never knew about property demotion. This sounds like the most flexible solution. I also like #2, however I wasn't aware that functoids inside of a map are executed in any particular order.

-c