A customer asked me recently if the BizTalk business rules engine was a good place to search for repeating elements containing duplicate values in an XML message that is received on a one way BizTalk hosted WCF service and to throw an exception back to a service consumer if a duplicate is found. My initial gut feeling was that the BRE wasn’t quite the best place to do this and I decided to search for alternatives before exploring this option further.
I was surprised to find out that the W3C standards for Xml schemas cater for unique constraints in your XML messages, defining keys scoped to a complex type that define which element/attribute or combination of elements and attributes are not allowed to repeat. This is not a feature in XSD schemas that myself or anyone I spoke to about had heard of before, and a common reaction was that no one had ever encountered such a requirement before. I decided to find out whether BizTalk supports these unique constraints and whether the XML validator pipeline component could be used to enforce these constraints.
The first thing I discovered is that there is no way in the schema designer to define the unique constraints, at least not that I could find. I decided I would follow the W3C guidelines and handcraft my constraint. Take a look at the below schema and take note of how the constraint I’ve defined prevents the same SupplierName value from being used across two Purchase records.
Validating the below instance file against the schema returns an error as expected since a duplicate SupplierName of Redmond has been used in two of the Purchase records. The error is reasonably detailed advising the name of the unique constraint that was not respected as well as which duplicate value broke the constraint. Providing unique values for the SupplierName elements results in the instance file validating successfully. Note that the XML editor in visual studio also highlights the fact that a constraint has been violated as in the below screenshot (look at the close tag of the second Purchase record).
Now what if we wanted to add a second constraint on the PurchaseOrder element as well. Not a problem, just add a second unique constraint. Now both constraints must be respected in order for an instance file to successfully validate.
Another interesting scenario is to extend the duplicate checking to actually check for combinations of multiple elements. Let’s throw attributes into the mix as well. The below unique constraint now allows for duplicate SupplierName or PurchaseOrder elements or duplicate PurchaseDate attributes but not duplicates of all three in combination.
We can also extend the duplicate check to elements in contained records. In the below example the ItemBarCode element in the child ItemDetails record has now been added to the unique constraint.
Running a duplicate message through an actual XML validator pipeline component also throw the error as expected.
Now what if one of the elements in your duplicate check is optional. In the below screenshot the optional ItemDescription element has been added to the unique constraint and even though neither of the Purchase records contain a ItemDescription element and all the other elements in the constraint are duplicated the constraint is not deemed to be violated. The constraint is only violated if the ItemDescription element is specified with a duplicate value. If it is missing, even if it is missing in two record in which every other element contains duplicate values, the constraint won’t be violated.
Another scenario… What if the element in your duplicate check is a repeating element? I extended the duplicate check to the SpecialDeals element as well and what I found was that the second I had more than one SpecialDeals element in a given Purchase/ItemDetails record I would get an error stating that only one element was expected. Adding an element to a unique constraint disallows it from being a repeating element.
The schema I’ve been working with has an ElementFormDefault of unqualified. If this is changed to qualified then a prefix will have to be manually defined in the schema and added to the constraints as below..
Even though the schema designer in Visual Studio does not seem to complain, the project seems to build, and the XML editor in Visual Studio tells you when the constraint has been violated successfully, the XSD validator (or validate instance) will always fail with a cryptic error saying – “The prefix ‘x’ in XPath cannot be resolved”. I have not as yet found a way to get this scenario to work (note that you are not able to explicitly state namespaces in your XPath fields or selectors so prefixes are your only option). You would also encounter this problem if any records or elements that are part of the unique constraint belonged to a different namespace than the root node of the schema. Is this a bug with BizTalk schema validation or is there another not so well known way to define element namespaces in your unique constraints?
It gets even worse for attributes with an AttributeFormDefault of qualified as the schema editor will complain if you try to add a prefix to your attribute. I have not found a way to get this to work yet either.
Anyone have any ideas on how to work around the problems I’ve described above? Download the sample source code and have a play and do let me know if you manage to crack the problem.
In conclusion, it appears that BizTalk has some support for Unique constraints in XSD schemas however it is either a bit buggy or not very well documented when it comes to dealing with elements with a namespace, and sadly in its current implementation (as of BizTalk 2010) it is probably only useful in limited circumstances.
