Connected Pawns

Mapping an XML data type from a SQL stored procedure to a BizTalk schema

mbrimble — Wed, 23 Dec 2009 04:55:31 +0000

If you use the the WCF adapter wizard to generate the schema from the stored procedure then the XML data type is now of string type in the result set schema. How do you convert the ‘XML string’ to XML?

This is what I did in several of my maps. Note the loop that I have highlighted. The result set is unbounded and the AnswerSet is an XML data type in the stored procedure is a string type in the generated SQL schema. The scripting functoid that populates the AnswerSet node on the other side is an Inline XSLT call template. The trick is to disable the output escaping in the XSLT.

Copying functoids from one map to another

mbrimble — Sat, 10 Oct 2009 19:46:08 +0000

Have you ever wanted to copy a complex set of logic from one map to another? One of my young colleagues, Stephen Scott showed me a quick way to do this the other day. Here it is.

I had this map with many pages.

I wanted to change it to a map with two inputs instead of one.

The trick is as follows;

Open the first map in a text editor. You will see something like

GenerateDefaultFixedNodes=”Yes” PreserveSequenceOrder=”No” CopyPIs=”No” method=”xml” xmlVersion=”1.0″ IgnoreNamespacesForLinks=”Yes”>

………..more

Copy every thing between the Pages tag to the clipboard.
Open the second map in a text editor. You will something like

    http://schemas.microsoft.com/BizTalk/2003/aggschema” xmlns:b=”http://schemas.microsoft.com/BizTalk/2003″ xmlns:ns1=http://SubmitContactDetails xmlns:ns2=http://SubmitContactDetailsResponse targetNamespace=”http://schemas.microsoft.com/BizTalk/2003/aggschema” xmlns:xs=”http://www.w3.org/2001/XMLSchema”>
      http://SubmitContactDetails />
      http://SubmitContactDetailsResponse />

Now paste the contents of the clipboard between the two pages tags.
Do a search and replace of [local-name()='SubmitContact'] with [local-name()='Root']/*[local-name()='InputMessagePart_0']/*[local-name()='SubmitContact']
Save the file.
Now open the change in your Biztalk project and you will see your all the functoids and other artifactd have been copied to the new map.

Catching a WCF fault in a BizTalk orchestration

mbrimble — Thu, 17 Sep 2009 08:59:50 +0000

I wanted to catch a WCF fault. A quick search gave many hits about catching un-typed WCF faults for example http://geekswithblogs.net/paulp/archive/2008/06/23/123096.aspx and http://blogs.msdn.com/paolos/archive/2009/05/22/how-to-throw-typed-fault-exceptions-from-orchestrations-published-as-wcf-services.aspx. There is also an SDK example referenced in the blog by Paul Petrov. This blog describes how to catch a standard fault instead of an un-typed fault because I did not find any examples of this simpler case. I have recorded this here because it was not obvious to me from the examples above how to do it.

The web service that i was working with did not return a custom fault just a standard fault. I decided to follow the same general pattern given in the articles above but instead of creating my own custom exception schema I referenced http://schemas.xmlsoap.org/soap/envelope/ from the BTS system schemas (BTS.soap_envelope_1__1.Fault). Thus there is one less step than the Petrov’s pattern i.e.

Create a new multi-part message type CalculatePremium_WCFFault that has part of type BTS.soap_envelope_1__1.Fault.
Add to my CalculatePremiumPortType operation a new GenericWCFFault fault of type BTS.soap_envelope_1__1.Fault.
Add an exception handler CatchGenericWCFFault for this new fault

Now I tried following the SDK example and modified the Inbound BizTalk message body area on the send port to my web service, doing the following:
        a.    Select Path–content located by body path option.
        b.    In the Body path expression field, enter /*[local-name()='Fault']/*[local-name()='Detail']/* | /*[local-name()='PremiumResponse'].
        c.    From the Node encoding drop-down list, select Xml.
I also made sure that the Propagate fault message option is selected in the Error handling area. This did not work and I got an error probably because I did not get the body path expression correct.

To my surprise when I switched back and selected the default Body under body path option the WCF fault was caught and processed correctly. Now all is clear. I don’t have to use an XPath expression on the send port configuration because the general SOAP fault schema is known to the receive pipeline. The fault was copied in the Copy Errors and Exceptions shape using the following code

//Assign variables for WCF Fault where wcfFault is the exception of type PremiumCalcWebServiceLink[MyNameSpace.CalculatePremiumPortType].Operation_1.GenericWCFFault

wcfFaultStringXDoc = wcfFault.faultString;
sErrorMsg = wcfFaultStringXDoc.OuterXml;

In summary I can now catch all unhandled exceptions from my external web service.

“Failed to update MessageBox server while creating BizTalk Host”

mbrimble — Sun, 09 Aug 2009 05:10:40 +0000

I was trying to add a new host to a new BizTalk server and got this error first;

“failed to create SQL login and grant SQL privilege configurations on the Management database server “SQL07DEV” / database name “BizTalkMgmtDb” while creating BizTalkHost “test”

with the following additional information:

“This operation requires granting or revoking of SQL database level user access. Ensure that you have sufficient SQL permission to perform this operation. You must either be: 1) a member of the ‘sysadmin’ SQL fixed server role; or a member of the ‘db_owner’ or db_ddladmin’ SQL fixed database role. Contact your SQL database administrators for further assistance”

So I contacted our DBA. I was in the BizTalk Administrators group and in the SSOAdministrators group on that SQL server. He granted the BizTalkAdminstrators group db_ddladmin on the database above.

I tried to create a host and got a different error. This time I needed db_accessadmin to the same database. This role was granted and then I got the similar error but on the BizTalkDTAdb database.

After granting these roles on all the BizTalk database I got the error in the title namely;

“Failed to update MessageBox server will creating BizTalk Host. Couldn’t create SQL login or update SQL roles for BizTalk Host Windows group “HOME\BizTalk Host User Group” to access the MessageBox server “SQL07DEV\BizTalkMsgBoxDb”. The role BTS_test_USERS’ does not exist in the current database. Cannot find the user ‘BTS_test_USERS’, because it does not exist or you do not have permission….”

What now? This time there are no instructions on what to do. After looking up http://msdn.microsoft.com/en-us/library/aa559845(BTS.10).aspx I discovered we had done all the right things namely;

User rights granted to BizTalk Server Administrators

securityadmin SQL Server role on all SQL Servers

db_securityadmin and db_accessadmin SQL Server Database roles in the BizTalk Tracking, Rule Engine, BizTalk Management, BAM Primary Import and BizTalk MessageBox databases

db_ddladmin SQL Server Database role on all BizTalk MessageBox databases

SSO Affiliate administrators

I have reached this stage many times with other DBA’s and at this stage they have tossed in the towel and granted my login ‘sysadmin’ rights while I created the BizTalk hosts. This DBA was different he started to dig deeper.

He discovered that i could not create a new role because the role cannot be owned by the group that was in. He solved the problem by adding my login a user of the SQL server. Now when i try to create a host there a no errors and the host is created. If you now look at the roles you find that they are owned by my login. The owner was then changed to a more suitable SQL user.

How to copy the entire node to element of string type in a map

mbrimble — Sat, 01 Aug 2009 21:54:54 +0000

I wanted to copy the entire Contact node in the map below to the ContactDetails element of type string.

I could have done this in an orchestration by assigning the node to an XmlDocument variable and then assigning the XmlDocument.InnerText to a distinguished field in the outbound message but I wanted to do this in map because I plan to put the map on the outbound port and not in an orchestration.

My first try almost worked. I used a scripting functoid with this Inline XSLT.

This almost works but I get this error on validation;

The element ‘http://schemas.microsoft.com/Sql/2008/05/TypedProcedures/dbo:ContactDetails’ cannot contain child element ‘http://Blah.ContactDetails:Contact’ because the parent element’s content model is text only.

The problem here is that it’s copying the node structure to the outbound message and not mapping the node structure to a string.

So i have tried two solutions from posts by Greg Forsyth namely http://www.biztalkgurus.com/forums/t/12350.aspx and http://connectedpawns.wordpress.com/2008/10/08/call-a-net-assembly-from-custom-xslt/ ( I had to use my blog because the original link seems to be broken). I have decided to record how I did it because Greg’s instructions are very brief and it took me a while to work out what to do.

The first step is to create a helper class and deploy it to the GAC .

using System;
using System.Collections.Generic;
using System.Text;
using System.Xml;
using System.Xml.XPath;

namespace CopyXML2TextNodeBTSDemo.Utility
{
    public class Utilities
    {
        public static string ConvertNodeToXmlString(XPathNodeIterator node)

{ return node.Current.OuterXml; }

}
}

Secondly reference this assembly in the BizTalk map project.

Add a scripting functoid to the map , choose external assembly as the scripting type with the values shown below and connect it to the a field in the outbound schema.

Validate the map to generate a xsl file and a Custom extension XML file. The output is shown below

Invoking component.….CopyXML2TextNodeBTSDemo.Maps\SubmitContactDetails2XMLdatafieldasTEXT.btm: The compilation is using the CustomXslt and CustomExtensionXml tags to generate the output. The map content is ignored.
….CopyXML2TextNodeBTSDemo.Maps\SubmitContactDetails2XMLdatafieldasTEXT.btm: The output XSLT is stored in the following file:
….\SubmitContactDetails2XMLdatafieldasTEXT.btm: The Extension Object XML is stored in the following file:
Component invocation succeeded.

Save both files and add them to the BizTalk map project.

Change the map to use a the generated xsl file as the custom XSLT and the Extension Object XML file as shown below.

Edit xsl file like so;

http://www.w3.org/1999/XSL/Transform” xmlns:msxsl=”urn:schemas-microsoft-com:xslt” xmlns:var=”http://schemas.microsoft.com/BizTalk/2003/var” exclude-result-prefixes=”msxsl var s0 ScriptNS0″ version=”1.0″ xmlns:ns0=”http://CopyXML2TextNodeBTSDemo.XMLdatafieldasTEXT” xmlns:s0=”http://CopyXML2TextNodeBTSDemo.SubmitContactDetails” xmlns:ScriptNS0=”http://schemas.microsoft.com/BizTalk/2003/ScriptNS0″>

Now if you test the map you get the desired output. Whew!!

Using LoadGen to test Wcf-NetTCP binding

mbrimble — Fri, 31 Jul 2009 08:02:57 +0000

I have just returned from holiday and have had to do a lot of performance monitoring on WCF adapters. It took me a while to work out the settings in the LoadGenConsole.exe to test a WCF-NetTCP binding so i thought I would reproduce it here. ( See http://connectedthoughts.wordpress.com/2007/11/05/loadgen-2007-and-a-simple-biztalk-wcf-test/ for good guide on how to use LoadGen to test WCF adapters ).

address="http://localhost/BizTalkWcfgetPremiumService/WcfgetPremiumService.svc"

binding="basicHttpBinding"

bindingConfiguration="WcfgetPremiumEndBinding"

contract="WcfTransport.IChannelAsyncNonTransactional"/>

address="net.tcp://localhost/WCFgetPremium.NetTCP"

binding="netTcpBinding"

bindingConfiguration="WcfnetTCPgetPremiumEndBinding"

contract="WcfTransport.IChannelAsyncNonTransactional"/>

openTimeout="23:59:59" receiveTimeout="23:59:59" closeTimeout="23:59:59" sendTimeout="23:59:59">

    –>

Performance of the WCF-Basic HTTP Adapter

mbrimble — Tue, 23 Jun 2009 08:27:34 +0000

Zeeshan recently wrote about performance problems using the SOAP adapter( http://zeetalks.wordpress.com/2009/04/29/biztalk-performance-problems-using-the-soap-adapter/ ). In that article one of the solutions he suggested to the problem was to use one of the WCF adapters. Here I describe performance measure measurements using the WCF-Basic HTTP adapter.

I used Thiago’s Load Gen 2007 test to measure request-response message latency that he has described previously ( http://connectedthoughts.wordpress.com/2007/11/05/loadgen-2007-and-a-simple-biztalk-wcf-test/). It is easy to measure the request-response latency if you use a dedicated isolated host for your receive adapter.

The first orchestration test contained a WCF-Basic HTTP receive adapter exposed by BizTalk as shown below. The average message latency was about 0.8-1.0 seconds.

The second orchestration test contained the same WCF-Basic HTTP receive adapter exposed by BizTalk but also sent the message on after some mapping to an external WCF web service as shown below. The average message latency was now about 10-11 seconds.

In order to identify where the bottleneck was the external web service was replace by a simple web service returning just a stubbed out response. In this case the average message latency dropped to about 1.6-1.8 seconds. This is about what I would have expected form the addition of some extra persistence points in the orchestration.

Finally the after the external web service had been tuned a message latency of about 2-2.2 seconds was achieved.

This example suggests that lower message latency can be achieved between BizTalk and web services if the WCF adapters are used instead of the traditional SOAP adapters. We expect that we can achieve even lower message latency if we tune BizTalk for low latency as suggested by Thiago in comments to Zeeshan’s article.

Receive port	Send Port	Loadgen testmessages/sec	Average Request- Response Latency (seconds)
WCF-Basic HTTP Request-Response	File Adapter	10	0.8-1.0
WCF-Basic HTTP Request-Response	File Adapter with outbound maps	10	0.8-1.0
WCF-Basic HTTP Request-Response	WCF-Basic HTTP Request-Response with outbound and inbound maps to external web service	10	10-11
WCF-Basic HTTP Request-Response	WCF-Basic HTTP Request-Response with outbound and inbound maps to external web service returning a stubbed out response	10	1.6-1.8
MOSS WCF-Basic HTTP Request-Response	WCF-Basic HTTP Request-Response with outbound and inbound maps to tuned external web service	10	2.0-2.2

The SQL Adapter and Apostrophes

mbrimble — Tue, 05 May 2009 01:10:52 +0000

I got an amusing call from a DBA the other day saying that the BizTalk SQL adapter was failing whenever he sent through a message that had an apostrophe in it. The message being sent to the SQL adapter contained;

Mum’s Apple Pie

The message had suspended in Biztalk with a failure to due a failed MSDTC transaction on the SQL port. The DBA had opened the message in the BizTalk Adminstrators Console Group Hub and then copied the message into his SQL analyzer to test the SQL query. He had noticed that the apostrophe was not escaped and rightly deduced that the SQL query would fail if it was not escaped. See http://www.kamath.com/codelibrary/cl003_apostrophe.asp for example.

I told him to calm down because this was not the source of his problem. I got him to start SQL profile and set up a filter to trace the SQL query coming from the faulting SQL adapter. Sure enough the trace showed that when BizTalk sends the same message to a SQL adapter it magically escapes the apostrophe and the SQL request now contains;

Mum’’s Apple Pie

The point here is that if you are troubleshooting a SQL adapter failure sometimes the suspended message does contain the exact SQL request that was submitted. I think that you need to trace the SQL transaction to see what is really being submitted.

Once he submitted the SQL from the trace to his stored procedure he could see that the error was due to a change he had made and it was not the fault of the BizTalk server solution. To finish with I can’t resist giving this link to an amusing cartoon about misuse of the apostrophe ( http://www.angryflower.com/bobsqu.gif ). Maybe we need a cartoon about misuse of the apostrophe in SQL queries.

Custom Party Resolution BizTalk Server 2006: Resolve party by POP3.From and square brackets

mbrimble — Sat, 28 Mar 2009 21:58:27 +0000

Recently I wrote a BizTalk Server 2006 custom pipeline component to resolve a custom party on the POP3.From context value when an email was received. I was surprised to find some unusual behaviour if the POP3.From context value contained square brackets like “HEWHOCARES [WORLD]” < HEWHOCARES@world.com >. I describe the behaviour and why it occurs below because it took me by surprise.

The base example for the custom pipeline component is based on http://msdn.microsoft.com/en-us/library/aa559134.aspx . The code fragment below shows the important changes;

/// This function gets called after the entire stream of the message has been read, guaranteeing that all promoted properties
        /// from the message have been populated to the message context.
        ///
        internal void EndOfStream()
        {
            string FromEmail = null;
            BizTalkParty party;

//obtain the FromEmail string from the message
FromEmail = (string)mBaseMessage.Context.Read(“From”, “http://schemas.microsoft.com/BizTalk/2003/pop3-properties”);

            if (FromEmail != null)
            {
                PartyResolver pr = new PartyResolver();
                party = pr.GetPartyFromAlias(“FromEmail”, “FromEmailName”, FromEmail);

                mBaseMessage.Context.Promote(“SourcePartyID”, “http://schemas.microsoft.com/BizTalk/2003/system-properties”, party.SID);
                mBaseMessage.Context.Promote(“PartyName”, “http://schemas.microsoft.com/BizTalk/2003/messagetracking-properties”, party.Name);
            }
        }

I set up my party like;

On testing this pipeline component with an email from “HEWHOCARES” < HEWHOCARES@world.com > it resolves the party to ThePeopleWhoCare. Now if I change the party value to “HEWHOCARES [WORLD]” < HEWHOCARES@world.com > and send an email from the same email address then it resolves to Guest not ThePeopleWhocare. What is going on? If you dig a bit further you find that to resolve the party this stored procedure is used;

USE [BizTalkMgmtDb]
GO
/****** Object: StoredProcedure [dbo].[admsvr_GetPartyByAliasNameValue]    Script Date: 03/24/2009 21:16:03 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER PROCEDURE [dbo].[admsvr_GetPartyByAliasNameValue]
@nvcAliasName nvarchar(256),
@nvcAliasQualifier nvarchar(64),
@nvcAliasValue nvarchar(256),
@nvcSID nvarchar(256) OUTPUT,
@nvcName nvarchar(256) OUTPUT
AS
SELECT      @nvcSID = bts_party.nvcSID,
            @nvcName = bts_party.nvcName
FROM bts_party, bts_party_alias
WHERE       UPPER(bts_party_alias.nvcName) = UPPER(@nvcAliasName) AND
            UPPER(bts_party_alias.nvcQualifier) = UPPER(@nvcAliasQualifier) AND
           bts_party_alias.nvcValue LIKE @nvcAliasValue AND
           bts_party_alias.nPartyID = bts_party.nID

The WHERE clause uses a LIKE statement for the criteria based on the passed POP3.FROM context value. [] in a like statement is evaluated as any single character within the specified range. Thus in our case we try find a party with an FromEmail value of “HEWHOCARES W” < HEWHOCARES@world.com > (or where W” is O”, R”, L” or D”). Indeed if I now change the party value to “HEWHOCARES W” < HEWHOCARES@world.com > then “HEWHOCARES [WORLD]” < HEWHOCARES@world.com > resolves to ThePeopleWhoCare once again.

Many-to-one mappings using the Table Looping Functoid

mbrimble — Sun, 08 Mar 2009 03:13:30 +0000

I have found another solution to the scenario 2 that I blogged about a couple of weeks ago (“The values in the output message are an aggregation of the two input messages with one value coming from only one input message“). This solution does not use custom XSLT just the BizTalk mapper.

I found if I used the Table looping functoid instead of the looping functoid I could get the correct transformed message as well.