BizTalk Send Handlers/SSO Bug?

My good friend Carlos ran into a situation while working with one of his clients that seems to be triggering what looks like a bug in BizTalk Server 2010. The specific case came up when changing the Windows User Group associated with a BizTalk Host when that host is already associated as a Send Handler for an adapter.

Apparently, when doing the change, BizTalk will correctly update the information stored in the Enterprise Single Sign-On (ENTSSO) database (SSODB) for Receive Handlers, but not with Send Handlers, which leaves the system generating errors when the host later tries to access the SSODB to access the stored adapter settings.

Here’s  the repro instructions:

  1. Create a new BizTalk Host for the test. Let’s name it BTTestHost. Assign to it the default BizTalk Application Users group, and then create a new Host instance.
  2. Add the BTTestHost as a send handler for an adapter; we’ll use the FILE adapter in this example.
  3. At this point, if you’ check the ssodb.dbo.SSOX_ApplicationInfo table, you should see an entry associated with FILE_TL_BTTestHost, so that’s our newly created send handler. You’ll notice that the ai_admin_group_name column has the value ”BizTalk Application Users”, as expected.
  4. Now create a send port associated to the new Send Handler and test to verify everything is working correctly.
  5. Stop and delete the existing host instance.
  6. Now let’s create a new Windows Users Group, let’s name it “BizTalk Test Group”. We’ll also create a new windows user we’ll use to create a new host instance in a bit; and we’ll make that user a member of BizTalk Test Group. Make sure the user is not a member of “BizTalk Application Users”.
  7. Open the properties for BTTestHost and change the associated Windows User Group, to the new “BizTalk Test Group”.
  8. Go ahead and create a new Host Instance, add a new Send Port associated with our Send Handler and run a test. You’ll notice you’ll get an error with something like “Access denied. The client user must be a member of one of the following accounts to perform this function.” (followed by a list of windows user groups).
  9. At this point, if you go back and check the SSOX_ApplicationInfo table again, you’ll notice that the row FILE_TL_BTTestHost still has the same value on the ai_admin_group_name column referencing the original group associated with the host, not the new group we assigned to it. This causes the access denied error for the host instance, because the host user is not considered to have access to the SSODB to read the adapter settings.

The ugly part is that working around this problem requires you to delete the original Send Handler and recreating it, which requires you to move all send ports already associated with it one by one.

BizTalk 2010 Config Tool Hanging

Last week I was installing BizTalk Server 2010 on my development Virtual Machine, which previously had 2009 installed. Installation went fine, but when I started the BizTalk Configuration tool, it started hanging for minutes at a time.

Strangely enough, the tool let me configure Enterprise Single Sign-On (ENTSSO) without any problems, but would hang every time I tried to configure a new BizTalk Group. After some tests, it became obvious it was hanging when trying to connect to SQL Server to check if the group databases could be created until the connection timeout would expire.

It would indeed eventually respond again, but trying to do anything would only cause it to hang again. It was really weird that it would hang here because of the database connection, when the SQL Server used was local and ENTSSO had configured straight away without any problems.

Fortunately, I managed to figure it out: The problem was that the SQL Server instance had TCP/IP connectivity disabled (only shared memory was enabled). Enabling TCP/IP and restarting the SQL Server service fixed the problem.

Essential BizTalk Debugging Skills

On the topic of debugging skills, there are many things said and to be said, but I thought I’d take a moment to reflect on a few skills I believe BizTalk developers should develop to solve problems on their solutions effectively (in no particular order):

  1. Trace the execution of BizTalk processes end-to-end, covering both messaging and orchestrations. While this seems obvious at first, it can be tough on complex solutions that rely heavily on pub/sub and direct binding!
  2. Discovering the reasons why a service instance might be getting suspended. In particular, quickly finding any possible exception information coming from faulted instances.
  3. Using the orchestration debugger to figure out where a given Orchestration instance is getting suspended.
  4. Matching any receive shapes in an orchestration to the origin of messages coming for it. This might require matching logical ports with physical ports as well as possible direct-binding sources of messages.
  5. Tracking possible reasons for routing failures, particularly when correlations are involved. This includes being able to understand routing failure reports, checking the state of any required promoted property, comparing subscription information either directly on the message box as well as from reading orchestration code, and so on.
  6. Detecting and solving situations that stop flow/processing of messages, related to thread pool issues or thread starvation.
  7. Detecting zombie message instances and their possible consequences.
  8. Knowing how to enable the BAM tracing infrastructure to debug and solve issues with BAM not writing data to the BAM primary import database.
  9. Using the Visual Studio Debugger or WinDBG to debug BizTalk processes. This includes figuring out the right processes to attach to!
  10. Debugging assembly-loading/versioning issues. This includes how to use Fuslogvw.exe and friends.

Anyone else has any other ideas?

PipelineTesting Per-Instance Config Bug

Mark Coleman found a bug in the implementation of Per-Instance Config support in my PipelineTesting library for BizTalk Server. The problem was related to how the config XML was parsed, and would only manifest itself on certain conditions, depending on how said XML was formatted.

Mark was also gracious enough to propose the fix, which I’ve now committed to the PipelineTesting GIT repository, along with a unit test to make sure the problem doesn’t come up again. Thanks a lot, Mark!

Problem Consuming WCF Service in BizTalk 2009

Update: I’ve been told now that the hotfix in KB979153 should correct this problem.

I’m posting this one problem I ran into this week so I don’t forget about it. The scenario was as follows:

We had an existing, working WCF service. Most of the operations in the service contract were not using explicit data contracts as input parameters; instead they were just using primitives (such as an operation taking a string as an argument), though data contracts were used for the response messages.

We were trying to consume this service from an orchestration in BizTalk Server 2009 RTM, so we added the reference using the WCF Service Consuming Wizard. This worked fine. All the schemas were correctly imported and new multi-part messages were defined for the input and output messages. Schemas were also created for the inputs and outputs.

The problem came when trying to actually populate the parameters of the input multi-part messages. In this case, this needs to be done by creating the input message with all the arguments using the right schema (which we had) and then assigning that message into the parameters part of the input multi-part message that would be actually sent to the send port.


The error was that the Orchestration Compiler (or Visual Studio) for that matter didn’t seem to recognize the parameters part in the message. Though it would pop up in Intellisense in an expression shape, any attempt to actually assign a value to it would cause an error about the assignment expression being invalid (because it doesn’t recognize the part).


I immediately went back to a BizTalk 2006 R2 installation and, sure enough, I was able to do this in minutes and have it working. So far, I had failed to find anything in the docs, the readme or on searches around the neck, but I was sure it had to be something relatively harmless. Well, just so I can find it easier next time:

The workaround is to import your WCF service metadata into a separate BizTalk 2009 project, then go and change the generated Port Types and Multi-Part Message Types so that they are have public visibility, and then add a reference to the new project from the one containing your orchestration. Then it will compile and work just fine.

Annoying little bug, isn’t it?

Causing Trouble

Last night I finally downloaded the RTM build of the recently released BizTalk Server 2009 and proceeded to install it on one of my dev virtual machines. Installation went without a hitch, very consistent with my experience with BizTalk installations since the 2006 release. (For what is worth, I never ran into any issues with the BTS2009 CTP installation before that either).

Unfortunately, not everything went as well. This particular VM was running BizTalk 2006 R2, so before installing 2009 I uninstalled the previous installation. While at it, I also uninstalled Visual Studio 2005, given how I was only using it for BizTalk dev and wouldn’t need it anymore on 2009.

As they said, this turned out to be a not so bright idea: I just noticed this morning that the VS2005 uninstall trashed my Visual Studio 2008 installation. Trying to open C# projects wouldn’t work anymore, complaining about the Windows Forms Visual Studio package being missing and not being able to create the compiler services and what not.

This kind of thing can completely screw up your morning and make you waste hours trying to get it fixed. Seriously, guys: supporting side by side installations also means supporting the corresponding uninstall process. One without the other is pretty damn useless.

I’m reinstalling VS2008 SP1 now and hoping that will fix it. If not, guess I might as well nuke this VM from orbit and start from scratch. This is not a good way to start my day…

Update: Nope, reinstalling SP1 did not fix it :(

BizTalk PowerShell Scripts

I finally put my set of PowerShell script samples for BizTalk Server administration together into a single git repository over at GitHub. As a reference, here are the blog posts covering each of those scripts:

PipelineTesting Released

Last week I spent some time working on a new minor release of my PipelineTesting library for BizTalk Server 2006/R2. The new release is available in the usual places: Binary and Source Snapshot, and the Source Repository at GitHub.

The new release includes some minor documentation updates, a couple of bug fixes and a new wrapper for doing basic testing of flat file and xml schemas that provides a somewhat equivalent functionality to that offered by the TestableSchema class offered in BizTalk 2009 I’ve talked about before.

Here’s the documentation and sample for the new functionality.

Testing Schemas

It is possible to target BizTalk schemas in your testing efforts using all versions of PipelineTesting if you don’t mind doing a little bit of grunt work. This is possible for both flat file and xml schemas, and is done simply by instantiating a pipeline (either a compiled pipeline or by creating a new one from scratch) and then running your instance documents through it and checking the output.

This works particularly well for advanced scenarios involving envelopes or batching/debatching, but it’s overkill for the simple scenarios.

Beginning with PipelineTesting v1.2.1.0, there’s a new feature for easy testing of schemas in those simple scenarios. Partially inspired by the new functionality offered in BizTalk Server 2009, the SchemaTester<T> class allows you to parse/assemble a document according to a single document schema with a single method call.

To use SchemaTester<T>, simply provide the type of the BizTalk schema to use and call one of it’s static methods. The options offered are:

  • ParseFF: Parses a flat file into an XML document
  • ParseXml: Parses an XML document
  • AssembleFF: Assembles an XML document into a new flat file
  • AssembleXml: Assembles an XML document

All method come with overloads that use streams or paths to files. Here’s an example of how to use it based on one of the unit tests for this feature:

Stream input = DocLoader.LoadStream("CSV_FF_RecvInput.txt");
Stream output = SchemaTester<Schema3_FF>.ParseFF(input);
// Load resulting XML document
XmlDocument doc = new XmlDocument();

If the document cannot be converted, an exception will be raised. Remember that you must consume the resulting stream (if using the stream-variants of the SchemaTester<T> methods). You can then check the resulting exception to look into why the parsing/assembling is failure.

To make it easier to deal with the different parsing exceptions, and how to extract meaningful information out of it, you can use the ErrorHelper.GetErrorMessage() helper method.