Testing Pipeline Components

As many of you know, there are basically two ways to test custom Pipelines and custom Pipeline Components in BizTalk Server:

1. To create them and test them by using them "for real" in biztalk messaging scenarios: i.e. configure ports to use your custom pipelines and feed messages through them.
2. Use the pipeline.exe tool included in the BizTalk SDK to run them standalone.

Option 1 is required, definitely a needed option, but it is inconvenient, at the least, for agile development: it is slow, cumbersome, and hard to automate. Option 2 is easier and faster, but it's also inconvinient to automate.

So I had been looking at ways to automate the testing of pipelines and pipeline components, and ran across this article, explaining how to use mocks to unit test pipeline components. Neat, but inconvenient as well. Furthermore, it does not allow you to test pipeline components inside a real pipeline, which is sometimes necessary to ensure your component works well with the built-in components in BizTalk (disassembler in particularly can be pretty picky).

I realized then that MS already provided most of what you needed to test pipeline components in a better way, and one which was possible to embed inside NUnit tests (or whatever unit testing framework you use): Pipeline.exe relies on a set of helper components in the PipelineObjects.dll assembly included with the SDK, which does the heavy work of mocking the most important BizTalk objects that execute pipelines, such as IPipelineContext, IBaseMessage and IBaseMessageFactory and so on. I figured that, with a nicer API built on top of this, I could come up with something that really made it easier to test your pipeline components in a more agile manner, and that complemented the core unit tests you are creating for internal functionality.

So I've been working on this for the past couple of days and I've already have a working implementation. Here's an example of what it might look like:

/// <summary>

/// Tests that we can execute successfully a loaded pipeline

/// with a flat file as input

/// </summary>

[Test]

public void Test_ExecuteOK_FF()

{

   ReceivePipelineWrapper pipeline =

      PipelineFactory.CreateReceivePipeline(typeof(CSV_FF_RecvPipeline));

 

   // Create the input message to pass through the pipeline

   Stream stream = DocLoader.LoadStream("CSV_FF_RecvInput.txt");

   IBaseMessage inputMessage = MessageHelper.CreateFromStream(stream);

   inputMessage.BodyPart.Charset = "UTF-8";

 

   // Add the necessary schemas to the pipeline, so that

   // disassembling works

   pipeline.AddDocSpec(typeof(Schema3_FF));

 

   // Execute the pipeline, and check the output

   MessageCollection outputMessages = pipeline.Execute(inputMessage);

 

   Assert.IsNotNull(outputMessages);

   Assert.IsTrue(outputMessages.Count > 0);

}

The code above does the following:

1. Loads a receive pipeline, which in this case contains a Flat File Disassembler
2. Creates a new input message with a Flat File loaded from a resource stream
3. Loads the Flat File Schema the FFDasm will use to parse the message and makes it available to the pipeline
4. Execute the pipeline and checks the output.

It's cetainly not  a lot of code, and personally I think the API is looking pretty nice right now. So far, I got it working with the following scenarios:

• Support for both receive and send pipelines
• You can create pipelines programatically, by adding components to any stage, or load an existing biztalk pipeline (the preffered method) using its type.
• You can make schemas known to the pipeline before executing it by loading the necessary biztalk schemas out of a biztalk assembly. I've tested it already with both XML and Flat File schemas, schemas with one or multiple roots, schemas with promoted properties, and envelope schemas.
• Both debatching receive pipelines and batching send pipelines are supported through the API.

Here's another example, this time of doing multiple document batching into an envelope using the XML Assembler:

/// <summary>

/// Tests we can execute a send pipeline with

/// multiple input messages and an envelope

/// </summary>

[Test]

public void Test_ExecuteOK_MultiInput()

{

   SendPipelineWrapper pipeline =

      PipelineFactory.CreateSendPipeline(typeof(Env_SendPipeline));

 

   // Create the input message to pass through the pipeline

   string body =

      @"<o:Body xmlns:o='http://SampleSchemas.SimpleBody'>

         this is a body</o:Body>";

   MessageCollection inputMessages = new MessageCollection();

   inputMessages.Add(MessageHelper.CreateFromString(body));

   inputMessages.Add(MessageHelper.CreateFromString(body));

   inputMessages.Add(MessageHelper.CreateFromString(body));

  

   // Add the necessary schemas to the pipeline, so that

   // assembling works

   pipeline.AddDocSpec(typeof(SimpleBody));

   pipeline.AddDocSpec(typeof(SimpleEnv));

 

   // Execute the pipeline, and check the output

   // we get a single message batched with all the

   // messages grouped into the envelope's body

   IBaseMessage outputMessage = pipeline.Execute(inputMessages);

 

   Assert.IsNotNull(outputMessage);

 

   using ( StreamReader reader = new StreamReader(outputMessage.BodyPart.Data) )

   {

      // d contains the entire output message

      string d = reader.ReadToEnd();

   }

}

One important difference with working with the API I'm creating and pipeline.exe is that the API is meant to be used with existing biztalk artifacts, which you can create with regular biztalk projects (though they don't need to be deployed to use them). Pipeline.exe, on the other hand, works with the raw *.btp and *.xsd files, which is simpler for some things, but far more inconvinient for testing, at least in my opinion.

I should be ready to post this in a few days, as I still need to test it more throughly and fix a few things, but if anyone is interested, let me know and I'll pass on what code I have right now. Do note I'm implementing this for BTS06 only, though I see no reason why you couldn't port it to BTS04!