Comments on the WF 4.0 Designer

I’ve been testing a few of the new features in Workflow Foundation 4.0 and getting to know the completely revamped Workflow Designer in Visual Studio 2010 beta 1. In a broad sense, I think it’s an improvement over the old designer in VS2005/8, but in the end I’m still far from happy with it. Here are some comments on what I like and don’t like:

What I like

  • It’s hard to argue that the new designer isn’t a big improvement in the looks category. It sports a somewhat minimalistic view, making much less use of gradients and other bling that was common in its previous incarnation.
  • The option to zoom in/out of the current workflow view is great. Not only is it useful, it’s very smooth as well.
  • The mini-map rocks when navigating a large map, and I find it particularly useful when I’m working on a zoomed designer window:
    vs10_wf_minimap
  • I like the breadcrumbs navigation list at the top-left of the designer as you dive deeper into nested activities. It could be a bit more visible, though.
    vs10_wf_nav
  • The variables and arguments windows are useful, and certainly the underlying concept is, in general, far more solid than the activity binding model supported in previous versions of the runtime.
  • The new Flowchart workflows types, and it’s associated designer is, in my humble opinion, a massive improvement over the previous models. It feels a lot more natural to write flows like this and it’s a lot easier to organize the workflow visually into something that makes sense.
  • There’s a Save As Image option in the designer popup menu that can be used to save a JPG file with an image of the workflow as it’s currently shown in the designer. It’s useful, though not thrilled that warnings/errors are rendered into the saved image.

What I don’t like

Unfortunately, this might very well be a far longer list:

The overall way of writing workflows through the designer remains extremely clunky, and it’s a pretty slow and awkward process. Maybe it’s just me, but I don’t find the overall experience very pleasant.

To start with, doing almost anything requires way too many clicks here and there. You find yourself constantly jumping between the workflow window, the variables window, the properties window and one or more popup tool windows to enter data and edit data.

To make things a bit worse, I find the lack of connections between all these to be somewhat unsettling, because no part of the process is oriented towards doing tasks but just small parts of tasks at a time. Because they are not connected across the entire process, I would find myself jumping back and forth between windows.

For example, let’s say I add a new activity and want to bind some data to one of it’s properties. When I go to the properties window and fire up the expression window, I realize I needed to move data from somewhere else and would like a workflow variable for that. So now I have to back out of the current spot, go back to the Variables window, add the variable, back to the properties window and into the expression window. Too many steps!

vs10_wf_inline Another thing I’m not a big fan of is having activities provide lots of built-in editing windows right on the designer surface. For example, the InvokeMethod<T> activity allows you to enter the expressions for Target Type, Target Object and Method Name right into the designer.

At first glance, this would seem like a good thing; after all, it saves you from a few trips to the properties window. But later, it becomes a bit more cumbersome, as it is data that you’re simply not interested in looking at the whole time. Also, most of the time the activity’s display surface will be too small to display entire values, so it’s not even that useful then.

Tool Windows

Tool Windows in the current beta are horrible to work with; they have very poor usability. I’m referring to windows such as the expression editor, the type selection window and so forth.

vs10_wf_exprwnd The first problem I ran into is that almost none of the popup windows are resizable. For most windows, this is really annoying, and what’s worse is that this includes the all around common Expression Editor window. Seriously, have we learned nothing from all the complaints around this from the BizTalk Orchestration designer?

For some windows, though, it gets even worse. Some windows, such as those that have grids, are not only not resizable by the user, but they also change size on their own. Drives me nuts!

A good example of this is the window that comes up when you try to edit the Parameters property of the InvokePowerShell activity: Initially, it contains just enough space to display a single row. After adding one row, the window resizes to show the original row and a blank new row, and keeps doing that as you add data to it. If you delete a row, the window becomes smaller again. There’s also no built-in scrolling at all, so if you need to add more rows than fit vertically on your screen, well, you’re hosed.

Another thing that gives a bad initial impression is that some windows react strangely to input from the user. For example, consider the Expression Editor Window. When it comes up, it has one size. Click on the actual text box to change the expression, and the window jumps around resizing itself twice (to end up with the initial size). Annoying.

To be fair to the Expression Editor window, user input isn’t always handled correctly in other places. For example, try to enter an expression for the default value of a new variable in the Variables window and try selecting the “All” button under the IntelliSense window when it pops up. Yep, it all goes away.

Here’s another that doesn’t make much sense to me: The Browse for Types window that comes up when selecting the type for a variable lists types by assembly, and inside each one ordered by type name, without consideration for namespaces. Yes, types are not grouped around namespaces at all.

Oh, and while we’re at it: All those new tool windows have nice and shiny white backgrounds all over the place. Considering plucking my eyes out as a workaround.

The Flowchart Designer

Flowcharts are (to me) the single most important feature in WF 4. They are a much nicer model to work with, I think. That said, the designer is not without faults:

  • Connection points are too sensitive, making them hard to target and pinpoint accurately. Sometimes links just end up going where you don’t want them to.
  • Arrows are too small, making it hard to grasp the flow at a glance (particularly when the view is zoomed out).
  • The automatic routing of arrows is sometimes quirky (though I guess it’s OK).
  • Sometimes selection isn’t handled correctly. Many times I’ve selected an arrow, seen it clearly rendered as selected, hit the Delete key only to find the entire workflow is deleted. At least Undo seems to work.
  • I love that you can resize the flowchart designer area. I hate that you have to scroll all the way to the bottom right to do it.

Variable Scope

In WF 4.0, Variables are scoped to composite activities. That is, you can define a variable either on the top level workflow, or on a specific activity, like a Sequence or a CancellationScope already in the workflow.

Unfortunately, the way this is exposed at the designer level is incredibly clunky:

  • The scope is not represented visually in the Variables window. Instead all you get is the Scope column in the variables list:
    vs10_wf_scope
    Even BizTalk 2004 got it right: It needs to be shown visually. Nested scopes are a tree, don’t try to represent it with a list; it doesn’t work.
  • The variable window will show / hide variables based on the scope. If you select a composite activity, it will only show variables declared at its scope and that of its ancestors. Variables defined in children of the activity are not included.
    I understand it’s trying to be useful, but this just gets in the way. If you want to define or look at a variable defined 3 levels deep, then you get to first navigate all the workflow to get there just to be able to get the Variables window to show you the definition. Gah!
  • When adding a new variable, scope defaults to the activity currently selected in the designer. This almost makes sense, but if you decide that’s not right, well, surprise! Scopes shown in the drop down list for the Scope column follow the same rules as above. That means you can’t add a new variable in a scope without being physically nested within it (i.e. can’t add to a sibling activity). Annoying.

Conclusion

Many of the issues I’ve brought up here are basic usability problems, but many correctable. One can hope there’s still time in the Visual Studio 2010 timeframe to see some of those UX issues addressed.

I’m not very hopeful, though, about having a designer that’s really more pleasurable to use and that addresses the bigger issues related to the experience of creating and editing a workflow more conveniently and with less clicking, as those might very well require rethinking the entire process over. It’s really more of a conceptual issue than a technical issue, in a way.

Don’t get me wrong, the new designer is an improvement over the old one. Unfortunately, that’s all it is: An improvement. It provides an interesting foundation to address some of the technical shortcomings that the original Workflow Designer has, but it doesn’t really provide a more meaningful/powerfull way of designing workflows. In that sense, I do think it’s a bit of a missed opportunity.

InvokePowerShell Activity

vs10_wf_ps I’ve been fooling a bit with the new Windows PowerShell-related activities included in Windows Workflow Foundation 4.0 alongside the Visual Studio 2010 Beta 1 release. It’s pretty cool seeing PowerShell being directly supported right out of the box!

There are two different activities: InvokePowerShell and InvokePowerShell<T>. The difference (as far as I can see) is that the latter allows you to capture the output of the PowerShell command being executed (that is, the results from the pipeline). I can understand why both variants might be needed, but not thrilled about the naming. At the least, it’s not immediately obvious what the difference between the two is, so it’s going to be confusing.

Using InvokePowerShell

I struggled a bit understanding how the InvokePowerShell activity was meant to be used. In particular, I didn’t grasp immediately the significance of the IsScript property of the activity:

vs10_wf_ps_params

If I understand it correctly, here’s the key difference: If IsScript is not checked (the default), then CommandText cannot be an arbitrary PowerShell expression. Instead it must be the name of the PowerShell cmdlet to invoke, but you cannot add any extra information there (such as the value of a parameter). By contrast, if IsScript is checked, then CommandText can be a list of (more or less) arbitrary PowerShell expressions, including variable declarations and all.

I believe that for a lot of people, enabling IsScript will make the activity behave in a more natural way.

There are a few more interesting details about the PowerShell activities:

Passing Data into PowerShell

There are two ways to pass data into the PowerShell script to execute:

  1. If you’re executing a cmdlet, then the Parameters collection can be used to provide input arguments to the cmdlet call. For example, if CommandText is “Get-Process”, then you can add a Name parameter to the Parameters collection with the value set to an expression returning a string, and it will be the same as if you had invoked “Get-Process -Name <value>”. I’m not 100% sure yet, but it would seem like this would only really apply if IsScript is unchecked (otherwise, it’s not very clear what you’re passing the parameters to).
  2. You can have WF push variables into the PowerShell runspace before the invocation. This is what the PowerShellVariables property is for. If you add a variable there call Error, then the PowerShell expression in CommandText can reference the value by using $Error. Again, this would appear to be mostly useful when IsScript is checked.

Based on this, one might suspect that the Parameters and PowerShellVariables properties are basically there to support both modes of operation of the activity: One when invoking a cmdlet directly and one when executing a more free-form script. Is this good? I’m not sure yet. I can see a sort of logic to both ways of using the InvokePowerShell activity, but it seems a bit complex and potentially confusing. I sure know I was confused the first time I tried to use it.

Pipeline Input

It should be obvious by now that I wasn’t exactly right when I said there were two ways to pass data into the PowerShell script. There’s a third (main?) one: Provide an object (or collection of objects) as the pipeline input, which is what the Input property of the activity is for.

It should be fairly obvious how to use it: If the command is a cmdlet, then the input will be feed just like if you had piped something to it in PowerShell. You might expect that if the command is a free-form script, then you’d be able to access it the inputs $_; however, I haven’t been successful in that so not sure if it’s supported.

Pipeline Output

Handling pipeline output is only done in the InvokePowerShell<T> activity, and it’s basically very simple: When adding the activity to your workflow, the designer will ask you to bind T to a type, and that’s what you expect your PowerShell command to return. I can imagine that in most cases, this should be some sort of collection (like IEnumerable) since most PowerShell commands will return multiple objects.

Then, just bind the Output property of the activity to a variable in your workflow that will receive the pipeline output after the activity executes.

Pipeline Errors

Errors resulting from executing the InvokePowerShell activity can surface in one of two ways:

  • Certain errors, such as trying to invoke a malformed expression or a non-existent cmdlet will result in an exception being thrown and the Workflow faulting, unless you have the activity inside try/catch blocks.
  • Anything that gets written by the command to the error stream (through the Write-Error cmdlet) will get exposed through the Errors property of the InvokePowerShell activity. To handle those, you’ll want to create a new variable in your workflow (of type Collection<ErrorRecord> or just IEnumerable), and then bind it to the Errors property.  Now you should be able to simply iterate over your variable right after InvokePowerShell finishes executing.

Designer Support

I’m not totally happy with the designer support for the PowerShell activities. Besides my general complaints about the Workflow Designer (a subject for another blog post), there’s something that really bugs me: There’s no proper editor support for entering the CommandText in the activity.

Sure, you can attempt to write your script directly into the text editor in the activity designer, but that’s way too small for anything more complex than, say, calling Get-Process. You could also attempt to use the property window, but that’s even worse because all it has is a single-line text box. At the very least, it needs to popup into a multi-line text editor window.

The Future of BizTalk/WCF/WF

Walter Michel commented on my recent entry on the BizTalk and WCF Messaging Models and asked if I had some guesses as to what the architecture of a future BizTalk version might look like in regards to WF/WCF integration. I think that, at this point in time, it is anybody's guess what it might look like, as it is just too early to tell.

Honestly, I don't know what it might end up looking like, and I can't even say I can make an educated guess (and there are people with far more criteria for making such educated guesses than me, such as Jon Flanders). Instead, I'll address the question in a different way, by mentioning what I consider a few challenges I see regarding the integration and other things like that.

The Impact of Significant Architecture Changes

One key aspect of the whole integration process between the three technologies is what the impact of significant changes to the core BizTalk architecture. Microsoft already went down that path during the transition between BizTalk 2002 and 2004, and while those of us developing BizTalk solutions are extremely grateful for the benefits that the new architecture brought over, companies with large investments in BizTalk 2000/2002 solutions were not so happy. For most of them, going to BizTalk 2004/2006 meant a complete rewrite of their integration solutions, and that can be very expensive, and though to justify to the business side, not only because of cost, but also because of the associated risks.

With BizTalk 2004/2006, BizTalk brought over not only a significantly revamped platform, but also a new, effective, and very welcome extensibility model that has been tested with time. That's why the current core product architecture has been good for three release cycles already, and the fact that even complex things like WCF could be integrated without significant changes to it is pretty substantial.

A lot of Microsoft clients and partners have made significant investments in that architecture and its not clear how that investment might be preserved (or at least their loss mitigated a bit) if significant architecture changes happen. I'm pretty sure the people at the Connected Systems Division at Microsoft are aware of this, and from my own, outside perspective, I can only say that it seems like a daunting and complex decision to make. Whatever way they decide to go, it's not going to be easy.

So, what happens if the complex changes do become a reality? We can only hope that good migration tools are not left by the side; but even with those migration tools, you'd probably be looking at a complete rewrite scenario again for the most part.

Messaging

In my previous entry, I commented that the current WCF integration in BizTalk 2006 R2 leverages the existing BizTalk messaging extensibility model (the adapter framework) on top of the messaging engine. However, I think that in the long run, keeping both is probably not the best solution. Despite their differences, there's still a lot of common ground and responsibilities between the BizTalk Messaging Engine and the WCF model, and simply keeping both models in place, with their independent extensibility models, would be too expensive (for example, keeping both the BizTalk adapter framework and the WCF transport channel model).

So my best guess is that, at some point in time, one of them would tend to disappear, and needless to say, that would have to be the BizTalk one. Does it mean that the BizTalk Messaging Engine will disappear? Probably not, but it will change and its responsibilities will adjust to that.

For example, assuming that the core message moving responsibilities are absorbed by the WCF stack, then the Messaging Engine would concentrate on it's other tasks: Managing Receive Locations/Ports and Send Ports/Port Groups (assuming those concepts still exists in some fashion), managing port configuration settings at runtime, managing errors and, assuming the Message Box concept remains (or some equivalent centralized routing agent), then submitting and retrieving messages from it.

There are some interesting aspects deriving from this:

  • The pipeline model would disappear. Personally, I'm a little saddened by this because I consider the pipeline extensibility model to be simple [1], elegant and very effective; and it's one of the great assets BizTalk has when integrating against legacy and third party systems with peculiarities in the messages they generate/consume. That said, WCF does have a good extensibility story, though the responsibilities of where each things happened are a bit more diluted across the stack, instead of being centralized like in the BizTalk pipeline concept.
  • The WCF model has some bias towards XML/SOAP. Certainly, it allows other things as messages, depending on how much work you're willing to invest in it. This is changing a bit, though, in .NET 3.5 with the REST support, and I expect it will change even more in later releases. To be an effective messaging stack for an integration platform, however, this needs to be a non-issue.
  • WCF supports a more complex set of interactions (and MEPs) than those supported by BizTalk at this time, and it's not clear how some of those could be used to extend BizTalk. This includes things like composite duplex channels, as well as the durable services/sessions model supported in .NET 3.5 (the current implementation of which makes little sense if the service is backed by a real workflow in WF/BizTalk).

The Message Box

An interesting question that comes up is whether a future BizTalk version might still have the Message Box. Currently, the MessageBox is the cornerstone and heart of the BizTalk architecture, and a key element in supporting BizTalk's highly reliable, loosely-coupled, asynchronous model.

However, I don't discard that, if a major architecture change is done, the MessageBox model isn't reevaluated, because it also is a disadvantage at times. For example, it can impact message throughput and latency in low-latency, synchronous scenarios, and after a certain point, the centralized model doesn't scale all that well without some really heavy-duty DB servers (which is expensive).

I don't think the message box concept will go away soon, but it may change in how it is implemented. I also think (but this is just a wild guess) that we might see some support for more lightweight operation modes were you trade off some of the features of the MessageBox for lower latency for purely synchronous operations (this would be interesting, particularly for some messaging-only scenarios).

The Orchestration Engine

As far as I know (at least from the little tidbits MS has let out), XLANGs is going away to be replaced by a WF-based orchestration engine. At the least, this means that some migration tools would be needed here, possibly with a set of custom WF activities that match more closely to the behavior of the built-in XLANGs shapes in current BizTalk versions.

There are some interesting implications and questions coming from this change:

  1. One of the core advantages that WF has over the current BizTalk Orchestration model is that it is extensible through custom Activities. However, as we've already seen, some custom WF hosts will restrict the set of activities that can be used in Workflows executing on them. Whether BizTalk will have an open or closed model here is anybody's guess.
  2. WF has one low-level mechanism for getting data into a workflow: Workflow Queues (getting data out of a workflow isn't quite as clear). While the queue model is very flexible, it is also too low-level for what most people are comfortable with. Unfortunately, the current WF releases don't really have a very good higher-level mechanism for this (the HandleExternalEvent and CallExternalMethod activities are woefully inadequate). In other words, the existing Port model in BizTalk is a far better model, which the new WCF integration into WF in .NET 3.5 is sort of related to (at least they are much more similar).
    This is, in a lot of ways, a key point in making WF usable as an XLANGs alternative. The current port model in BizTalk is tied to the BizTalk MessageBox, and it is instrumental in the design of loosely-coupled orchestrations and services, scaling processing to multiple hosts (via the BizTalk Host model and bindings), and versioning complex long-running processes (direct binding is extremely useful when breaking up complex processes into multiple orchestrations and letting each one be versioned independently).
  3. A key difference between WF and XLANGs is the fact that XLANGs is message oriented. This means that messages are a first-class language construct, and so message construction, manipulation and the receiving/sending of messages is a clear concept. WF, on the other hand, has no clear of what a message is at all, and execution is not related at all to how data (messages) flows around the process. On one hand, this is an advantage because it opens new possibilities and adds expressiveness to the tool, but it is also a downside because the message-orientated nature of BizTalk and BizTalk orchestrations has been one of BizTalk's strongest points and really aligns it with Service-Oriented Architectures.
    In fact, even in the WCF integration in .NET 3.5, the concept of message isn't directly mentioned; instead we talk about [data]contracts, which are still just CLR objects. If this remains so, then an interesting follow-up question to consider is what happens to the existing BizTalk features that are closely related to its message-oriented nature, like maps and the support for non-XML message formats (including EDI and Flat Files, but not limited to them). There are certainly alternatives for this, and I do doubt we'll see a significant reduction in functionality in this space, though how it is accomplished could change significantly.

Human Workflow

Traditionally, BizTalk hasn't been oriented towards automating processes with heavy user-involvement (the stronghold of more workflow oriented tools like K2.net or Ultimus). With BizTalk 2004, Microsoft made a weak attempt at providing some tools for building these kind of solutions as part of EAI projects with Human Workflow Services (HWS), which were deprecated on BizTalk 2006.

WF has more potential in this arena, although the core WF is too basic to build human workflow solutions out of the box without a large number of extensions. I don't expect a future BizTalk version to incorporate more Human Workflow oriented features; given that Microsoft has said they are investing in positioning Microsoft Office Sharepoint Server as a contender in that space. We might see some more BizTalk to Sharepoint integration tools, though.

Business Rules Engine

Both BizTalk and WF have some business rules functionality. While the BRE in BizTalk is, in general terms, more advanced than WF's, it's not clear exactly what's going to happen. I seem to remember reading that both engines are now owned by the same dev team, so we should certainly be seeing some convergence there.

What is not clear to me is whether there will continue to exist two separate engines (one for the core WF and one for server oriented products like BizTalk). If both continue to exists, then that does mean a bit more confusion for BizTalk developers as it will mean having to choose on a case by case basis which of both engines to use from orchestrations (now WF workflows). If they converge into a single rules engine, then I would expect the WF engine to absorb, one way or another, the extended functionality offered by the BRE.

Conclusion

The only conclusion I can make of this is that I have more questions than answers :-). It's certainly going to be interesting to see how the core framework evolves and how that impacts the architecture and functionality of BizTalk as a standalone server product, and for us BizTalk developers it will sure have some significant impacts. The best I can say is that learning a bit about WCF and WF and understanding their architecture now sure will help when the time to make the transition comes along.

[1] The only reason writing pipeline components is complex at times is because of the COM-compatible interfaces you need to implement to satisfy the old, internal unmanaged implementation in the Messaging Engine, as well as the need to do streaming as much as possible to minimize the performance impact.

Workflow Types and Prototypes

Windows Workflow Foundation creates workflow instances either from CLR Types describing the workflow, or from an XmlReader that provides the Workflow description in XAML. In either case, I've always viewed the entire workflow definition + creation as a variation of the prototype pattern. It's a loose interpretation, of course, but it rhymes with the idea that either the CLR type of the XAML workflow are basically ways to describe the initial state of a WF workflow before execution (i.e. they are templates).

Even at runtime, each activity in the workflow definition is used as a prototype to create instances of it to execute in a specific workflow context (see spawned contexts). Cloning of the prototype instance is done in WF by serializing the activity into an in-memory stream and loading it back out into a new CLR instance.

All of this works very well in most cases, but something I was trying to do recently led me to wish the WF team had provided a third way of specifying a workflow definition: As an actual CLR instance of an activity tree; which, after all, is what WF does with the CLR type you provide. This would've been a pretty interesting way of dynamically generating workflow definitions directly through code, without having to resort to either using self-modifying workflows or generating XAML at runtime.

System.Transactions and Workflows

Scott Bellware discusses some interesting things here regarding transactions and extensibility hooks and even considering Ruby on Rails as a web front end. I'll ignore the Ruby thing, but I do want to talk about his comments on transactions and workflows.

If I understand right, Scott is basically providing a way to extend his application through user-defined workflows that are executed at specific and controlled extensibility points. This is a very cool scenario for WF that enables very interesting possibilities.

His concern, however, seems to center around controlling what happens when one of those user-defined workflows fails, to ensure the entire system is not left in an inconsistent state. This is certainly a valid concern, but it is one that I feel isn't quite as simple to solve as simply "use transactions".

Let's take WF out of the picture for a moment and assume we were using a good old code-based extensibility model (or even a script-based one). Even with the help of System.Transactions and distributed transactions, there's really no way to guarantee that whatever code users of the system put into the extensibility hooks would still work correctly in the event of failure. This revolves around the fundamental fact that there's no guarantee that whatever is put in there is even transaction-aware. Granted, there's a big chance that code put in there is indeed transaction aware, but that's only because a lot of what comes directly into the core .NET framework is tightly integrated in itself. Indeed, for all you know, the user might have even explicitly bring out his operations out of the transactional context (for example using "Enlist=false" in his connection strings).

So even if you wrapped your extensibility hooks with a transaction scope built with System.Transactions, there's no guarantee that a transaction rollback will really bring the system to a consistent state. That's a constant danger with extensible systems, and one that it is not easily addressed. If you've noticed, most framework facilities focused on extensibility only concern themselves with the security problems, but not with extension behavior in the general sense.

This fundamental issue doesn't really get any better just because your extensions are now written as workflows. Indeed, it can become a bit more complicated. To me, using System.Transactions with WF Workflows used in this matter doesn't really make much sense, for several reasons:

  1. Yes, WF supports System.Transactions (indeed this is what a TransactionScope activity uses underneath), but the same danger exists of someone using an activity that's not transaction-aware.
  2. Wrapping the entire workflow in a System.Transactions transaction limits substantially what the user can do with the extension workflows. Are you willing to restrict your users to running only short-lived, simple workflows?

The second one is to me the most significant one, because putting that restriction in place limits substantially the power of WF for these scenarios, particularly when you allow workflow persistence and tracking as part of your application extensibility.

The one benefit that WF gives you over code-based extensibility for handling errors is indeed mentioned by Scott: Compensations. This is a very powerful mechanism, particularly for long running workflows, and can certainly be leveraged by extension workflows to put the system back into a consistent state.

In the end, just like with code-based extensibility, you still need to put guidelines in place to make sure users extending your system do so in such a way that they don't jeopardize the behavior and consistency of the entire system, but even with WF, it is still hard to put those guidelines in executable form.

In some cases, however, WF can indeed make it easier, if you're willing to live with some restrictions. For example, here are some ideas to think about to improve your chances:

  1. Restrict the vocabulary: WF can be extended through the use of custom Activities. Ideally, you will have devised a set of custom activities specific to your problem domain that the users can use to compose their extension workflows. If you're willing to restrict users a bit, and your activity set is complex enough, consider putting restrictions in place so that users can only compose extension workflows using a set of activities you've "approved" (those on your own set and some basic flow control ones, for example). This won't be a perfect model, but can improve the chances that users don't extend the system incorrectly, and can easily be done.
  2. Use validators: Take advantage of WF's ActivityValidators to enforce rules on your extension workflows. If you've also restricted the vocabulary, you can certainly take advantage of domain=specific knowledge to write the validation rules. For example, if you've got an UpdateXXX activity, you could validate that it always is used within a TransactionScope activity.
  3. Create templates your users can use to get started writing extension workflows. Ideally, make these into top-level workflow activities your users can derive from. So, for example, you could have users create a workflow derived from InsuranceExtensionWorkflowActivity instead of a simple SequenceWorkflowActivity, Your base workflow could also have custom validators enforcing, for example, that a global compensation flow is implemented by the users (if necessary).

None of these are foolproof, but they can improve the odds in your favor when used correctly.

Workflow Services in Orcas

One of the new power features in .NET 3.5, sorely missing when .NET 3.0 was released is the integration between Windows Communication Foundation and Windows Workflow Foundation, in the way of the new Workflow Services model. Besides providing an easy way to invoke WCF services from WF without resorting to using BasicHttpBinding on the services or invoking a [wrapper] client class form a code activity in your workflow, it also provides a way to expose a workflow as a WCF service to the outside.

The second functionality is the one that, naturally, interests me the most. After looking at it for a while, I've made a few observations that I'd like to share.

Implementing Services

A very interesting thing is how you implement services as workflows. If you know how to create WCF services the right way, then you [mostly] know how to implement services as workflows. The reason for this is that most of the tasks you'd normally when creating a WCF service still apply when creating a workflow service. That is, you will still declare your service contract (interface) as well as your message/data contracts. The only key differences really is how you implement that contract.

When using workflow services, you implement a contract by using the new Receive activity introduced in .NET 3.5, and select which operation of the contract you'd be implementing. You still need to do the activity binding dance for parameters as well as (possibly) create response messages for the incoming requests. Basically, the Receive activity works as a kind of sequential activity where the child activities represent the tasks to execute after the request message is received until the response message is sent (if this is a request/response service).

You'd also need to do a lot of the usual work in setting up your service host, only instead of using the regular ServiceHost class you'll be using the new WorkflowServiceHost class. Guy Burstein has a good entry on getting started on exposing workflows as services here you might want to check out.

Starting Workflows

One very interesting option in the Receive Activity is the CanCreateInstance property. You can use this property to have the WorkflowServiceHost automatically create and start a new instance of your Workflow when a new message arrives at the service endpoint for the implemented operation, without having to write manual code to receive the message and create the new workflow instance.

People familiar with BizTalk 2004/6 will recognize this option as being similar to the Activate property of the Receive Shape in BizTalk orchestrations. It also serves, in a way, a similar purpose as the CanCreateInstance property of the new [DurableOperationBehavior] attribute introduced for WCF in .NET 3.5 as part of WCF Durable Services infrastructure. Jesus Rodriguez has an excellent article introducing this new feature.

Something I found a bit curious about the CanCreateInstance property of the Receive Activity is that there's currently no validation around where you set it. As far as I can see, it doesn't make much sense to use it unless the receive activity is the entry point of your workflow, but currently nothing prevents you from setting it if is not. That said, I guess it might be difficult to validate this correctly given the WF execution model.

Continuing Workflows

The Receive Activity doesn't limit you to starting new workflow instances; it also allows your workflow instance to wait until a message is received (through a service invocation from a consumer) to continue execution. This is a key scenario for long-running processes because the workflow will have all the necessary context within the business process to interpret the coming message correctly and act on it.

How does the host know which workflow instance should process a given message, then? The answer lies in the binding context. WF requires the consumer of the service to send a token alongside the request message that tells it what workflow instance the message is for. This token is, essentially, the WorkflowInstanceId, and can be sent either as an HTTP Header or as a SOAP Header.

Using the WorkflowInstanceId as the instance selection token is a logical choice for WF, as the Workflow Runtime, as well as the persistence service, already have clear knowledge of what the ID means and how to use it, so getting the right workflow instance to pass the message to is basically just a call to the GetWorkflow() method of the WorkflowRuntime object.

For this entire process to work, both the server and the client must use the special WSHttpBindingContext class, which helps makes this process a bit more transparent (particularly for the workflow service). On the client side, naturally, the service consumer must be, somehow, aware of the correct workflow instance ID to use as the token. Sometimes, this will be handled in a very transparent way by the infrastructure, but sometimes you'll need to make it more explicit. And by the way, the WSHttpBindingContext class is also used for WCF Durable Services.

Serge Luca has an extensive sample of using WF Services in several scenarios that is pretty interesting to check out.

The Limitation of IDs

This entire mechanism works fine for WF Services. However, after I saw this the first time, there was something that I didn't quite like about it, and it kept nagging in my head. A few days ago I finally realized what it was: As a first option for having this feature, it is pretty useful, but it is, in a way, a significant step back.

The problem with using the WorkflowInstanceId as the linking element between the service consumer and the workflow implementing the service is that it has, essentially, no business meaning. While it is a very useful technical discriminating ID, most business scenarios where this kind of long running services and processes are involved already have natural correlating identifiers in the problem domain, like a purchase order id, a loan id, or a combination of customer id and tracking id.

There are several implications of this simple fact:

  • Consumers of the service become aware of the fact that there's a workflow on the other end of the service. This is actually merely an implementation detail of the service which should be transparent to the consumer as much as possible.
  • Consumers might now be forced to track yet another id, and one that has no business meaning at all. So instead of just having, say, the order id around they also need to find where to store the workflow id.
  • The WorkflowInstanceId becomes an added part of the service messaging, but it is one you might not always be aware of because it doesn't make part of the core messaging description.

Of all of this, I think the second one is what bothers me the most. If you're ever used the wonderful Correlation Set mechanism introduced in BizTalk 2004, then you'll definitely see where I'm coming from, as this is a bit more restrictive and less expressive than what we can do right now with BizTalk. In fact, this is more akin to the limited correlation mechanism we had back in BizTalk 2002 (though that required jumping through a lot more hoops).

Conclusion

All in all, this seems like a very welcome addition to both WCF and WF, even though there are limitations in this first release. I'll be watching closely how both develop as they take a bit more of the core messaging and orche stration strengths that Biztalk has enjoyed for the past few years.

Various Things

I've been a bit busy lately and with a lot of things on my mind and I've neglected this poor blog for a few weeks now. I promise I'll pick back up soon and I've got some ideas for some future posts, but they will have to wait for a better moment. Meanwhile, here's an assorted list of thoughts and links:

  • If you have to create Office Add-Ins using .NET, then give these guys a serious look. Their documentation isn't too good, but the stuff works very nicely, it's not hard to pick up, supports several Office versions, and, as far as I can see, beats the crap out of using VSTO.
  • JetBrain's UnitRun Visual Studio Add-In for running unit tests is pretty decent overall and I use it quite a bit. It's obviously not quite as full featured Jamie Cansdale's TestDriven.Net, but that's OK. Big downside? Becomes useless as soon as you have an [ExpectedException] attribute somewhere.
  • My good friend Sam Gentile continues his series of posts on Windows Workflow Foundation. Sounds like Sam finally saw the light and gave up on those pesky HandleExternalEvent and CallExternalMethod activities ;-).
  • Be suspicious if you find that running an entire suite of tests in the NUnit GUI tool takes significantly less time than you know just one of the integration tests included in the suite should take. I'm still not sure why I'm seeing this, though...
  • If you want to add me to your MSN Messenger contact list using the address on my blog, and you're not sure if I would recognize your address, please shoot me an email first. I'm happy to add new contacts I've got stuff in common with, but unless I can quickly find out who you are and why I should add you to my contact list, I'll probably just end up blocking you.

Activity Binding Limitations

One of the core features of Windows Workflow Foundation that enables authoring workflows in a declarative fashion [1] is activity binds, which enable you to connect inputs of one activity with the outputs of another (or properties of the workflow itself), saving the developer from having to write custom code to move data from one step of the workflow to another manually.

Activity Binding is actually a pretty powerful feature and works fine in many cases. Unfortunately, it also has a few limitations. One of the limitations is the very basic support for binding against items in a collection.

To clarify, you can most certainly bind a property of an activity that has a collection type (say, a List<T>) to an property of another activity of the same type. However, there's no easy way to bind, say, a property of an activity of type System.String to a specific item in the collection exposed by another activity's property (for example, binding it to the fourth item in the list).

One thing that surprised me a bit a couple of days ago was that, actually, the core ActivityBind class almost supports this, it's just not exposed through the Workflow designer in Visual Studio. For example, if Activity a exposes a property of type Dictionary<string, string>, you can bind a System.String property against it with an expression such as this:

Activity=DictionaryWorkflow, Path=Phones["Mary"]

Notice how the last part actually especifies the exact item against which to bind. While a cool trick, this isn't nearly as useful as it may appear at first since a) it requires the user to manually and carefully edit the binding expression in the Properties window in the designer, bypassing the Activity Bind dialog and b) it's hardcoded.

One could imagine an extended support for this kind of operation having the ability to allow the indexer to be represented by a nested activity binding expression, though it's not clear yet to me how that would work out. Don't expect something like that anytime soon, btw.

The way I've worked around this limitation in the past has been to create my custom activity that needs the binding with two different dependency properties: One of a collection type, and one of the type of the indexer to bind. This way I can bind the collection property itself directly to the entire collection, and in the second property specify the item I want to extract the value for. In the example above, my custom activity would have two properties then: 1) PhoneDirectory and 2) PersonsName. Using those two at runtime the activity would have all the information needed available.

This works very well, but it's less explicit, it requires writing more code, and worse, does not work if you have to work with an existing activity you can't modify. In the latter case, I've sometimes used wrapper adapter activities, but it can be quite a bit of work (depending on the complexity of the activity being wrapped) just to get this functionality.

Dynamic Properties

A second limitation in the current activity binding mechanism is one I've already heard a few people complain about: You can't bind against a dynamic property. This means that if you have a custom activity that exposes a pseudo property at design time by implementing IDynamicPropertyTypeProvider and a custom activity designer, there's no way (at least that I've found) to bind a property in a different activity against it. 

Or at least, if there is, I haven't found a way to get this to work. Surprinsingly enough, you can do the opposite with attached dependency properties, though the syntax is a bit more awkward (and it does indeed subvert the mechanism a bit).

[1] Not that this is any easy at all to do in many cases with WF...

BPEL support for WF

Paul Andrew announced recently a coming CTP [1] that will provide BPEL 1.1 support to Windows Workflow Foundation. This will provide a set of custom WF activities that map to BPEL 1.1 constructs as well as tooling to import BPEL processes into WF XAML workflows and export the other way around.

I think overall this is very good news and, as Paul says, highlights the power and flexibility offered by the domain-neutrality of the WF runtime. Paul also mentions that this will, in the end, be supported in a future release of BizTalk Server, once the orchestration engine is built on top of WF, and by that time the toolkit support should have already been expanded to provide BPEL 2.0 compatibility in both WF and BizTalk. Pretty exciting stuff which I'm sure will provide some good selling points for both products (even if many people don't use it eventually).

One important aspect of this that is interesting to consider: the WF approach to writing workflows means that, if BPEL compatibility is an aspect you care about, you'll need to restrict yourself to a subset of the power of WF in order to remain compatible (and meaning being able to later export your WF workflows into BPEL). This is something I completely expected and understand (and even welcome, to a point), but it's something some people might not realize right away.

[1] Am I the only one getting lost along the miriads of CTPs coming out of Redmond lately? Seems pretty imposible to keep up if you actually have a day job!

Programming Windows Workflow Foundation

I got a review copy of K. Scott Allen's "Programming Windows Workflow Foundation: Practical WF Techniques and Examples using XAML and C#" book and I just finished reading it recently, and wanted to take a moment to mention my opinion about it.

First of all, I'm probably not quite Scott's target audience, seeing how I was already familiar with WF before getting into the book. From my point of view, the book aims to provide a quick introduction to what WF is and how to get started programming it. In that aspect, I think it does a pretty good job.

The book starts by quickly introducing what WF and what it is composed of (the runtime, activities, designer, XAML and so on). Chapter 2 then talks in a bit more detail about how to create and design workflows, covering both the principal aspects of how the VS designer works, as well as writing workflows in code and XAML. But this chapter also goes a bit further and covers using the workflow compiler both from the command line (wfc.exe) as well as programmatically and XAML workflow activation.

Chapter 3 is dedicated to writing sequential workflows, though it also covers basic host <-> workflow communication using the workflow runtime events, workflow parameters and external data services. The latter are taken up in more detail in chapter 4, which covers most of the basic activities in the base activity library.

Chapter 5 introduces how to write custom activities. While the cover here is somewhat light (as expected given the breath of the topic), Scott does quickly cover important topics like activity binding and dependency properties and gives a quick look to readers of how to write custom activity designers and validators. Chapter 6 covers the hosting facilities in WF, including how to correctly use the WorkflowRuntime class, and all the built-in runtime services, including scheduling, persistence and tracking.

Chapter 7 covers State-Machine workflows, including some simple examples of how to interact with the workflow from the host (like quering possible transitions from the current state). Chapter 8 covers host <-> workflow communication in detail, including the use of correlation tokens, role authorization and an explanation of workflow queues. Finally, Chapter 9 covers the rules engine in WF and using code and rule conditions in activities.

As I said earlier, I'm probably not the the target reader for the book. Many of my readers probably won't also, so if you're looking for a comprehensive, in depth look at WF, you'll want to find some other source.

However, if you are not familiar with WF yet, I believe Scott's book is a pretty good place to start. Here's why:

  • The book is short, about 230 pages. Most people should be able to go through the book pretty quickly.
  • Scott's writing is very much to my liking. He's direct, goes straight to the point without any B.S. and he's style is pretty engaging overall, so you won't get easily bored. Chapters are also very reasonably paced and good in length, so you can very easily read it chapter-at-a-time (I hate books with never-ending chapters!).
  • Even though the book is short, Scott still manages to cram a lot of useful information and cover a lot of topics (some in much more detail than I honestly expected) and I think its fair to say he does a good job of putting all the options and facilities in WF in front of his readers.

While on the topic of introducing WF, my good friend Sam Gentile wrote a couple of days an introductory post on WF which is pretty good. Like others, I'm also looking forward to seeing Sam share a bit more of his findings and what he has learned about the platform just as he has in the past with WCF and other technologies.

Look for my take on Dharma Shukla's and Bob Schmidt's "Essential Windows Workflow Foundation" book in a post near you!