SOA vs. Service-Oriented Applications

[Updated 16-Oct to fix a typo]

Service-Oriented Architecture (SOA-arch) and Service-Oriented Applications (SOA-app) are different animals with a common name. I think this is a major point of confusion that hasn't yet been widely recognized. Just like the fact that three points of view exist about what "workflow" means. Here is the difference and I've also included advice for building a service-oriented application.

Service-Oriented Architecture

Here is a quick SOA-arch diagram I built after looking at various resources on the Web:

Service-Oriented Architecture Layers

Those like me who work on commercial ERP applications find this a little humiliating. All that work to build an enterprise application – and we wind up at the bottom of the SOA stack? We clearly have thin skins. Moreover, this implies that one of the chief reasons that SOA-arch exists is because apps can't integrate with each other and we have to fool them into working together. SOA-arch also exists to crack apart applications and cobble them back together to match a business process not envisioned by the application's authors. It's an expensive and invasive process.

ESB providers will eternally demand credit for inventing SOA-arch. But I think we should think about the "day after" effect of web services. People began putting SOAP layers around their existing applications or building new applications from scratch on the WS-* stack. But getting some type interoperability only solves one of the many problems around integrating complex apps into a specific business. There was nothing SOAP could do to match an SAP-formatted employee record to an HR-XML version. SOAP can't make my application email the HR department if I browse http://www.monster.com from my desk. SOA-arch puts those things together. SOA-app gives enterprises a way to control and access data maintained by a given application.

Service-Oriented Applications

SOA-app is about designing applications aggressively to be good citizens in someone's SOA-arch implementation. Service-oriented applications have a basic stack diagram as well, I mentioned some months ago. I won't repeat that explanation, but here is a more colorful picture:

Service-Oriented Application Stack

SOA-app revolves around designing elements of the enterprise application troika – message, data, and code – specifically to make it much easier to incorporate the resulting application into your SOA-arch and also promote a dynamic system that tolerates change more effectively. So, my app might be at the bottom of your SOA-arch stack, but I'll make sure my app is your best friend rather than some opaque, inflexible, monster. SOA-app means changing some classic approaches and, in some cases, fighting the dev tools to make that happen. Here are some key ideas behind SOA-app:

• Make sure callers can mix business intent into a single unit of work. Suppose you allow callers to update customers or update suppliers. Make sure you also allow callers to update a customer and a supplier in a single call. Each business function is represented by a distinct message part format, so your posting function should be able to take an arbitrary list of message parts as input and return a corresponding set of message parts containing results (if any). The reason for this is that SOA-arch may have an entity data aggregation function that updates multiple entities at once. Wouldn't it be nice if your application could accept composite documents with less initial shredding?
  
• Make sure your messages tolerate unknown or incomplete content. If a caller adds extra data that some other part of the SOA-arch needs, just ignore it. For XML, many type serializers do this pretty well. But for binary serializers, this is practically impossible. On the flip side, make sure your messages make "everything optional". There are reasonable limits here. For example, when a caller wants to create a new customer but doesn't include a name or address. Again the reason is to make life easier for callers syncing fractions of records.
  
• Use an inversion of control framework that lets your implementers re-sequence or replace logic declaratively. Windows Workflow Foundation is pretty well designed for just this kind of role. Going this route obviously has a big impact on the overall programming model of the system. For one thing, routines that update data should basically not ever return anything directly to the caller because makes the coupling too tight between components. The scenario to keep in mind is when an implementer needs to suspend a function and execute an approval routing or clear some business semaphore (like a credit hold).
  
• Give callers reasonable access to the query processor. One of the problems with most interfaces – no matter the language or platform – is that callers often can't get the exact data they need unless some designer first thought to add the right function. One of the biggest requests I see is for a correlated sub-query that our product managers didn't think of. I'm not saying that you should allow callers to send SQL – or even fragments of SQL. But you should implement some mechanism that lets callers retrieve data by submitting a query tree that can be vetted by your framework and that you can secure.
  
• Stay very document-oriented. This is easier in business applications because the data lines up well with real-world business artifacts (orders, shipments, etc.). Many application designers over-design the interface library by building a matrix of functions required by caller roles. The problem is that the matrix won't have an entry for many interactions required in the SOA-arch. Come up with a CRUDx standard for your entities and use that to determine message formats (ideally, via a transform). The "x" is whatever critical behavior a particular entity may possess. Keep the number of members of "x" as small as possible. For ERP, we've been able to do it with a single "t" – for "transition" which means to move a record from one transactional state to another (order:new -> order:cancelled).
  

It's troubling that for the past year I've talked to many people about SOA without anyone realizing we were somewhat cross-purpose. I've wanted to talk about the above points for some time, but when I would go over them with other solutions architects, it seemed like we weren't on the same page. Hopefully distinguishing between SOA-arch and SOA-app will be useful going forward.

Diagnosis: XML Fetish

The apparent medical experts who wrote According to Microsoft Windows Communication Foundation – Hands-On, have – in Chapter 3 – diagnosed me as having an “XML Fetish”:

Yet practitioners of contract-first development, working in the XML Schema language in an XML editor, tend to become distracted from those core concerns and start to worry about exactly how the data is to be represented in XML. Consequently, they begin to debate, among other things, the virtues of various ways of encoding XML, and become highly suspicious of anything that might inhibit them from seeing and fiddling with XML. The XML becomes a fetish, falsely imbued with the true virtues of contract-first development, and, as Sigmund Freud wrote, "[s]uch substitutes are with some justice likened to the fetishes in which savages believe that their gods are embodied" (1977, 66).

Wow – I’m practically paranoid! I think I understand the points the authors were trying to get across: Let the XML serializers do their job and stay out of the way. The preaching and the lame attempt to bring Freud into the discussion really put me off.

To (badly) paraphrase Norma Desmond, XML is big. It’s the toolkits that got small. Experience tells me that you actually have to keep a sharp eye on the WSDL and XML emitted from just about any toolkit. The W3C has a working group basically dedicated around that fact.

Let’s say I have a service that accepts purchase order lines. Purchasers can buy things that they receive and stock (items) or services that will be rendered. Items have a code and a quantity; services have a free-form description and an estimated cost. How do I create a service description for an operation that gives the caller that kind of choice in what gets sent?

In XML Schema, I can simply say “xs:choice”.

In C# (or most languages), I can implement this choice using a number of tactics. The most obvious way to go is polymorphism: create a base class POLine and two descendent classes StockedItem and Service. The function prototype then takes the base class as an argument. Hmm – try that once and see what kind of interoperable service description you get. My bet is something like xs:any.

To be fair, .NET has added a ton of attribution to the XML Serialization mechanism. You can implement a choice construct that works pretty well. But the attributes you need to faithfully build such a C# types are very specific to .NET and probably even less intuitive than XML Schema (never thought I would say that). Further, you have no guarantee that Microsoft (or any toolkit vendor) won’t change the XML Schema constructs mapped to a class construct when new framework versions shipped. We had this happen between .NET 1.1 and .NET 2.0.

I tend to build data contracts using XML Schema first. Then I roll the schemas through the various class generators to see how good the output is. Then I build a service using the generated classes to see how good the WSDL will be. Then I build yet another set of types using the generated WSDLs to simulate what service consumers will experience. Finally and if all looks good, I throw all of it away except for the original schema, which gets incorporated into the service description directly and, sometimes, into the service code base via XML Schema validation. I don’t like building static types for message payloads or data domain members. But that’s a whole other topic!