tag:blogger.com,1999:blog-142192972024-03-07T15:11:54.722-08:00Erik JohnsonSr. Director, Product Research, <a href="http://www.epicor.com">Epicor Software</a><p>
<b>Software Architecture</b><p>
<i>This blog is not associated with my employer.</i></p></p>Unknownnoreply@blogger.comBlogger31125tag:blogger.com,1999:blog-14219297.post-20819940326279502452009-06-01T10:40:00.000-07:002009-06-01T10:54:49.189-07:00The iPhone Post<span style="font-family:trebuchet ms;">I use a notebook computer at work, but all my off-hours-computing gets done using an iPhone which, ironically, is paid-for by my employer. The iPhone – like </span><a href="http://www.from9till2.com/PermaLink.aspx?guid=f8edaeba-c293-4cc7-9b39-bd2f57c85f39"><span style="font-family:trebuchet ms;">David Ing describes</span></a><span style="font-family:trebuchet ms;"> – isn’t just a phone that plays my iTunes (which is what attracted me). Safari, the AppStore, and Exchange compatibility makes the iPhone the most heavily-used thing I own (surpassing the margarita shaker). It’s most annoying and yet most ingenious feature, at least for us ex-Blackberry users, is the lack of a blinking light to indicate new information has arrived. </span><br /><span style="font-family:trebuchet ms;"><br />Finally, I’m weaned from those Pavlovian sneaky peeks to see if that little light is flashing while others – often significant others – are speaking to me. Yet in every dead, in-between-type moment – like waiting for take-out food, waiting at traffic lights, waiting out a design review, and especially waiting for the quarterly employee meeting to end – my hand involuntarily grasps presciousss (you know what I mean) and in the same move my thumb deftly swipes away standby mode. Password protection is such a waste of time. On the Blackberry, I surf email. On the iPhone, I surf the world.<br /><br />The iPhone is one hell of a head start from a company that I repeatedly declare I’m done giving money to. In my house we have one Mac and one PC. I don’t really compare them because they both annoy me. I like how the PC has much cheaper and open hardware and how I can do parts of my day job at home. I don’t like how I have to do more tech support when the kids use the PC. I like how the Mac makes my family happy. But I hate the fact that the cost to repair an Apple device is about 104% the cost of a new one.<br /><br />But nothing is going to catch the iPhone for quite some time. Like David mentions, it’s amazing to me how Microsoft (and practically everyone else) completely missed the point about mobile computing. Microsoft has spent multi-millions to make mobile development work for .NET programmers but has no market penetration beyond bar code readers. They have completely ignored improving the browser, falling into the tabbed browsing trap which is pure </span><a href="http://blogs.harvardbusiness.org/haque/2009/05/unnovation.html"><span style="font-family:trebuchet ms;">unnovation</span></a><span style="font-family:trebuchet ms;">. Mozilla totally gets it – </span><a href="http://design-challenge.mozilla.com/summer09/"><span style="font-family:trebuchet ms;">tabbed browsing sucks</span></a><span style="font-family:trebuchet ms;">. And in Microsoft’s obsession to go after Flash, they completely missed the importance of JavaScript running, well, fast. </span><br /></span><span style="font-family:trebuchet ms;"><br />But the biggest miss of them all? The Cloud. Believing every morsel of some apparent manifest destiny driving the Cloudrush, Microsoft is expanding its server capacity much faster than it can resolve any business model that will generate income. But they haven’t yet figured out that the first commercial apps (that matter) will target mobile users. They haven’t realized that practically all apps in the future will target mobile users. They haven’t realized how VERY few people run apps built on a native Windows Mobile stack. Mobile users are now the swing voters in deciding what and who succeeds in technology.<br /><br />It didn’t help that the Connected Systems Division, the SQL Server guys, and God knows who else started building Cloud bits without even comparing notes. They threw overlapping Azure bits at a wall to see what would stick. Of course, their customers and partners were standing against that wall at the time. Shoot-outs like that – born from CYA – are a troubling sign of indecision. Someone is either too involved in other things to own the strategy or unwilling to choose winners and losers in order to get to market. Eventually, the conflicting bits were factored out and the SQL Team has come through with better features. But missing from the entire effort is any recognition that mobile use matters.<br /><br />The Microsoft stack is fighting itself – too much time being spent making it look easy to do a handful complex things while making it harder to do simple things (WCF & Geneva come to mind). It’s time for Microsoft to stop wagging RAD toolkits at IT shops. Put the money into giving first-movers, shops rebooting their efforts for a mobile world, and innovators a chance to be successful. The technology needs are well-known: Microsoft needs a competitive mobile browser, development tools that target many devices (browsers and native), an SSO that work across cellular platforms, and a connectivity model that puts openness ahead of code expedience (yes, I mean REST).<br /><br />But most importantly, Microsoft needs to exploit the collision of enterprise IT and the consumer world. My ERP application is learning a lot from how people use Twitter and World of Warcraft. Maybe the iPhone’s head start is too great for Microsoft to seriously challenge. But even if Microsoft can’t deliver the device platforms that people crave as much as the iPhone, they ought to at least create tools for building the best apps for any device – connected securely to (what should be) the best cloud-based services. </span><br /></span>Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-14219297.post-67139406422219139152009-03-16T10:03:00.000-07:002009-03-16T18:57:04.293-07:00QCon London 2009<span style="font-family:trebuchet ms;">In the preface of <strong><a href="http://www.informit.com/store/product.aspx?isbn=0321606450">Software Language Engineering</a></strong>, Anneke Kleppe writes, “Academic computer science is in danger of slowly becoming an empirical field that studies the advancements being made in the industry”. Web programming as a fundamental platform, open source, and cloud computing movements have wildly democratized software production. Change is happening faster than most academics and vendors can keep up with – let alone attempt to lead. </span><br /><span style="font-family:trebuchet ms;"><br />I bought this book last week at <a href="http://qconlondon.com/london-2009/conference">QCon 2009 </a>in London. QCon is special because of its focus on solutions and practices without the usual vendor gravity. Sure, there is some hype lurking in the rafters. But the presenters are genuine problem solvers and conversations are intense, productive, and technical. The conference studies emerging topics in software development that people actually exploit. Those with traction, like functional programming, REST, and agile development gain industry velocity and become a bigger part of following conferences.<br /><br />QCon disciples epitomize Kleppe’s statements, bypassing the ancient waterfall travelling from computer science through toolkit vendors, application vendors, and then to users. So to me, it was a little ironic that the biggest treat at QCon was actually, um, a computer science academic. Turing Award winner Sir Tony Hoare, who invented both Quicksort (woo-hoo!) and the NULL reference (doh!), gave an outstanding keynote contrasting computer science and engineering. He said that while a scientist pursues the one great story, the engineer pursues many great little stories. Let the scientist seek correctness while the engineer creates dependability. He looks forward to a day that when something goes wrong, the software will be the last suspected cause.<br /><br />I tend to stay quiet at events like QCon because my software knowledge was self-taught. I worry that someone will throw some formalism at me I will not understand. Also, most attendees are from consultancies or large IT shops, where I work for an enterprise software vendor. My company builds ERP solutions, which are the kind of products that consultants sometimes complain are stuck in the past (while drawing good pay). Issues that resonate for me may not for others.<br /><br />That fish-out-of-water feeling faded this year, and I have Sir Tony to thank. He said “the engineer can’t afford to be certain of anything … good enough is always good enough”. That said, and with encouragement of others (like @psd), I’m looking to submit an abstract for QCon San Francisco this fall about the Restful Enterprise. We will see how that goes.<br /><br />Anneke Kleppe’s observation about computer science becoming empirical rather than the agent of progress may someday be true, but Sir Tony raised a point that made me realize, thankfully, that the academic role still has legs. During Q&A at a breakout session, someone asked about the value of proofs to verify programming languages. Sir Tony said that the commercial imperative for proofs was the virus. Viruses reach portions of a program that its normal execution never sees. Testing focuses on cases likely to arise but viruses will eventually exploit a case unlikely to arise. Resolving virus attack vectors will always be an analysis exercise, assisted by formal methods.<br /><br />I like QCon because the attendees, as a group, are of high caliber, genuinely interested in discussing issues, and know the problems real people face. I also like an event that keeps things simple. Sure, some big vendors sponsor QCon, but they are comfortable staying the background. QCon speakers are the practitioners Kleepe describes as seemingly driving the innovation balance away from academic computer science. The nice thing is that Sir Tony stepped onto his side of the scale and the practitioners simply yielded. Smart people can always recognize the smartest person in the room.<br /><br /><span style="font-size:85%;">*</span></span><span style="font-family:arial;"><span style="font-size:85%;"> PS. Many of the session slides are public (some need a password). You can find the links in the </span><a href="http://qconlondon.com/london-2009/schedule/wednesday.jsp"><span style="font-size:85%;">conference schedule grid</span></a><span style="font-size:85%;">.</span></span><br /><br /><br /><br /></span>Unknownnoreply@blogger.com2tag:blogger.com,1999:blog-14219297.post-27404110714251780002009-02-19T11:43:00.000-08:002009-02-19T11:52:29.660-08:00Let That Be Your Last Battlefield<span style="font-family:trebuchet ms;">The Gartner Group’s infamous Technology </span><a href="http://en.wikipedia.org/wiki/Hype_cycle"><span style="font-family:trebuchet ms;">Hype Cycle </span></a><span style="font-family:trebuchet ms;">is a great study in self-fulfilling prophesies. If you haven’t seen one, they are graphs with a single curvy line that measures hype levels over time. The line shoots up during the exciting times after some technology “trigger” is fired, but then soon plunges despairingly downward indicating the disappointment that whatever was being hyped doesn’t live up to expectations. Next, the line crawls back up (“enlightenment”) and then levels off somewhere midway between the expectation peak and disillusionment canyon. Interestingly, Hype Cycle charts do not gauge time or momentum. They are snapshots.<br /><br />The 2005 </span><a href="http://www.gartner.com/resources/130100/130115/gartners_hype_c.pdf"><span style="font-family:trebuchet ms;">Gartner Hype Cycle for Emerging Technologies </span></a><span style="font-family:trebuchet ms;">(PDF) shows SOA settling into the bottommost part of the cycle – the pit of despair. I think it’s there to stay. SOA was propelled up the hype curve based partly on the additive hype behind web services. And that was because the lynchpin underlying any SOA effort is integration with the constituent applications. Web services may have brought some sorely-needed platform interoperability. But while doing so, web services also perpetuated the problems SOA was meant to mollify.<br /><br />The real problem is that SOA was never about solution design. It was actually a reaction to the limitations inherent to most enterprise application APIs. It was also partly related to a perennial best-of-breed vs. single-vendor build conundrum: If you buy the best apps from different vendors, they won’t integrate, they’ll all look and act different, and maintenance costs are higher. But buying a single solution locks you into a monolithic dinosaur that (some say) over time hinders business agility. Most enterprises do some of both plus build their own stuff which adds to the problem.<br /><br />For users, these choices are just as political they are strategic. SOA gets pitched as a roadmap for having it all, which placates stakeholders (a key sales tactic). Big apps, little apps, and online apps happily interact while users blissfully see a single system. SOA is a complex way to hide other complexities, which is no way to solve a problem in the long term. If you worked for a big toolkit vendor with a consulting practice, it was like striking gold.<br /><br />SOA was invented to cover the basic failure of enterprise applications to provide loosely-coupled access to capabilities. It never represented progress in software architecture. Real change can only come by making individual enterprise applications the <em>best possible citizens</em> in an environment bounded by agreed-to assumptions, like the Internet. But it’s not just about declaring REST beats SOA. REST doesn’t help the situation unless it’s coupled with a solid information strategy – a well thought-out resource model addressing both data <em>and</em> behavior. That point has been overlooked by REST proponents, BTW.<br /><br />Nevertheless, I’m done with interfaces and typed containers. I’ve thrown it away and see no reason to go back. They do not scale well in large projects and a pain to deal with in integration projects. Their real failure is that specialized interfaces are limited by their authors’ ability to presuppose their use cases. That limits the vectors available to access data and capabilities of the application. It is especially problematic in data-driven solutions, which probably sparked the rise of SOA in the first place.<br /><br />For data-driven enterprise applications, REST can convey data and behavior with just as much fidelity as any programming model. This isn’t an idle claim – it’s based on several years of work to research how to build an ERP solution suite around REST architecture. Resource policies and URIs can hide data, govern state changes, and provide callers with information schemas. A good programming model tied to these principles can segregate capabilities, processes, and data. In the process it eliminates the trappings of static data container types and procedure-oriented functions that toolkits make easy to create but wind up creating coupling too tight for the Internet (and the Agility) age.<br /><br />I suppose it reinforces their branding, but Gartner pegs any and all technology to the same shaped line. It’s depressing to think that every new technology has to travel along the same path – especially that one. What’s the point of sticking with an idea through the first two-thirds of the journey? The smart money is to sit out the aggravating part of the cycle – as Gartner well knows. At some point during the enlightenment period, the software industry wakes up and, in scorched earth fashion, bludgeons the world with so much tooling everyone forgets the concept that originally triggered the hype.<br /><br />I don’t know whether Gartner has REST mapped to a Hype Cycle chart. If they have, then they’ve got it wrong. REST isn’t anything more than what the Internet already has had for many years. The Internet works – the trough of disillusionment (if it existed) happened long ago. Will that starve the vendors and the consultancies of the oxygen that feeds the tooling frenzy? Absolutely not.<br /><br />Like so many other issues the world faces, applications need to work together and in unique operational situations with far less effort. Applications must presume they are part of a greater machine that controls how and when the application runs and with what data. Whether in the Cloud or not, the successful applications will be those that obviate the need for SOA but work well in mash-ups.</span>Unknownnoreply@blogger.com4tag:blogger.com,1999:blog-14219297.post-59341639041349464492008-03-17T11:01:00.000-07:002008-03-17T11:08:00.597-07:00REST URI Spaces and Information Reuse<span style="font-family:trebuchet ms;">I’m on a plane returning from the </span><a href="http://jaoo.dk/london-2008/conference/"><span style="font-family:trebuchet ms;">QCon 2008</span></a><span style="font-family:trebuchet ms;"> conference in London. It was a top-notch event and among the great presentations, two things I learned stand out. The first was that I need want to learn </span><a href="http://www.erlang.org/"><span style="font-family:trebuchet ms;">Erlang</span></a><span style="font-family:trebuchet ms;">. I spent some time with Erlang inventor </span><a href="http://www.sics.se/~joe/index.html"><span style="font-family:trebuchet ms;">Joe Armstrong</span></a><span style="font-family:trebuchet ms;">, and had such good fun that I’ve already downloaded the bits and bought the book. Second, the REST rationale has really gelled and the proponents no longer see a need to argue their case – it’s time to mature the story.</span><br /><span style="font-family:trebuchet ms;"><br />In the REST track, it was said (repeatedly) that WS-* circumvented the “design” of the Web and thus ignored the Web’s innate capabilities (for example, </span><a href="http://www.eos1.dk/qcon-london-2008/slides/JimWebber_ACoupleOfWaysToSkinAnInternetScaleCatx.pdf"><span style="font-family:trebuchet ms;">Jim Webber’s slides, #14</span></a><span style="font-family:trebuchet ms;">). Sure, architectures often encapsulate subsystems for the sake of abstraction, which brings me to my point. Just as WS-* tunnels the Web, component architectures often manage persistence by tunneling a relational database. The query processing power of the database is rarely directly accessible by the solution consumers.</span><br /><span style="font-family:trebuchet ms;"><br />My company sells ERP applications, which are used to run enterprises. Our customers demand commercial RDBMS products because, in their minds, our application maintains their data. That data must be in a store they understand and (feel they) control. These enterprises invariably invent ways, bypassing our code, to productively reuse their data in ways their vendor (us) did not foresee. Conceptually, I don’t see much difference between this kind of reuse and the serendipitous reuse described by Roy Fielding (and summarized nicely </span><a href="http://steve.vinoski.net/pdf/IEEE-Serendipitous_Reuse.pdf"><span style="font-family:trebuchet ms;">here</span></a><span style="font-family:trebuchet ms;"> [pdf] by </span><a href="http://steve.vinoski.net/blog/"><span style="font-family:trebuchet ms;">Steve Vinoski</span></a><span style="font-family:trebuchet ms;">). If their data was stuck in an opaque blob, accessible only through our APIs, we wouldn’t sell very many systems. </span><br /><span style="font-family:trebuchet ms;"><br />But like I mentioned earlier component architectures relegate relational databases to soulless “persistence stores” devoid of independent capabilities. The conventional wisdom for component design says data is private to the component and access only through a public interface. But I think people have tended to militantly privatize data, denying support to anyone daring to connect to the data on their own. That practice is one reason some solutions are opting away from an RDMS stores altogether and going with more OO-friendly stores.</span><br /><span style="font-family:trebuchet ms;"><br />The obvious problem with taking this tenet too literally is that access to data is limited to a preconceived set of interfaces. Reusing data in unforeseen ways is accomplished by sneaking around the application or rebuilding it to accommodate new requirements. Databases are rather smart engines designed under an assumption that data, once created, is then retrievable in countless, unforeseen ways. Nevertheless, architectures are burying perfectly good query processors under layers of abstractions, objects, and interfaces. Being pushed to the bottom of the stack is one reason that physical data models are actually devolving, IMO, but that’s another story.</span><br /><span style="font-family:trebuchet ms;"><br />The public “API” for a RESTful application is its URI address space. You can invent a list of URIs mapped to resources and state sequences all you like. But the reuse potential is limited to whatever your callers can get out of that URI space. Like REST, SQL databases have a uniform interface. But look at the practically unlimited variety of resources you can access. Obviously, a REST URI shouldn’t be a SQL statement and I’m not trying to shoehorn XQuery into a URI. All I’m saying is that a URI space can incorporate parent-child and relational characteristics from a data model – using relational database behavior as a guide. This has been a key aspect (for 8+ years, BTW) in developing URI strategies for our products.<br /><br />The emerging specs and toolkits, like WADL and WCF, feature URI template constructs. But URI templates have no notion of resource linkages (parent, relational, or otherwise) and that limits their effectiveness. At QCon, there was little consensus that WADL was the right way to describe a REST application. But I think REST description <del>languages</del> resource types are coming and I’d like their creators to at least consider resource linkage features for URI templates. It’s all been done before.</span>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-14219297.post-52138845973944524012008-01-19T18:20:00.000-08:002008-01-19T18:48:02.872-08:00Got my name in the Financial Times!<span style="font-family:arial;">Only a few people know I like to hobby with macroeconomics during break-time around the office*. Most of my colleagues had begun the weekly yoga session in the area outside my office. So, the lights were out and the soothing music seeped under my door. I was reading the commentary in that day’s </span><a href="http://www.ft.com/"><span style="font-family:arial;">FT</span></a><span style="font-family:arial;"> – the top op-ed piece essentially said that the US should welcome <a href="http://en.wikipedia.org/wiki/Sovereign_wealth_funds">sovereign wealth funds </a>without looking too intently under the hood.</span><br /><br /><span style="font-family:arial;">Well, clearly <em>someone</em> had to say <em>something</em>. So, I channeled a little </span><a href="http://www.econlib.org/library/Enc/bios/Keynes.html"><span style="font-family:arial;">John Maynard K.</span></a><span style="font-family:arial;"> and typed up my first-ever letter to an editor. And in the spirit of what can be done to “some of the people some of the time”, the FT </span><a href="http://www.ft.com/cms/s/0/84ce2d3a-c56a-11dc-811a-0000779fd2ac,noOfParas=2,emailFormat=html,storyType=ultralight,dwp_uuid=dafed534-3001-11da-ba9f-00000e2511c8,print=no,_i_email=y.html"><span style="font-family:arial;">printed my letter</span></a><span style="font-family:arial;"> (requires free registration to read the whole letter**) in last Friday’s edition (at least in the US). I had no idea until I stumbled on it while on the train home from work. Cool!</span><br /><br /><span style="font-family:arial;">* Not really.<br />** You can try this </span><a href="http://www.ft.com/cms/s/0/84ce2d3a-c56a-11dc-811a-0000779fd2ac.html"><span style="font-family:arial;">link</span></a><span style="font-family:arial;"> until the FT scrolls the entry into the archives.</span>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-14219297.post-41827888214504826402007-10-29T17:16:00.000-07:002007-10-29T17:20:17.737-07:00Poking at ROA+1 to <a href="http://www.pluralsight.com/blogs/tewald/archive/2007/10/19/48805.aspx">Tim Ewald’s point</a> that the ROA crowd might be pressing too hard to make PUT do some uncomfortable things. -1 (with all due respect) to <a href="http://www.innoq.com/blog/st/2007/10/28/batch.html">Stefan Tilkov’s assertion</a> that “anytime you find yourself adding words like “operation” to your representation, you’ve violated one of the core RESTful HTTP principles, which is that the intent should be communicated using the HTTP verb.” IMO, that’s an unfair litmus test for ROAishness.<br /><br />Stefan was chiding a specific situation with respect to GData, which I do not know much about. But I do know that situations exist where I want to convey multiple “intentions” in a single physical call. You should be able to update a customer and a supplier in the same message -- to support arbitrary composition of the intent. What's the URI for that? That’s a rhetorical question, because a POSTing location and a schema indicating the message format are all you need.<br /><br />I’m a big fan of ROA, but I’m worried that ROA fundamentalism will create a quagmire (shared by SOA) where all advice seems to be about what *not* to do. ROA makes it easy to bind the HTTP verb with your intent, but it doesn’t <em>require</em> you to do so. If I define a format for a message that can contain multiple “intentions” and then expose POST endpoint for processing messages, have I broken some Law of ROA? I don’t think so.<br /><br />Or, like Tim asks, does ROA mean I have to PUT things in an imaginary basket and then PUT an imaginary thing into an imaginary place to make the basket get processed? No. There is no crime in using The Uniform Interface in a way that partners the payload, verb, and URI to dispatching logic or help give you a cleaner programming model.<br /><br />Sure, it’s best to keep business process protocol stuff out of the data in your payload, which is what I think Stefan was really alluding to. That obviously gives you the ability to reuse a message format by isolating intention to the URI. But resources often have internal processes, like special state transitions, which may need to be manipulated via flags in the payload.<br /><br />The free market rightly determined that WS-* is often too difficult to use and it probably doesn’t solve the problems you think it will. But if ROA forces people to use more effort or do unnatural acts to get a day’s work done, ROA will be out on its ass as well. In both cases, the good goes down with the bad.Unknownnoreply@blogger.com2tag:blogger.com,1999:blog-14219297.post-59652589118675860852007-09-28T14:10:00.000-07:002007-10-06T15:01:04.923-07:00Software Architecture as PrinciplesSo, I had <strong>The Talk</strong> with my 10-year old son today. He was a confused and even became a bit emotional as the gravity of the facts emerged. The discussion was, of course, about copyright law.<br /><br />He had burned a CD containing tracks from his iTunes library to give to a friend as a birthday gift. So, we talked about how this was in fact stealing and that we should just go buy a new copy the music outright, etc. Here’s a sampling of from his questions during the discussion:<br /><br />Q: <em>What if my friend has some songs on his iTunes, but his CD burner is broken. I have the same songs, so can I burn them from my computer and give him the CD?</em> A: Um, maybe that’s OK. I don’t know.<br /><br />Q: <em>How come it’s OK to lend out PlayStation disks?</em> A: Easy (whew!), because while they borrow it, you are not using it (note to self: is that really legal?).<br /><br />Q: <em>So, can I burn a CD, give it to someone, and just not listen to the songs myself until I get the CD back?</em> A: Um, you’re late for school – off you go.<br /><br />Like software architecture (and legal systems), copyright is a principles-based rather than a rules-based concept because it’s impossible to precisely spell out, up-front, all actions that constitute non-compliance. The principle says you can’t disseminate the work of others without permission. Laws assign penalties to broad forms of violations like producing counterfeit software. Court precedents over time develop the more specific lists of what’s OK (backing up your iTunes) and not OK (giving out copies).<br /><br />Software architecture works the same way. Software architectures are collections of principles that define guidance and boundaries for producing software systems. Lower-level guidance comes in the form of standards and practices for developing a software system that conforms to the principles in the architecture.<br /><br />Principles-based regulation means that laws are enforced by examining the <em>intent</em> of people rather than reconciling deeds against a roster of specifically banned actions. The Enron scandal, it’s said, grew unnoticed for years because Enron parsed the regulations in bad faith and created scenarios that somehow convinced Author Anderson to sign off on audits. Enron and Arthur Anderson both knew that accounting principles (and the law) were being violated but felt relatively safe because any by-rote analysis of their accounts against the rules (as written) would come up clean.<br /><br />UK regulators like to say they have no Enron situations because UK accounting standards are principles-based from the outset. I don’t know how true that is, but the infamous U.S. Sarbanes-Oxley Act of 2002 directs the U.S. Securities & Exchange Commission to examine the feasibility of moving the U.S. to a principles-based accounting standards system.<br /><br />Getting (at last) back to software architecture, I work for an independent software vendor (ISV) in the enterprise resource planning (ERP) market. One of the characteristics of an ISV is that we are highly principles-based and generally don’t rely on thick volumes of specific details telling our engineers exactly what to do. Sure, we have standards around code and UI styles. But developers are taught the principles within the product architectures and the intent behind the functionality. That in turn helps prevent issues like “the spec didn’t specifically say the shipment quantity <em>had</em> to be a positive number”.<br /><br />As we expanded product development overseas, we didn’t rely on outsourcing to contractors. We tried it, but it was too hard to convey the architectural principles and the business intent to a transient staff 13.5 time zones away. Without the principles as context, we had to type reams more specs and it the results consistently had to be reworked. We wondered whether our processes were just plain sloppy. But that wasn’t the case. Our development model just didn’t fit an environment where people came only with bespoken capabilities and never developed any lasting interest in our objectives.<br /><br />Instead, we opened development offices staffed by our own full-time employees. That meant we could convey our architectural principles, development standards, and train the teams up on business functions like cost accounting and manufacturing. Permanent employees doing the work, no matter where in the world, turned out to be cheaper than just outsourcing to a contractor. More importantly, we realized much better agility.<br /><br />The problem with developing long lists of rules is that they are expensive to maintain and easy to evade when the pressure is on. I would rather state a principle like “creating new records for purpose X MUST minimize the DB batch request count and NOT hold locks on Y, under typical loads, to significantly affect other processes” and then test for compliance rather than using, say, code reviews and a checklist to attempt to spot things that potentially violate the principles.<br />For us – a 400-person development organization – agility means efficiently releasing products year-after-year that incorporate rapidly changing markets. Believe it or not, technology shifts aren’t nearly as acute to us as market shifts from compliance and globalization. I never need to find 150 Ruby programmers on a moment’s notice. I need 150 people to understand how a chart of accounts works and how to make one that can be legally used worldwide.<br /><br />So, while we don’t do many scrums in our product development cycle, we also don’t do waterfall management. The middle ground works because our jobs revolve around incremental evolution of a few systems. It’s an easy place for principles-based management to work. Software architecture perfection to me is nothing but a short list of general capabilities and a long list of non-functional requirements. Those are the principles. Standards and practices do the rest.<br /><br />My son says he likes knowing how music is copyrighted. The thought he might be letting down people who really own the recordings actually horrified him (much more than me, anyway). By understanding the principles, he can (hopefully) figure out for himself what scenarios violate the law. Now, I guess we’ll have to do the “fair use” talk pretty soon, though. What other talks am I forgetting about?Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-14219297.post-51895334180716308972007-09-19T20:50:00.000-07:002007-09-19T20:54:52.422-07:00CorporationletsIn the Financial Times on 13 July 2007 (European Edition, <a href="http://www.ft.com/cms/s/6169db6c-3093-11dc-9a81-0000779fd2ac,dwp_uuid=ece9f348-2eea-11dc-b9b7-0000779fd2ac.html">special insert</a>), Linda Gratten asserts that collaboration is poised to upstage competition as a primary business strategy – that partnerships can create value more efficiently than pure competition. The X and Y generations were born wired to the Web, have been raised in an “everyone wins” mentality, and are averse to organizational hierarchies. So, once that new generation rises to corporate power, partnerships will form the dominant agent of business value creation. That last bit smacks me as unsubstantiated age discrimination – but creating value via joint ventures obviously works.<br /><br />Since the focus of the piece was collaboration, describing Second Life as a vehicle for collaboration couldn’t be helped. Gratten mentions the “experiments” at IBM (“very successful”) to use Second Life for team interaction. Given that the GenY mastery of social networks is so entrenched, something akin to Second Life could become the wheel grease of tomorrow’s joint ventures. After all, any mass proclivity toward partnerships requires a place where participants can sniff each other out – an online speed-dating environment for stakeholders.<br /><br />A gold rush has started to converge social networking and business strategy. The idea has moved beyond making money by connecting people together – that’s so last month. People need to make money for themselves by easily forming ad-hoc commercial partnerships. That kind of value creation can reap a huge economic benefit, especially if small businesses become enfranchised. So, the European Commission is trying to act on this vision.<br /><br />The EU25 is home to 23 million individual businesses where more than 90% are “micro-enterprises”. In the EU15, only 10% of businesses have any computer integration with either side (supply and demand) of their supply chain*. Connecting these 23 million companies, so they can inexpensively form partnerships and new supply chains, is in the EC’s economic thinking. It also ties back to Gratten’s point about next-generation entrepreneurs choosing joint venture strategies over brute competition.<br /><br />Micro-enterprises – like your corner laundry – are not generally capable enough to build inter-enterprise orchestrations. But there is one IT skill that the entire population has in-hand: Web browsing. So, the EU is smitten with the notion that Web 2.0, the Social Web, Second Life-ish solutions, SOA, and SaaS can all come together to form a solution – and so the European Commission is <a href="http://cordis.europa.eu/ist/ict-ent-net/index.html">funding efforts</a> [cordis.europa.eu] toward that goal. It’s a Grand Unification where businesses find each other and collaborate like Second Life players while the business transactions are easily, properly, and legally pumped across business entity boundaries.<br /><br />It’s certainly my nature to drool over the software architecture challenges in building this dream. But my biggest worry, if I was picked to do the software estimate**, is determining when you’ve done enough to displace existing business practices.<br /><br />Supply chain commerce, commodity purchasing, service industries, and consumer businesses all have buyers and sellers. But the operational characteristics of how buyers find sellers, how trust chains are established, how contracts are executed, and how payments are managed are highly evolved and optimized within each sector. Can a solution that undermines the efficiencies in business growth developed during the last century be successful?<br /><br />Here’s an example: A few decades ago, businesses (especially manufacturers) kept many active suppliers on file. It was believed that having many suppliers for a widget meant more competition for your business kept purchase prices low. But it was later realized that reducing the number of widgets kept on-hand inventory improved profits much more than buying your widgets at rock-bottom price.<br /><br />Just-in-time (JIT) manufacturing was invented for this reason. Material arrived at the precise moment and location where it would be consumed – which meant daily, weekly, or ad hoc replenishments. Perfection meant having zero widgets sitting in your warehouse. Purchasing strategies were simplified –you only have 1 or 2 suppliers on long-term contracts rather having to manage bidding cycles with multiple suppliers. But there were also the matters of trust and reliability. If a supplier failed to deliver, production would likely shutdown and employees would be furloughed.<br /><br />The supply chain partnerships in JIT operations created more value than traditional brute competition. Manufacturers eschewed continuously fighting to get the best deal from a transient supplier to create strategic arrangements with a single supplier. Boeing and Airbus have gone even farther by having their suppliers share risk in major programs. Suppliers absorb some up-front costs in exchange for revenue to come later if the airplane sells well in the marketplace.<br /><br />But doesn’t that fly in the face of the model the EU is envisioning? The goal is to foster easy, ad-hoc partnerships with extremely low cost of entry. However, manufacturers revolutionized their businesses by getting away from a scattered roster of transient. Is the EU is looking for technology to bring enterprises together despite entrenched, non-technical aspects?<br /><br />The answer involves, as usual, a tipping point. Some sort of foundation technology – the goo in the Petri dish that feeds life – needs to be developed (no small task). Then, an entire population has to learn that partnering is feasible, online or not, and they have to know how to do it legally. Finally, participants have to know how to measure the quality of a trust chain.<br /><br />From a business-to-business perspective, this is the final frontier of the Internet. It’s achievable and inevitable if the IT industry can get non-technical stakeholders into the game.<br /><br /><span style="font-size:85%;">* These figures are from Eurostat via a keynote presentation by Christina Martinez of the European Commission at the 2007 WS-I Spring Plenary in Brussels.</span><br /><br /><span style="font-size:85%;">** My estimating practices were pretty much codified </span><a href="http://www.from9till2.com/PermaLink.aspx?guid=8f80ad53-f9ba-43e1-a654-3db5c0d3621b"><span style="font-size:85%;">here</span></a><span style="font-size:85%;"> in 2005.</span>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-14219297.post-68380834873424280112007-09-13T16:00:00.000-07:002007-09-17T06:33:08.614-07:00REST by way of SOAP<span style="font-family:trebuchet ms;">It’s OK to love REST (as I do), but I try to not let it blind me completely from WS-*. As I’ve often said, choosing REST – or any resource-oriented architecture – means you’ve decided to adopt a uniform interface. That does not necessarily mean dropping WS-*. Unfortunately, SOAP 1.0 emphasized an RPC programming model even though a pure message-based model was included as well. SOAP v1.2 sort-of/kind-of attempted to make amends:</span><br /><br /><span style="font-family:trebuchet ms;font-size:85%;"><em>Some early uses of [</em></span><a href="http://www.w3.org/TR/2007/REC-soap12-part0-20070427/#R11"><span style="font-family:trebuchet ms;font-size:85%;"><em>SOAP 1.1</em></span></a><span style="font-family:trebuchet ms;font-size:85%;"><em>] emphasized the use of this pattern as means for conveying remote procedure calls (RPC), but it is important to note that not all SOAP request-response exchanges can or need to be modelled [SIC] as RPCs. </em></span><br /><br /><span style="font-family:trebuchet ms;">So, SOAP from the beginning identified two payload models: the Document model indicated that the payload is simply a message. The RPC model meant that elements within the payload should be mapped to functions and parameters.</span><br /><br /><span style="font-family:trebuchet ms;">SOAP also included a basic type system – SOAP Encoding – to help implementers know how to serialize objects defined – as </span><a href="http://www.pluralsight.com/blogs/dbox/default.aspx"><span style="font-family:trebuchet ms;">Don Box</span></a><span style="font-family:trebuchet ms;"> might put it – in languages that use dots. It had a rather short life because the W3C was about to deliver XML Schema and having 2 type systems in SOAP wasn’t going to be helpful. For some reason, using XML Schema to define a message format was called Literal.</span><br /><span style="font-family:trebuchet ms;"><br />So, SOAP messages could be considered <em>RPC-Encoded</em>, <em>RPC-Literal</em>, <em>Document-Encoded</em>, or <em>Document-Literal</em>. </span><span style="font-family:trebuchet ms;">The WS-I Basic Profile working group unanimously banned <em>Document-Encoded</em> and <em>RPC-Encoded</em> because they were obsolete, leaving <em>*-Literal</em> in play.</span><br /></span><span style="font-family:trebuchet ms;"></span><br /><span style="font-family:trebuchet ms;">But what people generally don’t know is that some members including Microsoft and IBM argued hard to go farther and eliminate RPC-Literal as well. </span><a href="http://www.pluralsight.com/blogs/tewald/default.aspx"><span style="font-family:trebuchet ms;">Tim Ewald</span></a><span style="font-family:trebuchet ms;"> had said “the message <strong><em>is</em></strong> the task” meaning a payload identified with a namespace URI is all you need to dispatch a call. But others (like JAX-RPC users) argued that keeping RPC and was critical for the adoption of web services. </span><br /><br /><span style="font-family:trebuchet ms;">In an attempt to mollify the RPC advocates, Tim, Don Box, and Keith Ballenger went so far as to write up a spec for mapping object graphs to an XML tree so implementers could align their type builders and serializers using a common interpretation of a message schema. The idea was to standardize an object-graph to XML conversion and include it with the WS-I Basic Profile. </span><br /><span style="font-family:trebuchet ms;"><br />For bureaucratic reasons, the WS-I couldn’t publish it (but you can find it: </span><span style="font-family:trebuchet ms;">USPTO Application 20040239674). BTW, think about how useful that (now patent-pending) spec would be today in standardizing JSON/XML conversions. It might have also helped out </span><a href="http://www.w3.org/2002/ws/databinding/"><span style="font-family:trebuchet ms;">these guys</span></a><span style="font-family:trebuchet ms;"> [W3C XML Schema Patterns for Data Binding].</span><br /><br /><span style="font-family:trebuchet ms;">So, the SOAP Document/Literal approach was an early favorite approach to web services – and its closer to REST than many realize. It standardized on GET and POST, which certainly aligns with my thinking. The URL for a GET can easily be cast as a straight resource fetch. The URL for a POST is generally cast as a “processing endpoint”, which reflects real-world thinking, IMO (you never really PUT what you GET). That leaves dispatching – invoking the right code when a message is received.</span><br /><span style="font-family:trebuchet ms;"><br />Both REST and WS-* users can tell you it’s difficult to rely only on the value of a URL path to determine how to process a message. Sooner or later, you have to look at the payload. The original SOAP mechanism involved an HTTPHeader called SOAPAction, which was a hint instructing the application about how to process the message. Many REST implementations use the content-type HTTP header in exactly the same way (which technically violates the HTTP 1.1 spec, BTW).</span><br /><br /><span style="font-family:trebuchet ms;">Using an HTTP header value to indicate what your payload is makes some sense if your payload can be anything in any format. But if your payload is XML (or a well-formed JSON array), then why not dispatch the call based solely on the (qualified) name of the root element? The only real difference between SOAP and REST here is that SOAP has an envelope construct and your data goes inside an element called “body”. If it’s a non-SOAP call, don’t use an envelope – it’s OK. </span><br /><span style="font-family:trebuchet ms;"><br />What about the WSDL? Don’t use one unless you have the need to build RPC-oriented proxies for your service. All you really need is a schema for the message format and away you go. If your endpoint is HTTP (if you’re looking at REST, when is it not?), you don’t even need a proxy built for you at all – another reason REST is getting popular.</span><br /><br /><span style="font-family:trebuchet ms;">In my REST work, I’m keeping an eye out to avoid straying too far from the bits of WS-* that might come in hand sooner rather than later. Security is my biggest worry. I don’t want to architect around the notion that transport level security is all I can have. I might need to sign that XML and encrypt bits of it anyway. I might, someday, actually come across an intermediary processor. Maybe that header element can be useful.</span><br /><br /><span style="font-family:trebuchet ms;">Most of WS-* is now beyond my comprehension and I don’t see ever using major parts of the stack because the toolkits won’t support resource-orientation to the degree I need. But HTTP, SOAP, the WS-I Basic Profile, and XML Schema can be useful even in completely RESTful projects. If you want.<br /><br /></span>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-14219297.post-60128836988383977032007-06-21T12:07:00.000-07:002007-06-21T12:21:09.489-07:00Metadata Pathways (Rewrite)I got a comment from <a href="http://markclittle.blogspot.com/">Mark Little</a> a couple of weeks ago about my 2005 blog entry <a href="http://appside.blogspot.com/2005/08/patterns-for-soa-20.html">Patterns for SOA 2.0</a> saying (nicely) that he couldn’t figure out what I was talking about. I completely agree – it’s like I took a quick discussion about metadata and the last-place entry in a <a href="http://www.mcsr.olemiss.edu/~egjbp/faulkner/faux.html">Faux Faulkner contest </a>and shoved them into that machine from <em>The Fly</em>.<br /><br />I’ve been meaning to re-write the whole thing for a while. And earlier this week, I saw a comment from Jörg Schäfer <a href="http://www.innoq.com/blog/st/2007/06/19/rest_axis2_and_hypermedia.html#c86426">here</a>, which relates to what I was trying to say back on 2005:<br /><br /><span style="font-family:lucida grande;font-size:85%;">Philosophical remark: Maybe what we need is a programming model that takes one of the few mathematically sound models of CS, namely relational theory, seriously (the other mathematically sound foundation is lambda calculus of course leading to functional programming). So rather than continuing with OO we need a proper programming abstraction for programming with relational algebra (entities, relations, RFD…).</span><br /><br />I also saw David Ing asking Microsoft to <a href="http://www.from9till2.com/PermaLink.aspx?guid=0e3eb794-ec0c-4dd5-804a-b9f5c7a0667b">please get on with</a> whatever building 42 has cooking. I don’t know anything about that except to keep sunscreen with a high UV rating handy during the unveiling. The point is that all 3 examples (and I’m just guessing on the Microsoft one) are rethinking the abstractions used to define systems. Obviously, any benefits from improving the abstractions have great leverage for the rest of the system.<br /><br />So, here goes another try:<br /><br />If you take a typical approach to programming a data-driven application, you have a data access classes, business logic classes, (and now) service classes, client-side adapters, and UI classes. If it’s a large system you start breaking down common areas and define a framework. But in any case, you can wind up with a mass of interfaces, components, serialization, data binding, and event management.<br /><br />It works initially – and you get great <a href="http://msdn2.microsoft.com/en-us/library/hcw1s69b(VS.71).aspx">Intellisense</a> support. But anyone who sells enterprise apps for a living knows that most any deviation from the initial functionality requires a rebuild of at least some application. The better solutions (like those from <a href="http://www.epicor.com/">my employer</a> :) ) go through significant hell trying to put the flexibility users want back into systems built on classic interface/component architectures and programming models. But that means fighting the toolkits and building lots of framework code. And on top of that, it turns out the classic interface-based programming model might not be the best way to go in the first place.<br /><br />My point back in 2005 was this: Systems need to tolerate changing data models and changing behavior without downtime. Systems must accept varying message formats for a given business intent. Callers should be able invoke an arbitrary set of functions and specify the unit(s) of work. All data must be addressable by at least one human-readable identifier. Callers should not have to generate code or otherwise build proxies strongly typed to my application domain – but they can if they like (that’s a recent addition to the list).<br /><br />I want to build systems around engines and metadata as much as possible. That doesn’t mean eliminating any work by a competent programmer. But I would like the work a programmer produces to stick to the explicit functional task at hand and be exquisitely reusable. I’m also not talking about DSLs or software factories (which I’m pretty much over with). But I like the idea of a system factory, where engines manage messages, data, and execution workflow – the primary colors of an enterprise system. The factory is a runtime that adapts the system as information about the application changes. The robustness of that system thus depends on the quality of its metadata and, crucially, the quality of coupling between metadata and the processes that consume metadata.<br /><br />To get there, the industry needs to begin improving how abstractions are expressed through metadata – which is an architectural activity. Metadata now often spawns other metadata. Say I add a field to the customer entity in my data model (metadata). That infers a change to the message formats for my customer service – described in, say, XML Schema (metadata). My data model change also initiates a change to the physical data table (handled by an engine). The side-effects of metadata consumers are changes to system behavior, the generation of derived metadata, and many other possible actions.<br /><br />Some attributes in my data model metadata affect message formats but not the physical tables (or vice-versa). In other words, data model attributes have aspects that drive processes that consume the metadata. For example, a concurrency model aspect tells the DDL engine to emit specialized columns to manage, say, record versioning. That same aspect also causes the message format description to include a required version value. The process that receives messages also sees this aspect and then knows to validate that messages actually the version value and checks to see if it is current.<br /><br />Aspects might not just be name-value pairs, but complete metadata categories on their own. A good example is a life-cycle chart defined using a state sequence. As I’ve mentioned, I think you can derive all behavior (in my problem domain – ERP) by mapping CRUD constraints to a state chart – all of which can be invoked by a uniform interface. Putting my entity life-cycles into metadata and the understanding into (for example) the DDL and message description/processing engines is a pretty powerful approach because I can revise them to match business requirements as needed.<br /><br />Aspects should be composable. For example, a <a href="http://blogs.msdn.com/pathelland/default.aspx">Pat Hellend</a> aspect in my data model metadata with values like resource, reference, and activity might combine the life-cycle and versioning aspects to make sure reference data stays read-only (affects the service behavior) and that resource data is versioned (affects message format, service behavior, and the physical data model).<br /><br />Getting back to Jörg’s comment, let’s say that each attribute in a given metadata type represents a category A. You can then think of each aspect f as a <a href="http://en.wikipedia.org/wiki/Morphism">morphism</a> connecting A to another category B, where B is a consumer of the metadata type (<em>f</em>: A->B). As I mentioned, the target for a given aspect can be other metadata (like the message description), a running processor or engine (like the service layer), or anything else (like SQL DDL). So, “B” in this case is a collection of objects (like other metadata) or a behavior specification. I can also put an aspect g on category “B” to create a category “C” (<em>g</em>: B->C). That implies a compositional ability because for each element in “A” (<em>a</em>), you can get to a corresponding element in C by way of <em>g</em>(<em>f</em>(<em>a</em>)). If I have an engine that relies on metadata “C” and an aspect element a is changed, the engine knows it needs to execute the transforms <em>g</em> and <em>f</em> to get the revised member of “C”.<br /><br />So, I have 2 questions for the world in looking to develop these lines of thought. The first is about dealing with the pervasive metadata pathways in complex applications. If we look at what all that metadata represents and how it affects a system, I think we could start to categorize metadata-oriented scenarios. In other words, we can identify software architectural patterns in the use of metadata – in both its content and the effects on systems. Does anyone agree?<br /><br />The second question is about applying some sort of mathematical rigor to metadata architecture. The categories and morphisms I mentioned come from category theory, which may or may not have any promise as a basis for the formalism I’m looking for. I was an SDSU music major – what do I know? Category theory has been tried as a basis for programming languages without a lot of commercial success. But it also was helpful in connecting Cartesian Closed Categories and lambda-calculus (which is some of the math behind LINQ and F#). So, if metadata and metadata consumers are used to define systems, and we can cast these artifacts as collections of categories connected by morphisms, can some part of a mathematical discipline like category theory be employed to help formalize the way we define systems?<br /><br />That was what I was trying to articulate in 2005 without success. I hope this comes off a little more clear.Unknownnoreply@blogger.com2tag:blogger.com,1999:blog-14219297.post-37992141347868122942007-06-05T14:21:00.000-07:002007-06-05T14:23:20.011-07:00REST Over HTTP<span style="font-family:arial;">Harry Pierson </span><a href="http://devhawk.net/2007/06/05/A+REST+Question.aspx"><span style="font-family:arial;">asks a direct question</span></a><span style="font-family:arial;"> about whether REST is necessarily bound to HTTP. I agree with Harry (and Anne Thomas-Manes as described </span><a href="http://searchwebservices.techtarget.com/originalContent/0,289142,sid26_gci1256796,00.html?track=NL-110&ad=590742&asrc=EM_NLN_1507744&uid=5532089"><span style="font-family:arial;">here</span></a><span style="font-family:arial;">) that REST is an architectural style as much than anything else. And I also noticed </span><a href="http://www.base4.net/Blog.aspx?ID=447"><span style="font-family:arial;">this</span></a><span style="font-family:arial;"> from the Alex James, who has a (self-described) rant on the subject. As much as the point is totally arguable and understandable, I think it’s an unwinnable battle.</span><br /><br /><span style="font-family:arial;">I was at a WS-I meeting (the most productive one ever, BTW, because it was held </span><a href="http://www.turtlebayresort.com/default.asp"><span style="font-family:arial;">here</span></a><span style="font-family:arial;"> in Hawaii) a couple of years ago and casually asked </span><a href="http://www.mnot.net/blog"><span style="font-family:arial;">Mark Nottingham</span></a><span style="font-family:arial;"> (probably over a </span><a href="http://www.mnot.net/blog/2004/07/25/monkey_lunch"><span style="font-family:arial;">Monkey’s Lunch</span></a><span style="font-family:arial;">) about loosening the Basic Profile to allow for SOAP over TCP/IP. His answer was simply, “Why?” His point was this: Sure, you can drive SOAP over TCP/IP, Netware, whatever. But is there an interoperability goal in doing so? The answer is not really. You would need to own both sides of the pipe to make it work.</span><br /><br /><span style="font-family:arial;">The IT industry is cozy with ports 80 and 8080. Opening up anything else practically requires an </span><a href="http://sec.gov/about/forms/form10-k.pdf"><span style="font-family:arial;">SEC 10-K</span></a><span style="font-family:arial;"> disclosure. Come to think of it, we used to actually say “SOAP over HTTP” like they were independent and someone might actually use SOAP over SMTP. </span><br /><br /><span style="font-family:arial;">I see the term “REST” in a similar situation. It’s tied to HTTP and it is probably impossible to wrest it away. I don’t have a problem with that – I’ll just learn to speak architecturally about “resource-orientation” and then about how to tie in “REST” out on the edge.</span>Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-14219297.post-85140979291367743882007-05-31T18:46:00.000-07:002007-05-31T18:50:32.741-07:00How Do Bandwagons Fit into a Project Plan?That thunder behind you is the REST bandwagon comin’ your way. Anne Thomas-Manes at the Burton Group has proclaimed that <a href="http://searchwebservices.techtarget.com/originalContent/0,289142,sid26_gci1256796,00.html?track=NL-110&ad=590742&asrc=EM_NLN_1507744&uid=5532089">the future of SOA lies in REST</a>. I’m not saying the Burton group is itself just jumping on a bandwagon – but their declaration is significant enough to convince others to take a leap. The REST dust-up is in such a very early phase and it’s far from clear whether the bandwagon is going to help me or hurt me. David <a href="http://www.from9till2.com/PermaLink.aspx?guid=0a2bdabd-e343-4b7c-b5f9-10ba3bd3af0a">thinks he might jump on</a>, but let someone else do the driving. Nothing wrong with that strategy, but does it feels like <a href="http://osdir.com/ml/windows.devel.soap.general/2002-08/msg00049.html">history has come around a bit</a>, eh David?<br /><br />I think the REST bandwagon, the long-time Lords of the Web, and the WS-* camps are heading toward a colossal and rather fun collision. There will be friction between the stewards of HTTP and those who warp it to meet their RESTful needs. The REST camp likes to say HTTP is all you need – and I agree. But don’t be fooled into thinking that the foresight of the authors of <a href="http://www.ietf.org/rfc/rfc2616.txt">RFC-2616</a> and <a href="http://www.ietf.org/rfc/rfc2396.txt">RFC-2396</a> took all this into account. The RESTafarians probably won't want to see their momentum slowed by the standards trust. And then someone is going to say, “Hello! Is that an XML document you’re handing me? Care to sign it?” The WS-* folks are already figuring out how to chase that ambulance.<br /><br />I’ve wasted a lot of cycles worrying that the Lords of the Web are going to tell me I’m perverting the Web’s majesty. For whatit's worth, here is my advice. First, it’s <a href="http://pluralsight.com/blogs/tewald/archive/2007/05/10/47273.aspx">OK to say you are doing REST</a> even if you haven’t read the dissertation. Personally, I think the magic is in the URIs and the links embedded in the payload. But that’s just me. Second, use the HTTP methods however you see fit. I use GET and POST exclusively myself. I use the occasional <a href="http://www.w3.org/Addressing/rfc1737.txt">URN</a> and I don’t think it’s an <a href="http://norman.walsh.name/2006/07/25/namesAndAddresses">evil thing</a>. Some people use the HTTP media-type to indicate what the payload contains. But RFC-2616 “discourages” the use of media-type values that are unregistered (section 3.7). Just use media-type if it helps you. When my payloads are XML, I have a namespace (a URN, BTW) to indicate what it is. For HTML, I haven’t really settled on anything yet.<br /><br />No matter how far ahead of the REST bandwagon you think you are, it *will* overtake you. I’m sure our marketing department will snap up REST as fast as we did “web services”. I blinked one day and the word “SOA” appeared on the collateral. But you don’t need a REST toolkit – use the bits you have now. More importantly, don’t let the toolkit vendors say “just make objects – we’ll hide the REST part for you”. Good musicians learn to improvise by first knowing their scales. If you know the fundamentals of data modeling, URIs, and HTTP you shouldn’t need WADL, WCF, O’Reilly, or the bandwagon to be successful.Unknownnoreply@blogger.com3tag:blogger.com,1999:blog-14219297.post-72918247480971291932007-05-29T10:15:00.000-07:002007-05-29T10:34:15.608-07:00Harry & David<span style="font-family:arial;"><a href="http://www.devhawk.net/">Harry </a>and <a href="http://www.from9till2.com/">David </a>(not <a href="http://harryanddavid.com">these guys</a>) are blogging comments to each other about REST. I wasn't sure where to leave my own, so I guess it's here...</span><br /><br /><span style="font-family:arial;">David</span><span style="font-family:arial;"> </span><a href="http://www.from9till2.com/PermaLink.aspx?guid=5f9ac25b-6e8d-412f-9a16-f9f0c0dbf9be"><span style="font-family:arial;">wonders</span></a><span style="font-family:arial;"> whether widely varying interpretations of REST is an interop killer. Now that is an interesting question. It’s arguable that if two systems have completely different notions of state transitions and URI constructs, interop will suffer and you’re back to writing glue – or pitching for standards. On the other hand, there are those who argue that </span><a href="http://www.w3.org/Protocols/rfc2616/rfc2616.html"><span style="font-family:arial;">RFC-2616</span></a><span style="font-family:arial;"> has everything you need to be “good enough”, which, conveniently for them, makes interoperability an app domain issue. I think the web wonks have a good point. But it’s hard to know if resource-orientation (sorry) is a way forward unless you first let go of interfaces. One thing that scares the crap out of me is that someone will wire URIs and BPEL together and declare “Mission Accomplished”.</span><br /><br /><span style="font-family:arial;">In Perfect Land, you can look at a URI and know what it does. URIs aren’t overloaded by packing the query string. Behavior is consistent and payloads for a given URI have the same format no matter whether you GET, POST, or PUT. But the RFCs (rightly) do not attempt perfection – real-world experience shows you rarely POST exactly what you GET. Also, ambiguities in the specs used to be settled by the whims of browsers. But now those specs are being used to connect systems instead of browsers to servers. I’m not ready to look a trading partner in the eye and say “hmm, let’s see how Opera handles an HTTP 415”. I’ll instead go hide in a closet and do just that.</span><br /><br /><span style="font-family:arial;">So, even with the flawed cast of characters you see a lot of whining about – HTTP, URI, XML, and even (gasp!) XML Schema – the pieces are there to build good systems that also make great constituents in anyone’s SOA. The specs, with </span><a href="http://www.w3.org/TR/xmlschema-1/"><span style="font-family:arial;">one glaring exception</span></a><span style="font-family:arial;">, are easy to digest. My advice is to start by thinking about a URI strategy that people can follow intuitively. Don’t underestimate the value of a good URI set or the design skills it takes to build. I would also ignore worries about pissing off the Old-Hands of the Web by somehow not using the Web exactly as intended. Their crystal balls weren’t clearer than anyone else’s and these guys do put pants on the usual way (not that I’ve personally witnessed it). </span><br /><span style="font-family:arial;"><br />And on </span><span style="font-family:arial;">Harry’s</span><span style="font-family:arial;"> <a href="http://devhawk.net/2007/05/25/This+Isnt+The+Droid+Im+Looking+For.aspx">point </a>that REST and CRUD, I agree that </span><a href="http://www.pluralsight.com/blogs/tewald/default.aspx"><span style="font-family:arial;">Tim</span></a><span style="font-family:arial;"> was simply advising people not to limit their comprehension of REST around entities accessed via GET and PUT. REST resources are like views that may or may not be underpinned by an entity model. REST state transitions use URIs to label and invoke services which may or may not use an entity/CRUD programming model under the hood. I agree with Tim in that the presence of those URIs as links is what actually defines REST – NOT simply that you’ve labeled data with URIs.<br /></span>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-14219297.post-49473023896349868672007-05-16T00:49:00.000-07:002007-05-16T06:30:49.716-07:00REST Protocols are the Service Layer<p>Business operations look a lot like protocol state machines. One of the areas I’ve been looking at (for several years) is how to better leverage state machine thinking into data-driven applications design. I read the REST dissertation, but didn’t see how Roy Fielding’s notion of state transitions applied to my projects. A couple of years ago, <a href="http://www.from9till2.com/">David</a> and I looked at how server-side state transitions could drive a service layer (and more). We talked about writing up a paper, but I flaked out so David went ahead with a <a href="http://www.from9till2.com/PermaLink.aspx?guid=ab3fd85b-4f2a-4ea3-be61-b8bed6bbaf48">post</a>. </p><p><br />So, I read <a href="http://www.pluralsight.com/blogs/tewald/default.aspx">Tim Ewald’s</a> posts about REST (first post is <a href="http://pluralsight.com/blogs/tewald/archive/2007/04/26/46984.aspx">here</a>) with a lot of interest. Tim’s description of REST is centered on client protocol state transitions and, critically, ensuring responses to GET requests include the URIs (links) that transition between client states. Up until now, I ignored the idea of URIs for client states that are not mapped to system state transitions on the server. I also questioned the wisdom of including links at all in REST-style responses. Here’s why.</p><p><br />A classic blunder in applications development is designing the programming model (and it's API) around a specific client interaction style. An acid test for an API is how well it works for different calling scenarios, especially those unknown at design time. On that second point, only time can tell you if you have a winner. SOA itself was invented largely to counter unsuitable APIs (and fill space on collateral). Service layers usefully loosen the coupling (you don’t need my binaries to call me) and serve as adapters between an application and the potentially numerous and radically different callers.</p><p><br />So, isn’t it a slippery slope to presumptuously inject URIs into messages for operations that might not apply to the client’s intention? In Tim’s <a href="http://pluralsight.com/blogs/tewald/archive/2007/04/27/47031.aspx">example</a>, the server returns a list of itineraries along with URIs that transition the client to one of 2 potential next client states: getDetails or Reserve. One of these URIs changes system state and the other doesn’t. Rightly, you use GET on one of the URIs and POST for the other (although the example doesn’t indicate which is which). The “getDetails” tag bothered me because in a non-trivial case there might be numerous operations available relative to the resource. Or, the resource might be a summarization (like a report). If the data has a thousand customers IDs and there are 10 or more links possible per customer – you get the point. Why offer up a bunch of links that the client won’t use and potentially outweighs the actual data? Is it even possible to determine the complete set of links?</p><p><br />What I failed to remember, until now, is that data and protocols are independent. There can be multiple protocols with the same data as the starting point – but not the same URI. The URI identifies what data the caller wants and under which protocol. </p><p align="center"><br /><em>Data + protocol = resource -> URI</em></p><p><br />The protocol embodies the caller’s intentions and the server’s constraints. Protocols are easy to secure (URI = security descriptor) and they work like interfaces for data hiding. My mistake until now was in believing that:</p><ul><li>Each entity (I work on ERP apps and we still think about entities) is a resource that has a protocol defined by a system state sequence.<br /></li><li>The only useful links in resources are URIs for posting value changes or transitioning states<br /></li><li>Views of data representing many resources have no links because it’s difficult to generate a complete set and the caller’s intentions are unknown.</li></ul><p><br />After working through Tim’s description, I’ve reached different conclusions about how to design a REST-style system:</p><ul><li>I still like defining a state sequence and change constraints for entities in the application. It’s a good programming model for ERP because the server-side state transitions are handy places to hang workflow logic. Also, entity/life-cycle thinking aligns well with the business operations.<br /></li><li>Systems have protocols that describe data interactions with reasonably specific purposes in mind. The views passed in and out of the system are resources.<br /></li><li>A URI identifies a resource, a protocol, and where in that protocol the caller sits.<br /></li><li>Resources contain links to other resources or invocation points specific to the protocol. This solves my issue with (potentially) large numbers of unhelpful links.<br /></li><li>Protocols buffer the application from conversations – protocols are the new “service layer”.<br /></li><li>Getting back to the first point, a “default” protocol can be derived from the server-side state sequence for an entity.</li></ul><p><br />Protocols can carry links to places other than the initial server and resources can be more than just XML representations of entity data – think of the mash-up potential. The URI – via the protocol – tells the system what data to get, what format to use, what stylesheet to apply, and which links are useful to the caller. Adding new protocols to a system needs to be an easy thing to do.</p><p><br />I thought I knew REST and had taken a decent crack at applying it to ERP applications. This re-think has me wondering what else I haven’t figured out yet. In the meantime, I’m looking back at our web services, metadata, and customer-driven scenarios to see what features protocol definitions might need. The goal is to find a way to make it easy for our developers and our customers to create protocols. I’ll write up what I find.</p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-14219297.post-1175057365495396802007-03-27T22:49:00.000-07:002007-03-28T07:39:37.273-07:00WS-I and Transparency: WTF?<span xmlns=""><p>Over 300 people from 100 companies came to San Francisco in early April 2002 for the first ever WS-I community meeting. I was there. Like most of the attendees, I had never worked on a standards effort of any kind. But the "founding board" members of the WS-I – IBM, Microsoft, Oracle, SAP, HP, Intel, etc. – had each reached out to their customer bases to drum up participation. The carrot hung out the window for us smaller companies to sniff was that we could hang out with the big guys to work on web services standards and, importantly, we could get a WS-I conformance logo to grace our product.<br /></p><p>It's exactly five years later and I'm in Delray Beach Florida for the WS-I Spring 2007 Community Meeting. The event's web page (for members) list says I'll be one of 32 people attending. Only 18 companies are being represented. The WS-I Board is made up of 11 companies, who, one would think, are obligated to participate. So the "community" beyond the leadership team will be me plus 6 other companies. Over 70 other companies pay dues to keep their up their membership, but I'll wager that many (like my company) have fallen out of good standing.<br /></p><p>WHAT THE HELL HAPPENED?<br /></p><p><strong>Marketing</strong> – There are test programs that determine whether an application conforms to a WS-I profile. You don't need to belong to the WS-I to run them – or even claim you've run them successfully. You can just say your app conforms to a WS-I profile. It's up to someone else to call your bluff. With membership optional and conformance claims free for the taking, it's no wonder the membership is practically extinct. My company's dues are paid out the marketing budget. Hmmm.<br /></p><p><strong>Resources</strong> – A tenet behind the WS-I charter says that, for any profile, a set of sample applications and a test suite identifying test assertions will be developed on multiple platforms. The idea was to show best practices and demonstrate the potential interoperability benefits behind the effort. But developing this work is enormously expensive. Microsoft, IBM, SAP, Fujitsu, and others have deep pockets. But over time the contributions have declined to the point where the organization can't "afford" to charter new profile work. But there is a bigger challenge to the long-term solvency of the WS-I…<br /></p><p><strong>Transparency</strong> – This is the big one. To be fairer than necessary, the WS-I was conceived at a time when web services hype was still growing and the founding members were deeply suspicious of each other's motives (the players, I think: IBM, Microsoft, Oracle, HP, Intel, SAP, Accenture, BEA, and webMethods). The hype issue has almost disappeared. And the mutual suspicion problem is in a little better shape except for a lasting grudge within the WS-I Board over the fate of an effort 3 years ago to incorporate SOAP with attachments into the WS-I Basic Profile. </p><p>But there is a bigger problem. There are 2 WS-I organizations: the working groups and the WS-I Board. Of course, all organizations have boards that are independent and need to be able to deliberate privately. But the WS-I Board – and I mean the institution, not the individuals or their companies – is frankly an anathema in the face of the WS-I's public mission.<br /></p><p>WS-I working groups deliberate within narrowly defined charters, but the discussions are open. Members are free to discuss and describe deliberations wherever they like. The only caveat – and it's significant – is that no "intellectual property" developed by working group members can be published as WS-I material without Board approval and (depending on the nature of the work) as membership-wide vote. Most members can live within those constraints. But the Board is another issue.<br /></p><p>The WS-I Board was intentionally designed to be a vendor-driven black-box that controls what is advertised (with a bit of mockery) as a membership community. This isn't hyperbole (despite my usual tendencies). Here are the rules that are driving the WS-I straight into the ground:<br /></p><p>- ALL WS-I Board proceedings are confidential – even to the membership. There was an effort by IBM a year ago to prove that Board members could discuss what happened in meetings even if the minutes were secret. But that theory was rejected by others in the Board as a reckless interpretation of the bylaws.<br /></p><p>- Virtually all proposed Board actions – like approving a document for publication or launching profile work – requires 9 of the 11 Board members to vote "yes". Abstentions count as "no" votes (a rule which is potentially being addressed)<br /></p><p>- Of the 11 Board positions, 9 are allocated on a permanent basis to "founding members". The other 2 are elected. Of those 2, one is effectively (and justifiably) locked up by Sun Microsystems, who was obviously blackballed from participating when the WS-I was formed. But the point is that there is no real opportunity for WS-I members to become part of the leadership.<br /></p><p>- The Board determines what work is in scope for the WS-I, but has been utterly unable to declare the criteria behind making this determination. Part of the reason is that the Board is trapped by a wacky bylaw treatise inured with a declaration of unambiguous consensus. In other words, the board can't give its own membership, even internally, any feedback without 9 of 11 votes affirming both the act and the message.<br /></p><p>How in the hell can members have any involvement in determining future profiles and other work for the WS-I when (a) there are precious (and highly subjective) rules about what work is in scope and (b) Board members can't agree about what, the bylaws allow them to say to their own membership? The WS-I is the epitome of dysfunction and the reason is mainly the misguided invention of that whacked-out cocoon in which the Board comfortably resides.<br /></p><p>So, I'm participating in a Working Group chartered to come up with requirements for future WS-I profile work. To be fair, I've been away from the WS-I for a year, so I feel bad about making waves on the first day back. But I know the workings having chaired their XML Schema Planning WG in 2004-2005. Anyway, the Requirements WG is only allowed to submit draft Working Group charters for WS-I Board approval. Astonishingly, our Working Group can't even give feedback to OASIS or ask the W3C a question without a 60-day approval processes and WS-I Board vote. We can't contact the organizations that own the material the WS-I is considering for potential profiling efforts<br /></p><p>I'm going to propose that the WS-I Requirements working group take several actions: (1) Questions and feedback to other organizations are deemed "non-material work", which means the whole WS-I membership does not need to vote before publication. (2) That approval to make materials developed by working groups or committees public requires a simple majority (I can be talked into a 2/3 margin) to be approved for publication. (3) That votes by the WS-I Board on work submitted by a working group or committee are made public to the membership (and by extension – the world). Of course the Board and the bylaws will have to yield and be amended, respectively.<br /></p><p>Without these reforms (and others), the WS-I will continue its utter collapse into a small club of vendors seeking to satisfy a few constituents that, for now, are propping up the justification behind the entire organization. The rest of the world (spot the pun) will overtake the WS-I like callous covers a splinter on your heel. And while I strongly think there are much better programming models on the web for many situations, I also think there is a large place in the economy for WS-*. Large organizations will need the mechanisms behind WS-* as provable standards for to bind their touch points. And large IT companies will wave WS-I profiles, and the like, as bespoke competencies fulfill them. <br /><br />Maybe the WS-I should be disbanded for the shear arrogance of its founders who squander the enthusiasm and contributions of the membership. But starting over also sucks.<br /></p></span>Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-14219297.post-1159649026870246642006-09-30T13:43:00.002-07:002006-10-16T07:07:53.423-07:00SOA vs. Service-Oriented Applications<span xmlns=""><p><span style="font-family:arial;font-size:85%;">[Updated 16-Oct to fix a typo]</span></p><p>Service-Oriented Architecture (SOA-arch) and Service-Oriented Applications (SOA-app) are different animals with a common name. I think this is a major point of confusion that hasn't yet been widely recognized. Just like the fact that <a href="http://appside.blogspot.com/2006/07/workflow-flavors.html">three points of view</a> exist about what "workflow" means. Here is the difference and I've also included advice for building a service-oriented application.<br /></p><h1>Service-Oriented Architecture<br /></h1><p>Here is a quick SOA-arch diagram I built after looking at various resources on the Web:<br /></p><p style="TEXT-ALIGN: center"><a href="http://photos1.blogger.com/blogger/951/1280/1600/BigSOA.1.jpg"><img style="MARGIN: 0px 10px 10px 0px; CURSOR: hand" alt="" src="http://photos1.blogger.com/blogger/951/1280/320/BigSOA.1.jpg" border="0" /></a><br /></p><p style="TEXT-ALIGN: center"><strong>Service-Oriented Architecture Layers<br /></p></strong><p>Those like me who work on commercial ERP applications find this a little humiliating. All that work to build an enterprise application – and we wind up <strong><em>at the bottom of the SOA stack?</em></strong> We clearly have thin skins. Moreover, this implies that one of the chief reasons that SOA-arch exists is because apps can't integrate with each other and we have to fool them into working together. SOA-arch also exists to crack apart applications and cobble them back together to match a business process not envisioned by the application's authors. It's an expensive and invasive process.<br /></p><p>ESB providers will eternally demand credit for inventing SOA-arch. But I think we should think about the "day after" effect of web services. People began putting SOAP layers around their existing applications or building new applications from scratch on the WS-* stack. But getting some type interoperability only solves one of the <strong><em>many</em></strong> problems around integrating complex apps into a specific business. There was nothing SOAP could do to match an SAP-formatted employee record to an HR-XML version. SOAP can't make my application email the HR department if I browse <a href="http://www.monster.com">http://www.monster.com</a> from my desk. SOA-arch puts those things together. SOA-app gives enterprises a way to control and access data maintained by a given application.<br /></p><h2>Service-Oriented Applications<br /></h2><p>SOA-app is about designing applications aggressively to be good citizens in someone's SOA-arch implementation. Service-oriented applications have a basic stack diagram as well, <a href="http://appside.blogspot.com/2005/09/canonical-soa-stack.html">I mentioned some months ago</a>. I won't repeat that explanation, but here is a more colorful picture:<br /></p><p style="TEXT-ALIGN: center"><a href="http://photos1.blogger.com/blogger/951/1280/1600/SOApp.jpg"><img style="MARGIN: 0px 10px 10px 0px; CURSOR: hand" alt="" src="http://photos1.blogger.com/blogger/951/1280/320/SOApp.jpg" border="0" /></a><br /></p><p style="TEXT-ALIGN: center"><strong>Service-Oriented Application Stack<br /></strong></p><p>SOA-app revolves around designing elements of the enterprise application <em>troika</em> – message, data, and code – specifically to make it much easier to incorporate the resulting application into your SOA-arch and also promote a dynamic system that tolerates change more effectively. So, my app might be at the bottom of your SOA-arch stack, but I'll make sure my app is your best friend rather than some opaque, inflexible, monster. SOA-app means changing some classic approaches and, in some cases, fighting the dev tools to make that happen. Here are some key ideas behind SOA-app:<br /></p><ul><li>Make sure callers can <strong><em>mix business intent</em></strong> into a single unit of work. Suppose you allow callers to update customers or update suppliers. Make sure you also allow callers to update a customer <em>and</em> a supplier in a single call. Each business function is represented by a distinct message part format, so your posting function should be able to take an arbitrary list of message parts as input and return a corresponding set of message parts containing results (if any). The reason for this is that SOA-arch may have an entity data aggregation function that updates multiple entities at once. Wouldn't it be nice if your application could accept composite documents with less initial shredding?<br /></li><li>Make sure your messages <strong><em>tolerate unknown or incomplete content</em></strong>. If a caller adds extra data that some other part of the SOA-arch needs, just ignore it. For XML, many type serializers do this pretty well. But for binary serializers, this is practically impossible. On the flip side, make sure your messages make "everything optional". There are reasonable limits here. For example, when a caller wants to create a new customer but doesn't include a name or address. Again the reason is to make life easier for callers syncing fractions of records.<br /></li><li>Use an <strong><em>inversion of control</em></strong> framework that lets your implementers re-sequence or replace logic declaratively. Windows Workflow Foundation is pretty well designed for just this kind of role. Going this route obviously has a big impact on the overall programming model of the system. For one thing, routines that update data should basically not ever return anything directly to the caller because makes the coupling too tight between components. The scenario to keep in mind is when an implementer needs to suspend a function and execute an approval routing or clear some business semaphore (like a credit hold).<br /></li><li>Give callers reasonable <strong><em>access to the query processor</em></strong>. One of the problems with most interfaces – no matter the language or platform – is that callers often can't get the exact data they need unless some designer first thought to add the right function. One of the biggest requests I see is for a correlated sub-query that our product managers didn't think of. I'm not saying that you should allow callers to send SQL – or even fragments of SQL. But you should implement some mechanism that lets callers retrieve data by submitting a query tree that can be vetted by your framework and that you can secure.<br /></li><li>Stay very <strong><em>document-oriented</em></strong>. This is easier in business applications because the data lines up well with real-world business artifacts (orders, shipments, etc.). Many application designers over-design the interface library by building a matrix of functions required by caller roles. The problem is that the matrix won't have an entry for many interactions required in the SOA-arch. Come up with a CRUD<em>x</em> standard for your entities and use that to determine message formats (ideally, via a transform). The "x" is whatever critical behavior a particular entity may possess. Keep the number of members of "x" as small as possible. For ERP, we've been able to do it with a single "t" – for "transition" which means to move a record from one transactional state to another (order:new -> order:cancelled).<br /></li></ul><p>It's troubling that for the past year I've talked to many people about SOA without anyone realizing we were somewhat cross-purpose. I've wanted to talk about the above points for some time, but when I would go over them with other solutions architects, it seemed like we weren't on the same page. Hopefully distinguishing between SOA-arch and SOA-app will be useful going forward.<br /></p></span>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-14219297.post-1157228867335533422006-09-02T13:22:00.000-07:002006-09-02T13:30:36.646-07:00Diagnosis: XML Fetish<span xmlns=""><p>The apparent medical experts who wrote According to <a href="http://www.amazon.com/Microsoft-Windows-Communication-Foundation-Hands-on/dp/0672328771/sr=8-1/qid=1157228283/ref=sr_1_1/104-4828900-0622336?ie=UTF8&s=books"><em>Microsoft Windows Communication Foundation – Hands-On</em></a>, have – in Chapter 3 – diagnosed me as having an “<strong>XML Fetish</strong>”:<br /></p><blockquote><p><span style="color:black;"><em>Yet practitioners of contract-first development, working in the XML Schema language in an XML editor, tend to become distracted from those core concerns and start to worry about exactly how the data is to be represented in XML. Consequently, they begin to debate, among other things, the virtues of various ways of encoding XML, and become highly suspicious of anything that might inhibit them from seeing and fiddling with XML. The XML becomes a fetish, falsely imbued with the true virtues of contract-first development, and, as Sigmund Freud wrote, "[s]uch substitutes are with some justice likened to the fetishes in which savages believe that their gods are embodied" (1977, 66).<br /></em></span></p></blockquote><p>Wow – I’m practically paranoid! I think I understand the points the authors were trying to get across: Let the XML serializers do their job and stay out of the way. The preaching and the lame attempt to bring Freud into the discussion really put me off.<br /></p><p>To (badly) paraphrase Norma Desmond, XML <em>is</em> big. It’s the toolkits that got small. Experience tells me that you actually <em>have</em> to keep a sharp eye on the WSDL and XML emitted from just about any toolkit. The W3C has a <a href="http://www.w3.org/2002/ws/databinding/">working group</a> basically dedicated around that fact.<br /></p><p>Let’s say I have a service that accepts purchase order lines. Purchasers can buy things that they receive and stock (items) or services that will be rendered. Items have a code and a quantity; services have a free-form description and an estimated cost. How do I create a service description for an operation that gives the caller that kind of choice in what gets sent?<br /></p><p>In XML Schema, I can simply say “xs:choice”.<br /></p><p>In C# (or most languages), I can implement this choice using a number of tactics. The most obvious way to go is polymorphism: create a base class <em>POLine</em> and two descendent classes <em>StockedItem</em> and <em>Service</em>. The function prototype then takes the base class as an argument. Hmm – try that once and see what kind of interoperable service description you get. My bet is something like <em>xs:any</em>.<br /></p><p>To be fair, .NET has added a ton of attribution to the XML Serialization mechanism. You can implement a choice construct that works pretty well. But the attributes you need to faithfully build such a C# types are very specific to .NET and probably even less intuitive than XML Schema (never thought I would say <em>that</em>). Further, you have no guarantee that Microsoft (or any toolkit vendor) won’t change the XML Schema constructs mapped to a class construct when new framework versions shipped. We had this happen between .NET 1.1 and .NET 2.0.<br /></p><p>I tend to build data contracts using XML Schema first. Then I roll the schemas through the various class generators to see how good the output is. Then I build a service using the generated classes to see how good the WSDL will be. Then I build yet another set of types using the generated WSDLs to simulate what service consumers will experience. Finally and if all looks good, I throw all of it away except for the original schema, which gets incorporated into the service description directly and, sometimes, into the service code base via XML Schema validation. I don’t like building static types for message payloads or data domain members. But that’s a whole other topic!</p></span>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-14219297.post-1155066487825094172006-08-08T12:43:00.000-07:002006-08-08T12:53:37.696-07:00ERP Data as Hypermedia<span xmlns=""><p>I work on ERP applications for a living, so I tend to see lots of ways business intent is manifested in an API. For the past few years, I’ve been trying out different architectural ideas to merge the success of the Web to enterprise application API development. The two aspects that have garnered the most payback have been (a) casting the data domain as a hypermedia set and (b) rationalizing constraints around GET, PUT, and POST to convey business intent and manage work. This blog entry is about (a). Making the data domain flexibly addressable is critical to workflow and EAI, which in turn makes it critical to SOA -- no matter how REST or WS-* might be utilized.<br /></p><p>Most reasonably normalized databases already have clear semantics for referential data and child data. Reference columns link tables together by record ID. Child tables inherit the ID of their parent plus their own additional anonymous key value. The referential aspects form hypermedia-like links and the parent-child aspects provide document scope (not to mention the nice fit with XML).<br /></p><p>Putting these together means that we can map a record as a URL using the same key values that identify records in a database. We can also extend the URL format across referential values. Here are some examples from an implementation we put out several years ago:<br /></p><p><a href="url://MySite/orders/101.xml"><span style="font-family:courier new;">url://MySite/orders/101.xml</span></a> returns the XML representation of order “101” in all its glory (details and all)<br /></p><p><a href="url://MySite/orders/101/1.xml"><span style="font-family:courier new;">url://MySite/orders/101/1.xml</span></a> returns just the first line item of order “101”.<br /></p><p><a href="url://MySite/orders/101/CustomerID.xml"><span style="font-family:courier new;">url://MySite/orders/101/CustomerID.xml</span></a> returns the ID of the customer<br /></p><p><a href="url://MySite/orders/101/CustomerID/Customer.xml"><span style="font-family:courier new;">url://MySite/orders/101/CustomerID/Customer.xml</span></a> returns the entire <em>customer record</em> for the customer identified in the “CustomerID” link<br /></p><p>The last item is the most significant, because you can traverse the entire data domain – if the linkages exist – and access data many degrees of separation away from your starting point. We’ve been doing this for several years now in Business Process Management (BPM) architectures because the data one needs to make a business policy decision is <em>rarely</em> in the message. In other words, workflow-driven architectures need expression formats that bind the data in question with data already in the system.<br /></p><p>For example, an administrator might set a policy to disallow orders for items belonging to a product class that itself is under approval hold by, say, the U.S. FDA.<br /></p><p><a href="url://message/orders/lines/items/holdnotices/fda.xml"><span style="font-family:courier new;">url://message/orders/lines/items/holdnotices/fda.xml</span></a><span style="font-family:courier new;"><br /></span></p><p>Or, you don’t want to allow orders for customers on credit hold: <a href="url://message/orders/CustomerID/holdnotices/credit.xml"><span style="font-family:courier new;">url://message/orders/CustomerID/holdnotices/credit.xml</span></a><br /></p><p>Note the “message” bit rather than “MySite” in the previous examples. The idea is to not only traverse entities in a database – it’s also to traverse <em>from</em> the message payload <em>to</em> the database in one expression. Non-programmers can actually read this and figure out what it’s doing. That capability is important in BPM because line managers control the policies in force. Line managers understand hierarchies like directory structures. The key is to keep the URN constructs uncluttered and simple.<br /></p><p>That seems straightforward, but you have to work out how to pipeline referential constructs through the entity chain for multiple situations: inner-join, outer-join, existential, etc. You also have to work out how to constrain certain kinds of URLs that bring back too much data. Our initial work let you do something like this: <a href="url://MySite"><span style="font-family:courier new;">url://MySite</span></a>. It spit out every record in the database.<br /></p><p>BTW, you can swap “.xml” for “.xsd” and get a schema for the results. You can also use “.htm” to get an HTML rendering. In fact, you can go to town with MIME types and style sheets to provide lots of viewable interpretations of whatever data is being sought. That’s pretty powerful.<br /></p><p><br /></p><p><br /></p></span>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-14219297.post-1154732868263251662006-08-04T16:05:00.000-07:002006-08-04T16:07:49.386-07:00Are GET and PUT Symmetrical in REST?Does REST require symmetry? In other words, must the payload schema for messages I GET be the same as the schema for messages I PUT? I hope not, because (in my world) what I PUT is rarely a new, complete authoritative version of the data. In the work I've done, the PUT (or POST) payload explicitly indicates whether the item is new (in which case it is authoritative) or if I am changing an item that already exists.<br /><br />The REST idea – in my mind – is that (a) you can address instances of data unambiguously and completely and (b) that what you GET and what you PUT (for a given URL) are semantically related -- but not necessarily exact copies.Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-14219297.post-1154580548975638132006-08-02T21:49:00.000-07:002006-08-03T08:24:10.750-07:00Stephan's List: REST vs. SOAP<a href="http://www.innoq.com/blog/st/2006/07/31/rest_vs_ws_statistics.html">Stefan’s List</a><span style="font-family:Calibri;"> is an interesting effort to categorize some very good bloggers into REST, SOAP, and (um) “either” camps. I know a few people on both sides of the isle and I don’t think any of them are on some sort of web-oriented death march. I mainly think about how business intent is most effectively expressed in message payloads – something neither REST nor WS-* are really helping with (it’s semantics after all).</span><br /><br /><span style="font-family:Calibri;">My friend Paul Downey (</span><a href="http://blog.whatfettle.com/">http://blog.whatfettle.com/</a><span style="font-family:Calibri;">) of British Telecom (</span><a href="http://blog.whatfettle.com/archives/000406.html">erroneously placed</a><span style="font-family:Calibri;"> in the “wrong” camp in an earlier version of Stefan’s List) put together a brilliant slide deck (</span><a href="http://www.whatfettle.com/2006/05/WebServices.pdf">pdf</a><span style="font-family:Calibri;">) that unambiguously (and literally) illustrates his point of view about WS-*. I’ve been meaning to steal from it for some time. BTW, Paul is chairing a </span><a href="http://www.w3.org/2002/ws/databinding/">W3C working group</a><span style="font-family:Calibri;"> to advise toolkits about mapping language constructs to XML Schema constructs – something that will help both sides of the WS isle. I think Paul’s working group will enable more XML Schema constructs to be supported in the mainstream, which in turn increases the vocabulary I can employ to represent my business intent.</span><br /><br /><strong><span style="font-family:Calibri;">REST Doesn’t Obviate WS-*</span></strong><br /><span style="font-family:Calibri;">There is a VERY large IT constituency that relies heavily on modeling non-functional requirements. Pioneers have the luxury of trying these things out and mitigating the shortcomings as they go. Obviously, current enterprises want to extend their architectural core concepts into more open realms. But they need to prove that service-level requirements are achievable </span><em><span style="font-family:Calibri;">and </span></em><span style="font-family:Calibri;">that non-functional aspects can be modeled using approved best practices.<br /></span><br /><span style="font-family:Calibri;">WS-* has grown around those needs. The specs provide CYA support for the big IT shops. I sometimes get upset that WS-* is no longer simple or particularly exciting. But people are often compelled into multilateral infrastructures and to make their complex interactions interoperable. There’s apparently a lot of money in achieving that.<br /></span><br /><span style="font-family:Calibri;">PS. For the record, I use REST and tunnel it through SOAP and/or WS-* where needed.</span>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-14219297.post-1154181606877095702006-07-29T07:00:00.000-07:002006-07-29T07:01:33.010-07:002.0 is to SOA what SOA is to ESBAbout a year ago, I put out a blog post describing ambitions for SOA as an architectural basis within a single application domain. It was about relating messages, code, and data bi-directionally using transforms. Formal descriptions for those transforms could then become a new order of patterns in software architecture. The goal wasn’t to solve the gap between describing requirements and cutting code. It was to link the overall solution concerns more intelligently and pave the way for more declarative aspects and dynamic features.<br /><br />The mistake I made was in trying to put a moniker around the concept – SOA 2.0! At the time, the ESB community was pushing really hard to say ESB==SOA. I wanted to distinguish architecting integration solutions – the bread and butter of an ESB – from architecting the traditionally opaque integral applications within an enterprise. <br /><br />But calling it SOA 2.0 was clearly a bad choice. It’s sort of nice to be near the top of a search result (Google: “<a href="http://www.google.com/search?hl=en&lr=&rls=GGLR%2CGGLR%3A2006-12%2CGGLR%3Aen&q=soa+2.0&btnG=Search">SOA 2.0</a>”). Maybe your experience will vary, but I cringe when I see what company I have on the same page (but I did beat Oracle by a few months!). The post has been linked by others (<a href="http://hinchcliffe.org/archive/2005/08/27/1817.aspx">Hinchcliffe</a>, <a href="http://markclittle.blogspot.com/2006/05/soa-20-ignorance.html">Little</a>) but not exactly the way I hoped. A rookie blogging mistake if there ever was one.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-14219297.post-1154026167065512792006-07-27T11:43:00.000-07:002006-07-27T11:49:27.563-07:00Workflow Flavors<p>“Workflow” came up during a panel discussion I was in at Microsoft’s Tech-Ed conference this year. The word itself – like many IT terms these days – is overloaded to the point where it is hard to distinguish what a given “workflow toolkit” is meant to achieve. When explaining workflow concepts, I’ve started by making sure the audience understands the differences between 3 major categories of workflow:</p><ul><li><em>Human workflow</em> is where information is conveyed to real people for action or simple notification. Workflow systems that present documents to employees have been around for years – so human workflow wins the right to actually use the word “workflow”. The key issue for human workflow is to have a system where non-technical users can actually program the routing. </li></ul><p> </p><ul><li><em>Orchestration</em> is collaboration between application domains (and, by extension, between enterprises). BizTalk, Sonic, and ESB’s are all orchestration tools even though their target markets and general features may differ widely. </li></ul><p> </p><ul><li><em>Service Agents</em> route execution of logic <strong>within</strong> a specific application domain. This is the sweet spot for Workflow Foundation. Service agents are under-served by toolkit vendors, which is surprising given the demands for content-based logic routing in the SOA world. </li></ul><p>The trick for WF is to prove it can actually become the primary message pump for an enterprise application. But what drives me crazy is when people think of WF as some sort of “BizTalk Light”, which does both products a disservice.</p><p>PS. I haven’t blogged in quite a few months for no reason other than not being sure what to go into. So I thought a gentle entry like this might get the wheels moving forward.</p>Unknownnoreply@blogger.com2tag:blogger.com,1999:blog-14219297.post-1134413717282233152005-12-12T10:55:00.000-08:002005-12-12T11:25:48.960-08:00The W3C Schema Patterns WG is not Misguided<p><span style="font-family:Trebuchet MS;">Dare Obasanjo’s </span><a title="The Misguided Efforts of the W3C's XML Schema Patterns for Databinding Working Group" href="http://www.25hoursaday.com/weblog/PermaLink.aspx?guid=69af51d0-ab7e-411c-95d4-429febfbfdea"><span style="font-family:Trebuchet MS;">recent post</span></a><span style="font-family:Trebuchet MS;"> labeling the W3C </span><a title="W3C XML Schema Patterns for Databinding Working Group" href="http://www.w3.org/2002/ws/databinding/"><span style="font-family:Trebuchet MS;">W3C XML Schema Patterns for Databinding Working Group</span></a> <span style="font-family:Trebuchet MS;">as misguided seems like an overreaction. The toolkit vendors put out bad XML Schema processors and invented those leaky abstractions in the (misguided?) rush to make XML painless and web services a transparent feature for programmers with typical skillsets and approaches. </span></p><p><span style="font-family:Trebuchet MS;">Many (including me) think that moving to XML as a primary integration mechanism for applications should a great step forward. But many IT staff who actually have to link different apps together are complaining that their job is much harder now than it ever was. Industry consortia are having a hell of a time publishing good standardized schemas because the inclusion of some seemingly innocuous XML Schema features will unknowingly break constituent implementations.</span></p><p><span style="font-family:Trebuchet MS;">You can’t tell developers to simply avoid statically typed languages (at least not yet). You also can’t tell developers to wait a bit longer and the toolkit vendors will somehow watertighten their abstractions in an interoperable way. You *can* tell developers to avoid leaky abstractions to process XML, but you get resistance (which is regrettable). Worse, developers have to sometimes fight their toolkits to even do get to the message payload.</span></p><p><span style="font-family:Trebuchet MS;">So, I don’t know what is so wrong about the W3C trying to alleviate the situation by attempting to shine a light on issues that seriously impact users. It may perpetuate XML <-> OO binding, which many people — including me — think is a problematic strategy. </span><span style="font-family:Trebuchet MS;">But maybe the W3C can get the Infoset in more hands sooner. More people can walk before they run, if you will. </span></p><p><span style="font-family:Trebuchet MS;"></span></p><p><span style="font-family:Trebuchet MS;"></span></p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-14219297.post-1134094779971117542005-12-08T18:19:00.000-08:002005-12-09T08:48:01.086-08:00UPA is your Friend (Repost)<p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="font-family:'Trebuchet MS';font-size:10;"><span style="font-size:85%;">The W3C XML Schema specification has a rule called unique particle attribution (UPA) that confuses, well, many.<span style="mso-spacerun: yes"> </span>And after 4+ years of XML Schema in the wild, it still amazes me how inconsistent toolkits handle the issue.<span style="mso-spacerun: yes"> </span>One toolkit – I think XMLSpy – at one time rebelled against UPA by enforcing the rule only if the user wished it.<span style="mso-spacerun: yes"> </span>Just this week, I found a discrepancy in handling UPA between SQL 2005 and .NET Framework 2.0 (more on that later).</span></span></p><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="font-family:Trebuchet MS;font-size:85%;"></span><br /></p><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="font-family:'Trebuchet MS';font-size:10;"><span style="font-size:85%;">One problem in trying to comply with the UPA rule is that it is hard to describe in words.<span style="mso-spacerun: yes"> </span>Here are a couple of examples to get the point across.<span style="mso-spacerun: yes"> </span>This schema type is perfectly fine:</span></span></p><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="font-family:Trebuchet MS;font-size:85%;"></span><br /></p><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="font-family:'Trebuchet MS';font-size:10;"><?xml:namespace prefix = o /><o:p></o:p></span></p><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><o:p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:element</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" > </span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:red;" >name</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >=</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;" ><span style="color:#000000;">"</span><span style="color:blue;">MyType</span><span style="color:#000000;">"</span><span style="color:blue;">><o:p></o:p></span></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><span style="mso-tab-count: 1"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:complexType</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><span style="mso-tab-count: 2"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:sequence</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><span style="mso-tab-count: 3"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:element</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" > </span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:red;" >name</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >=</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;" ><span style="color:#000000;">"</span><span style="color:blue;">Foo</span><span style="color:#000000;">"</span><span style="color:blue;"> /><o:p></o:p></span></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><span style="mso-tab-count: 3"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:element</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" > </span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:red;" >name</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >=</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;" ><span style="color:#000000;">"</span><span style="color:blue;">Foo</span><span style="color:#000000;">"</span><span style="color:blue;"> /><o:p></o:p></span></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><span style="mso-tab-count: 2"></span></</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:sequence</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><span style="mso-tab-count: 1"></span></</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:complexType</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ></</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:element</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="font-family:'Trebuchet MS';font-size:10;"><span style="font-size:85%;">But this type violates UPA:</span></span></p><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="font-size:85%;"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><br /></p></span></span><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:element</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" > </span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:red;" >name</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >=</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;" ><span style="color:#000000;">"</span><span style="color:blue;">MyType</span><span style="color:#000000;">"</span><span style="color:blue;">><o:p></o:p></span></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><span style="mso-tab-count: 1"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:complexType</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><span style="mso-tab-count: 2"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:sequence</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><span style="mso-tab-count: 3"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:element</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" > </span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:red;" >name</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >=</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;" ><span style="color:#000000;">"</span><span style="color:blue;">Foo</span><span style="color:#000000;">"</span><span style="color:blue;"> </span><span style="color:red;">minOccurs</span><span style="color:blue;">=</span><span style="color:#000000;">"</span><span style="color:blue;">0</span><span style="color:#000000;">"</span><span style="color:blue;">/><o:p></o:p></span></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><span style="mso-tab-count: 3"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:element</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" > </span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:red;" >name</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >=</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;" ><span style="color:#000000;">"</span><span style="color:blue;">Foo</span><span style="color:#000000;">"</span><span style="color:blue;"> /><o:p></o:p></span></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><span style="mso-tab-count: 2"></span></</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:sequence</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><span style="mso-tab-count: 1"></span></</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:complexType</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ></</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:element</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >></span><span style="font-family:'Trebuchet MS';font-size:10;"><o:p></o:p></span></p><br /><span style="font-size:85%;"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"></span></span><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="font-family:'Trebuchet MS';font-size:10;"><o:p></o:p></span></p><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="font-family:'Trebuchet MS';font-size:10;"><span style="font-size:85%;">Why? Suppose a schema processor is checking a document that looks like this:</span></span></p><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="font-family:'Trebuchet MS';font-size:10;"><span style="font-size:85%;"><o:p></o:p></span></span></p><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="font-size:85%;"><span style="font-family:Courier New;"><span style="font-family:'Trebuchet MS';font-size:10;"><o:p><br /></p></o:p></span></span></span><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >MyType</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><span style="mso-tab-count: 1"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >Foo</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" > /><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ></</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >MyType</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >></span><span style="font-family:'Trebuchet MS';font-size:10;"><o:p></o:p></span></p><br /><span style="font-size:85%;"><span style="font-family:Courier New;"><span style="font-family:'Trebuchet MS';font-size:10;"><o:p><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"></o:p></span></span></span></p><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="font-family:'Trebuchet MS';font-size:10;"><span style="font-size:85%;">The processor can’t figure out which element declaration in the schema to match the element “Foo”.<span style="mso-spacerun: yes"> </span>It must be able to find the match unambiguously.<span style="mso-spacerun: yes"> </span>You might think that it just doesn’t matter: the document still fits the description.<span style="mso-spacerun: yes"> </span>But it’s critical for a schema processor, because it must be able to align the current XML document node with exactly one schema declaration.<span style="mso-spacerun: yes"> </span>This disconnect between how people feel they should be able to describe a document and how a schema processor really works is what’s really behind the criticism of UPA. </span></span><span style="font-family:'Trebuchet MS';font-size:10;"><span style="font-size:85%;">But most documents and schemas are WAY more complicated than these examples, and the UPA rule keeps the validation mechanisms manageable.<span style="mso-spacerun: yes"> </span>By the way, this change to the schema fixes the UPA problem – can you see why?</span></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="font-family:'Trebuchet MS';font-size:10;"><o:p></o:p></span><span style="font-family:'Trebuchet MS';font-size:10;"><o:p><br /></p></o:p></span><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:element</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" > </span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:red;" >name</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >=</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;" >"<span style="color:blue;">MyType</span>"<span style="color:blue;">><o:p></o:p></span></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><span style="mso-tab-count: 1"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:complexType</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><span style="mso-tab-count: 2"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:sequence</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><span style="mso-tab-count: 3"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:element</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" > </span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:red;" >name</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >=</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;" >"<span style="color:blue;">Foo</span>"<span style="color:blue;"> /><o:p></o:p></span></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><span style="mso-tab-count: 3"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:element</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" > </span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:red;" >name</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >=</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;" >"<span style="color:blue;">Foo</span>"<span style="color:blue;"> </span><span style="color:red;">minOccurs</span><span style="color:blue;">=</span>"<span style="color:blue;">0</span>"<span style="color:blue;">/><o:p></o:p></span></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><span style="mso-tab-count: 2"></span></</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:sequence</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><span style="mso-tab-count: 1"></span></</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:complexType</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ></</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:element</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >></span><span style="font-family:'Trebuchet MS';font-size:10;"><o:p></o:p></span></p><br /><span style="font-family:'Trebuchet MS';font-size:10;"><o:p><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="font-family:'Trebuchet MS';font-size:10;"><o:p></o:p></span></p><span style="font-size:85%;"><span style="font-family:'Trebuchet MS';font-size:10;"><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="font-family:'Trebuchet MS';font-size:10;"><span style="font-family:trebuchet ms;">When the schema validator hits the <u>first</u> “Foo” element, it <i style="mso-bidi-font-style: normal">knows</i> that “Foo” <i style="mso-bidi-font-style: normal">must</i> have been declared in the first element declaration within “MyType”.</span><span style="mso-spacerun: yes"> </span></span><br /></p><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="font-family:'Trebuchet MS';font-size:10;"><o:p></o:p></span><br /></p><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="font-family:'Trebuchet MS';font-size:10;">The UPA problem can come up just as result of schema factoring.<span style="mso-spacerun: yes"> </span>I came across this situation when looking at a schema for an XML document containing a SQL expression tree:<o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:element</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" > </span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:red;" >name</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >=</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;" >"<span style="color:blue;">Subquery</span>"<span style="color:blue;">><o:p></o:p></span></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:complexType</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:choice</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:sequence</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:group</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" > </span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:red;" >ref</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >=</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;" >"<span style="color:blue;">tns:grpColumn</span>"<span style="color:blue;"> </span><span style="color:red;">minOccurs</span><span style="color:blue;">=</span>"<span style="color:blue;">1</span>"<span style="color:blue;"> </span><span style="color:red;">maxOccurs</span><span style="color:blue;">=</span>"<span style="color:blue;">1</span>"<span style="color:blue;"> /><o:p></o:p></span></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:group</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" > </span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:red;" >ref</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >=</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;" >"<span style="color:blue;">tns:grpQuery</span>"<span style="color:blue;">/><o:p></o:p></span></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span></</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:sequence</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:sequence</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:group</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" > </span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:red;" >ref</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >=</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;" >"<span style="color:blue;">tns:grpColumn</span>"<span style="color:blue;"> </span><span style="color:red;">minOccurs</span><span style="color:blue;">=</span>"<span style="color:blue;">1</span>"<span style="color:blue;"> </span><span style="color:red;">maxOccurs</span><span style="color:blue;">=</span>"<span style="color:blue;">1</span>"<span style="color:blue;"> /><o:p></o:p></span></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:element</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" > </span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:red;" >name</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >=</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;" >"<span style="color:blue;">ListItem</span>"<span style="color:blue;"> </span><span style="color:red;">minOccurs</span><span style="color:blue;">=</span>"<span style="color:blue;">1</span>"<span style="color:blue;"> </span><span style="color:red;">maxOccurs</span><span style="color:blue;">=</span>"<span style="color:blue;">unbounded</span>"<span style="color:blue;">><o:p></o:p></span></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:complexType</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span><span style="mso-spacerun: yes"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:attributeGroup</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" > </span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:red;" >ref</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >=</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;" >"<span style="color:blue;">attGrpScalarValue</span>"<span style="color:blue;"> /><span style="mso-spacerun: yes"> </span><o:p></o:p></span></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span></</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:complexType</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span></</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:element</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span></</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:sequence</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span></</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:choice</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:attributeGroup</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" > </span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:red;" >ref</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >=</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;" >"<span style="color:blue;">tns:attGrpSubqueryItems</span>"<span style="color:blue;"> /><o:p></o:p></span></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span></</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:complexType</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ></</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:element</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >><o:p></o:p></span></p><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"></span></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="font-family:'Trebuchet MS';font-size:10;"><o:p></o:p></span></p><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="font-family:'Trebuchet MS';font-size:10;"><span style="font-size:85%;">This schema type gives a choice of two sequences, both starting with an element group called “grpColumn”.<span style="mso-spacerun: yes"> </span>So, when the schema validation processor encounters the group elements in an XML document, it can’t tell which choice is in play.<span style="mso-spacerun: yes"> </span>Also, the schema spec designers did not want schema processors to look “past” the current node to try and resolve which schema node matches a document node.<span style="mso-spacerun: yes"> </span>By the way, this is the schema construct that .NET Framework 2.0 compiles with no errors, but SQL 2005 (via MSXML 6.0) throws a UPA violation error.</span></span></p><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><br /><span style="font-family:'Trebuchet MS';font-size:10;"><span style="font-size:85%;">I refactored the schema a little to eliminate the UPA violation:</span></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="font-family:'Trebuchet MS';font-size:9;"><o:p><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:element</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" > </span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:red;" >name</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >=</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;" >"<span style="color:blue;">Subquery</span>"<span style="color:blue;">><o:p></o:p></span></span></p></o:p></span><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:complexType</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:sequence</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:group</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" > </span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:red;" >ref</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >=</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;" >"<span style="color:blue;">tns:grpColumn</span>"<span style="color:blue;"> </span><span style="color:red;">minOccurs</span><span style="color:blue;">=</span>"<span style="color:blue;">1</span>"<span style="color:blue;"> </span><span style="color:red;">maxOccurs</span><span style="color:blue;">=</span>"<span style="color:blue;">1</span>"<span style="color:blue;"> /><o:p></o:p></span></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:choice</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span><span style="mso-spacerun: yes"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:group</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" > </span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:red;" >ref</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >=</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;" >"<span style="color:blue;">tns:grpQuery</span>"<span style="color:blue;">/><o:p></o:p></span></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:element</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" > </span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:red;" >name</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >=</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;" >"<span style="color:blue;">ListItem</span>"<span style="color:blue;"> </span><span style="color:red;">minOccurs</span><span style="color:blue;">=</span>"<span style="color:blue;">1</span>"<span style="color:blue;"> </span><span style="color:red;">maxOccurs</span><span style="color:blue;">=</span>"<span style="color:blue;">unbounded</span>"<span style="color:blue;">><o:p></o:p></span></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:complexType</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:attributeGroup</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" > </span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:red;" >ref</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >=</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;" >"<span style="color:blue;">attGrpScalarValue</span>"<span style="color:blue;"> /><o:p></o:p></span></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span></</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:complexType</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span></</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:element</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span></</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:choice</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span></</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:sequence</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:attributeGroup</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" > </span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:red;" >ref</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >=</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;" >"<span style="color:blue;">tns:attGrpSubqueryItems</span>"<span style="color:blue;"> /><o:p></o:p></span></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ><span style="mso-spacerun: yes"></span></</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:complexType</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" ></</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:maroon;" >xs:element</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:9;color:blue;" >></span><span style="font-family:'Trebuchet MS';font-size:9;"><o:p></o:p></span></p><span style="font-family:'Trebuchet MS';font-size:9;"><o:p><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"></o:p></span><span style="font-family:'Trebuchet MS';font-size:10;"><span style="font-size:85%;"><br /></p></span></span><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="font-family:'Trebuchet MS';font-size:10;">So again, you could argue that both schema examples are identical when it comes to describing an XML document.<span style="mso-spacerun: yes"> </span>Why should UPA <i style="mso-bidi-font-style: normal">make me</i> use a specific methodology?<span style="mso-spacerun: yes"> </span>It’s because the people who created the XML Schema specification were also envisioning how schema validation tools would be created.<span style="mso-spacerun: yes"> </span><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; TEXT-ALIGN: center" align="center"><span style="font-family:'Trebuchet MS';font-size:10;"><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="font-family:'Trebuchet MS';font-size:10;">In the case above, UPA forced me – IMO – to create a better schema.<span style="mso-spacerun: yes"> </span>So aside from just general unfamiliarity with UPA and inconsistent toolkit support, does UPA really do any harm?<span style="mso-spacerun: yes"> </span>Schema authors sometimes want to allow users to add extra elements to their XML documents.<span style="mso-spacerun: yes"> </span>Since you don’t know what the XML will look like ahead of time, it’s nice to use a wildcard like </span><span style="font-family:'Courier New';font-size:10;">xs:any</span><span style="font-family:'Trebuchet MS';font-size:10;"> in the schema.<span style="mso-spacerun: yes"> </span>The problem is that UPA restricts the occurrence of a wildcard element depending on the occurrence of the element previously declared.<o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="font-family:'Trebuchet MS';font-size:10;"><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="font-family:'Trebuchet MS';font-size:10;">For example, this is a legal sequence:<o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="font-family:'Trebuchet MS';font-size:10;"><o:p></o:p></span></p><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="font-family:'Trebuchet MS';font-size:10;"><o:p><br /></p></o:p></span><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:sequence</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><span style="mso-tab-count: 1"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:element</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" > </span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:red;" >name</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >=</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;" >"<span style="color:blue;">Foo</span>"<span style="color:blue;">/><o:p></o:p></span></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt; mso-layout-grid-align: none"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ><span style="mso-tab-count: 1"></span><</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:any</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" > /><o:p></o:p></span></p><br /><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" ></</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:maroon;" >xs:sequence</span><span style="mso-no-proof: yes;font-family:'Courier New';font-size:10;color:blue;" >></span><span style="font-family:'Trebuchet MS';font-size:10;"><o:p></o:p></span></p><br /><span style="font-family:'Trebuchet MS';font-size:10;"><o:p><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"></o:p></span><span style="font-family:'Trebuchet MS';font-size:10;"><span style="font-size:85%;"></span></span></p><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="font-family:'Trebuchet MS';font-size:10;"><span style="font-size:85%;">However, you can’t change the occurrence of “Foo” to any value other than “1” (the default) or you violate UPA.<span style="mso-spacerun: yes"> </span>UPA does not allow any wildcard to be next to an optional element.<span style="mso-spacerun: yes"> </span>Also, you can’t put a wildcard before any element in the same namespace.<span style="mso-spacerun: yes"> </span>Some feel this constrains extensibility in some cases.<span style="mso-spacerun: yes"> </span></span></span></p><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="font-family:'Trebuchet MS';font-size:10;"><span style="font-size:85%;"></span></span></p><p class="MsoNormal" style="MARGIN: 0in 0in 0pt"><span style="font-family:'Trebuchet MS';font-size:10;"><span style="font-size:85%;">But honestly, I haven’t found a situation where UPA caused pain that a workaround doesn’t fix.<span style="mso-spacerun: yes"> </span>I also think you can extend schemas better by wrapping them rather than extending them.<span style="mso-spacerun: yes"> </span>In other words, create a schema that imports the schema you want to extend, add some new elements and tie them to the original schema through key/keyref declarations.<span style="mso-spacerun: yes"> </span>It’s really the only way for schema validation to work with content unknown at design time.</span></span><br /><br /></p>Unknownnoreply@blogger.com3tag:blogger.com,1999:blog-14219297.post-1128309068979722512005-10-02T20:11:00.000-07:002005-10-02T20:26:53.530-07:00Islands of Metadata<span style="font-family:trebuchet ms;">Does anyone remember the term “islands of automation”? I’m not sure where it was coined, but it described - in the late '70s - the situation where an enterprise implemented multiple systems with little or no integration. Pretty much <em>every </em>enterprise application has a customer table. Having a CRM system, a logistics system, and, say, a financials system with individual ideas of a customer creates obvious problems.<br /><br />The “islands of automation” problem was solved in two unsatisfying ways. One way was to install a single big applications suite. The customization costs were big and enterprises were forced to adapt their business to whatever functionality was baked-in the suite. The second way was to buy a set of so-called “best of breed” point solutions. But integrating mission-critical apps can be more expensive than customizing a single app. Neither approach avoided vendor lock-in – whether from an apps vendor or a systems integrator. But I guess that’s sort of the idea, isn’t it.<br /><br />The metadata-based toolkits under development at Microsoft seem in a similar conundrum. On one side are the individual mechanisms like Indigo, Windows Workflow, DSL, and distributed solutions. Individually, they are interesting toolkits – but they have no awareness of each other. On the other side is Microsoft Business Framework, which uses a single, rather big pile of metadata to describe an application from the database up to the UI. Here we go again: Do I want disconnected best-of-breed metadata, or closed, monolithic metadata.<br /><br /><br /><strong>A Conceptual Leap at PDC 2005</strong><br /><br />I saw one very promising demonstration at the Microsoft PDC 2005 conference – one that broke a conceptual barrier. Don Box and Dharma Shukla showed Windows Workflow Foundation (WWF) acting as an agent for some Indigo (a post-Whidbey version) services. Indigo-tagged components were dragged into workflow design surface that in turn represented a specific service action. Voila! The service description was <em>inferred </em>from the message “needs” of the application components.<br /><br />This is significant because loose coupling between application components and the agent layer of the (</span><a href="http://appside.blogspot.com/2005/09/canonical-soa-stack.html"><span style="font-family:trebuchet ms;">canonical</span></a><span style="font-family:trebuchet ms;">) SOA stack means that <em><u>the agent describes the services</u></em>. Only the agent knows the complete content of the message. Only the agent knows how to distribute message parts to components.<br /><br />The other thing I liked was that – for once – the metadata behind WWF wasn’t just source code behind a code generator. Microsoft likes to write tools that spit out reams of partial classes like shot from a Holland & Holland (and about as constructively). Maybe Microsoft feels that OO coupling and static types is how us minions want to develop systems. A more cynical explanation is that nothing sells an idea – at least internally – like good support for IntelliSense. But I digress.<br /><br />The XOML files used in the WWF/Indigo future-ware demo were processed dynamically (or at least portrayed to be - it was a demo). Generated code may be all the rage, but I only like generated code in one usage pattern: transient assemblies. Here was a great example of an agent that was adept at managing work instances while all the while knowing that the processing plan can change at any moment.<br /><br />I used to think I was old enough to just let Kool-Aid roll down my back. But seeing the WWF and Indigo teams pick up on this concept definitely brought gave me something to smile about.<br /></span><span style="font-family:Trebuchet MS;"></span>Unknownnoreply@blogger.com0