First, let us review what is the concept behind services, with particular mention of the web services themselves.

A service is a business functionality exposed by a simple interface. Note that it is not an object, nor a method, nor a module, not a procedure. It is pure business functionality.

Services are consumed, not called, nor executed nor invoked. We define the service consumption as the interaction between agents, a client and a provider, which will communicate using a messaging system, where the data unit is a document. Consumption may be local or remote (meaning a service may not require remote communication).

A service may live in the web, where they are modeled as resources, identified by a URI. If not living in web, as suitable endpoint implementation should be provided.

A service is described at business level with a contract, where a glossary of the terms is agreed upon, and the semantics of all business concepts and processes are clearly stated and known by interacting parts (this is important). The service implementation is defined by the service name, its contract, an endpoint (unique entry point/port) where to send messages, the operations the service offers, the message interaction flow, the message composition (including the document definition) and optionally a binding information.

Nice definition. What is wrong with it?. Not much, actually. Just that when Web Services Architecture group decided to make those concepts into a standard, they did a great job, but also committed a couple of fatal mistakes.

The first one was the actual selection of SOAP as the messaging protocol. Yes, SOAP is a big mistake, not because it is SOAP, but because of what it was created for. You can read here a very complete explanation of the SOAP origins by one of the people that worked on it, Don Box. There, we can understand the main problem to solve was creating a protocol to better support RPC (actually RMI) in a object distributed system.

Yes, SOAP (Simple Object Access Protocol) was not created to support services documents and messaging. It was created to provide a technology neutral data/class definition system, plus supporting remote method access (that actually meant Remote Method Invocation).

So, why is this a mistake? Well, unless your idea of services was, from the beginning, that of distributed objects and the exposition of their methods, SOAP was no fit for a Services architecture. Why is SOAP there, not sure, but my common sense tells me it was not its place. If the idea was to support document oriented messaging, an RPC protocol was the weirdest decision I can think of. You will need to tweak the protocol to support what it was not supporting before. The only reusable aspect is that it was meant for HTTP transportation, which by the way is not a constrain of the WSA (it is the only implementation though).

History tells that soon after, SOAP was modified to allow document style definition, relaxing the literal RPC format. The SOAP acronym meaning was actually changed! So it is not an object access oriented protocol anymore, but a service oriented one. That should prevent people using it for RPC, was the hope.

Ok, if that was fixed (SOAP adjusted), we may have no more problems with that, right?
What is the second mistake?

Well, a subtle one: they kept the RPC idea in SOAP. Backwards compatibility? Not sure, but the RPC literal was there, in SOAP. Worse, the WSDL included that option and WSA did not discouraged the practice! I may think it was because there were tools supporting this and the idea was not banning anyone from using them. So, people started (or continued) using RPC and calling them services!
Is that wrong? Well, a little:

a. The problem is RPC is not Services,
b. RPC does not share the service metaphor,
c, RPC has different implementation side effects,
d. RPC usually forces fine grain while Services try the contrary
e. RPC couples (per method name, parameters and interaction)
f. RPC is in the IT domain, while services should be in the business one (this is a big one).
g. RPC usually works in the distributed environment, but services not are always distributed.
h. RPC forces request-response flow, while services may have any combination of flows.
i. RPC is usually blocking, synchronous. Services are asynchronous per messaging basic concepts.
j. RPC parameters usually require serialization of objects (to be passed as parameters). Services do not even know what objects are nor the parameter concept, they deal with plain messages.
k. In HTTP, as a particular implementation, RPCs mapping of concepts adds overhead to the call, while Services uses HTTP just as a transport and should not add more overhead.

Ok but, what about that? In case the RPC was kept due to tools, the existing tools at that time may be kept as RPC tools, and new tools to work with services and their concepts should had been created. Easy solution.

But that didn’t happen.

Actually, the tools took over and started to fight back! There was almost no support for document style. There was an RPC style non encoded. There was no standard way to represent a document (that you can feed the service call with). Yes, the tools claim they supported document style. But when I did some research projects to compare the tools, I found the services created using document styles didn’t work at all! I created a service, then I created a client using the same tool, and the client failed talking to the service! I made it work by manually structuring the XML document to have the first element named as the method that implemented the service! It was RPC all along, and since nobody complained, I assume no one was expecting to work document style. Well, there were a few.

But the worst comes afterward, with the creation of the Wrapped and Bare modifiers. They are no were in the WSDL nor the WSA. The wrapped is a way to use document style, but doing what I did manually in the lines above: formatting the XML to reassemble an RPC format message! You know what, testing the BARE modifier yield what I suspected: didn’t work.

That means the tools and server vendors are still selling RPC, creating RPC “services” and RPC clients. The ones that supported the document style, did a great job making people stay away from it, like Axis, whose example of RPC is a couple of lines long, but the same example using document style is gross and long, not even I could read.

Ok, granted, the web services implementation are just RPC in disguise. So what? We’ve been using RPC all our lives and we are happy with it, right?

Well, SOA is a style that looks forward to have its structure and components made out of business functionality. Under that assumption, we can build lots of additional features, like business composition (business processes), orchestration, governance, etc. But, for that we need services as business functionality, decoupled by an uniform interface. RPC adds coupling to the mix, since we are adding the object or library concepts (for many, those two are the same!) , the method invocation interaction, the data coupling in the arguments lists, and the stiffness of non-variable method definitions (just change one argument and all breaks!).

Point (f) above is important. All other features of SOA are built on top of the concept of a business functionality exposed as a service. Governance, composability, visualization, all those work with services, not with function calls. So, as mentioned in point f, what surfaces is IT, not business, and thus the features may not work as expected.

Add to the mix that, to actually create the services, you must be a programmer. Since they reflect the RMI concept, we assume distributions, method and parameter passing knowledge, etc. Even more, the actual code is converted into exposed services, bottom up approach. This will create services shaped as the actual legacy implementation, which may not match the actual business at hand. The top down approach, which should start with the definition of the service at the business domain level, is discouraged. Tools offer not help there (or very little), and only a few architects follow that path. So, the RPC influence is breaking the possibilities of actually taking advantage of the Services metaphor.

Can I say that the whole failure of SOA as a agent of change, an encourager of business over IT in systems, a provider of business valuable concepts like business processes, governance and product composing, is just the intrusion of RPC in the whole dance? May be, but there are other causes as well. Still, SOA may not raise from he tomb if no paradigm shift is achieved.

Wait a minute. REST is in the title, it is another problem in your list? Or is it just the enemy of SOA?

Well, not really. REST is another architectural Style. It was made for a very different problem and I can assure you it is not the replacement of SOA.

It is in the title because of two reasons:
1. People think the problem of SOA were the Web Services using SOAP,
2. and also people believe REST is a simple way of creating the next generation of services.

So, they are adding another point of failure to the new SOA by trying to construct SOA using what they call “RESTFull Services“.

Why is it a fail point? I’ve heard REST services rock!

Well, first I own REST an apology. You see, actually REST has nothing to do with this. The problem is with the wrongly named REST services, which are no more than an API based on HTTP. Now, there could be REST services, but they are much more complex that the ones most people use. And bigger. So, let’s simply call those “Not-So-REST Services” as Web API Services, (WAS? Nice acronym, a provider for one of those would be a WASP!).

Well, our WAS are, as usual, more oriented to functional calls (yes, RPC). There could be a way the WAS achieves a truly service meaning, when using HTTP as the messaging transport, and the Hypermedia as the documents. Yep, it is a nice match. But that requires people thinking on documents and interactions through messages, and they are not doing that. If you check, it is the same old RPC story again.

So, what to do then? Are we doomed, I mean, is SOA doomed?

Not really. We need to understand what a service is about, and provide a suitable implementation that allows developers to work with it without mapping. I mean, why does a service need to implemented as an object method call? It breaks the metaphor and thus breaks design and implementation, making it flawed.
I’m working on a DOSE (Document Oriented Service Engine) implementation in Java, WAS and WSA oriented (WSDL 2.0, that is), with no RPC. The programming model should be quite different that the proxy and RPC model, plus a set of articles to show how design using document orientation for services should be done.

So, stay tuned.

Agile taste for working code over documentation may be misleading to some. As seen in many posts and discussions, it seems that it means no documentation at all for some people. For another group, it means you just need to start coding, nothing else matters. This post is to differentiate plain coding from development, and to explain that working code is more that just executable sentences.

Let’s start by saying that code is the description of a solution in some sort of language. I defend the position that the description of a solution may not be executable. That description is to be read sometimes by the machine, some others by humans, and some others by both. The description has to have an abstraction level. That is, based on the approach we use to describe and to whom, it may be machine language (assembler for instance) or a high level code like a DSL. And we don’t just write ones and zeros because we will need later to revisit that code, and be able to read it and to make changes. Bear with that in mind for later.

Development is not just coding. It involves finding the why in the business needs and decisions, deciding the how we will solve that, actually describe the solution (to stakeholders and to the executers), and verify that solution meets the needs.

One point to clarify here is the executers concept. A solution is not a chunk of code. A solution may be a complex system, that is software intensive. That means it is not just code, it usually contains processes, people, and many other elements. All together are the solution. And that system is to be built in development, not just the code part.

There are several other tasks that should be performed during the development process. As we deal with humans, you need to have people management. A group has always issues, problems, questions. Also, humans need material to work with. Meetings, information, computers, and many other assets. During the development, several decisions should be made, and not just about code: deliverables, QA scheduling, assignments, feature modifications, priorities, etc.

People needs to communicate. Creation of communications channels, management of those, coordination, moderation, etc. Some voices say all that work should be done by developers themselves. Not so sure about that. Some other say that is not needed, we just need code writers to write code. Not sure that works either.

So, so far, I believe development is far more a complex process that just writing code. And coding is far more complex that just writing an executable set of instructions for a machine (Description of a solution, remember?) Thus, I proclaim that coding is not the same as developing. Let’s continue.

Going back to the coding definition. Some people think that a non executable code is waste. Specifically, design is seen as useless documentation, wasted time. Here then I start to defend what we may call design.

Non executable code is that one that describes de solution, but may not be read be a machine and may not be executed by it. As we mentioned before, the executer may not be a machine, but let’s assume the executable code is executed by a machine not to mess with concepts. Some of that code is the one generated as strategic and tactical description, aka, architectural description and design. If it is not executable, why do we need that kind of code? There are several reasons:
- The ones to design are the developers, why? Because it is a description of the solution, and thus code.
- Executable code is often done quicker when the ideas and major decisions are already made in the design.
- When code is done, it is usually reviewed and almost always changed to improve it. Describing at design level is quicker than writing low level code, and thus fixes and improvements can be made earlier, saving time and effort (changing executable code, large portions of it, may have a great cost and impact).
- Once done, the executable code should be verified. Checking against a high level description helps to verify. If it is different, then it means either the decision at high level or at low level need a review. It is important that both descriptions are kept in synch.
- Why doing the work twice? Because they are different works. Description at high level is used at some time, and the low level description later. But both are used. Missing one means there is something in development that may have more cost later. See next point.
- Development is never finished with complete code delivery. There are always enhancements, additions, changes. To decide why, how and what, it is always important to know what is done, how and why. Reading machine code means almost impossible changes. Reading executable code is possible, but at high cost. Reading a high level description, that was kept in synch, is great.
- Design is not a documentation process, it is a description process that should be used through the development. If design outcome is not used, the it is waste. Design that is done trying to be complete before operational coding, it will also be a waste of time. Design should be made in regular chunks, not to little ones that misses the tactical view, not so big so that you end up making decisions with not enough information.

So, given the ideas above, I want to proclaim that used design is actually working code, and thus, if well done, complies with Agile thinking.

First of all, let’s clarify a couple of things.

1. This post is not intended to list “bad ways” of thinking about REST, but to list the actual way people really think about it. Some of them are perfectly correct per REST constrains.

2. From articles, presentations, discussions and the like, I extracted commons ways of thinking, and tried to categorize them. So, this is a totally subjective classification.

So, let’s start.
People know about REST. Many are trying to get into the REST world for the first time, some other have more flight hours, still not so many had read Roy’s dissertation. Even more, some of the ones that actually read the dissertation, may think about REST in different ways.

Usually, when in a discussion, the same concept may be understood differently by the participants, which yields in non-useful conclusions. This may be actually true about REST. That is why I composed the following list of “ways people understand REST”. The list may help the reader to find out where he stands in understanding REST, and how may other people understand it. Also, it may show how some people is using REST.

First, an cautionary word: the names I use are not intended to make fun of people, but to be easy to remember.

The list is divided into three major groups: The API makers, the Mappers and the Fad Followers. This last one may seem a bad one, but it is actually a normal one, trust me many people fall in it.
Then, each group contains categories. Those categories may be present in more than one group, since the characteristics may fit both groups. Finally, people may fit in more than one category, since the work is complex in the world of REST to reduced to just one characteristic.

Here is the list.

Major Groups 1: API Makers.
People in this group I find them everywhere. Most of the time they have a system, usually not built thinking on REST at first (REST as an afterthought), and they want an API created to access that system. They usually think about REST like an API making technique or recipe, for the web.

Categories:
- URI Jugglers. These are the ones that think REST is all about creating URIs, and nothing more. So their discussions are solely focused on URIs, and their presentations are about URIs definitions. One way of spotting this category is when the application API is documented as a URI construction process using templates

- RPCers. This group think REST is a way to map RPC in disguise using URIs in a web API. The most of them don't know they speak RPC at all. Easily spotted when you see the API defined as a set of calls using a URI construction. Most typical is a URI format that includes the actual method name or action, followed by several parameters. There are many of them coming from SOAP based web services (the RPC kind), that made the change believing is easier to define the RPC call in a simple URI than using the complex SOAP envelope.

- Exposers: This type is repeated below. Those are the guys that think you need to expose things in REST using resources. So REST is an API for exposing things on the web. They usually look for objects or domain entities, but mostly processes or methods, and expose them using a URI formatted call. The slight difference with the RPCers, is that the exposers may not use RPC, but still expose the processes and entities through an API.

- CRUDers: Another repeated group. They think REST is a web API for CRUD. The similarities of the HTTP operations and the CRUD ones, plus the reference of resources as Data elements, support this way of viewing REST.

Second Group: Mappers.
This other group may use the API idea, but they actually think REST is a way of representing things and thus the work to be done is to map everything using that new way. Interesting?

Categories:
- CRUDers. Again, the idea is that CRUD operations can be mapped naturally to HTTP operations, and that makes it RESTful.

- HTTPers. They believe REST is HTTP. Deep enough. This category is hard to spot, since many people talking about REST implementation may talk much about HTTP, but think REST is far more than just the protocol.

- Exposers. Again too. They usually try to map all classes, data entities, elements into resources, and then call their systems RESTful. This category includes people that take into account REST constrains, and also people that just maps the domain to resources.

Third Group: FAD followers.
This is a group that is identified by themselves. That is, they expose what they think explicitly. Usually, they tend to follow a lead or the current business idea.

Categories.
- Standard Haters: Here you have all those that think Standards are evil and that REST is an anarchy where you have the freedom to do whatever you like, so they follow REST doing whatever they want.

- KISS lovers. These are the ones that like things to be simple. And someone told them REST is easy, so they follow doing easy things with URIs. There are lots of URI jugglers in this group. People here may not follow all REST constrains when they get complex enough. The idea is to keep the API or system with not much complexity.

- Servicers. They think Services is good, and someone told them REST is a way to do services without SOAP. So they follow. There are several RESTFull service creation techniques. Many RPCers in this category too.

- BuzzWorders. This is a vast majority. They like buzz words, so they follow REST just because it is cool and all people talk about it.

Is there some one I'm missing? Well, yes, probably the group that knows REST as it actually is and understands it. That may be a one person group (yes Roy).

What do you think? Do you find yourself in any of those groups? Which other category may you add?

What if everybody is wrong? Well, not everybody, but majority.

Reading Kelly Water’s Post about “Agile Project Management: Avoiding The Waterfall” I came to think about this. Almost nobody claims today that they are following waterfall, because of the very bad connotation that word now has, but not actually because they understand what are they doing. Same thing for Agile followers.

From the post, I rescue that any process that has phases reassembles a waterfall. From actual presentations, blogs and discussions, Agile often means just following methodologies like Scrum, Lean or XP. Now I question: are we right about this? Let me see if we are, or not.

Waterfall is a term, it seems nobody knows who coined it, related to a phased methodology. Dr. Winston W. Royce wrote about it (without given that name), describing an ideal form that was not so good. The phases were the requirements gathering, analysis on those requirements, system design, coding, testing and operation. The ideal was each step may iteratively cycle with the next down and the next up, in little steps. The flaw was it was usually the case that testing will not just go back to coding, but it may go up till design, and from there, jump back to requirements gathering again. That flaw (the large back jump) was caused not because the phases, but because the testing phase is where the actual product is finally validated against design and requirements.

The problem is, there is no solution to this. We can dilute or minimize the impact of the testing findings, using several techniques, but we cannot avoid it. At any level, team or individual, the same steps are repeated: I see what it is needed, I analyze what is possible, I think on how to do it, I do it and finally I check if it works. That can describe the process of developer creating a simple three line method! That is, phased development is there, everywhere, and that is not the problem. The problem is when that phased process is done in a way that the impact of facing reality with needs is too big that multiplies the work and the cost of the back jump.

So, the waterfall illusion is when you think that rain is not a waterfall, simply because the raindrops are so tiny you don’t see them fall individually. But they fall anyway. It means, people that think eliminating the analysis and design steps and jump into coding breaks the phased process, I’m sorry, it doesn’t, it may avoid a large body of water fall into one place, but will make millions of little bodies fall everywhere. See the point?

Ok, there is another idea into this, and that is to make the water flow up. You start coding, with no idea of what you need, and then the phases are completed backwards. (Hey! If you want to start really backwards, then start by testing! Oh, yes, we have TDD). The code will make a design “emerge”. The next logical steps if to create requirements that match the design. Well, I guess that does not work. In that case, the requirements are still presented before coding. So the new phases are: requirements, coding, design, testing. Or, requirements, testing, coding, design, testing, coding, design, etc. Anyway, we always have phases.

Pages: 1 · 2 · 3

Yep, my own. Although it may have some shared ideas, this is not a group agreement, nor the savior of what is now known as SOA (which is what is being done so far), nor something made for others to follow. It is just what I believe, what I follow, and what I teach each time I can.

So, here it is.

SOA MANIFESTO

1. Commons.
a. SOA is an acronym composed of the words Service Oriented Architecture, which relates to an architectural style that is based on the rules of the service metaphor.
b. A Metaphor is an application of the figure of speech concept to the IT design discipline. Using metaphors in architectural design actually refers to observing the rules of behavior of a particular entity in the business domain, and mimic them with an architectural element.
c. A Service is conceived as a cohesive (and coherent) business functionality, technologically neutral, offered through a uniform interface. A Service is a plain, homonymous metaphor of a business service.
d. For an architecture, Orientation describes the guidelines, principles and decisions that are based on the rules that govern the metaphor behavior.
e. The Architecture refers to an actual style that defines the architectural constrains, suitability and consequences of using it.

From (a) to (e) :

2. Delta
By definition, and in contra-position from some usual implementation, SOA IS NOT:
a. An Indirection Layer for interoperability.
b. A Decoration Wrap for legacy code accessibility.
c. A Component from a whole that represents one architecture instance
d. A Service Group or container of services
e. A Business Process Container.
f. A Modernization Agent
g. An Object Distribution Technology
h. An Antagonist of REST
i. A ROI salvation
j. A Way of Life, A Philosophy of Doing or a Trend (It is an architecture style!)

By definition, and in contra-position from some usual implementation, Services ARE NOT:
a. Necessarily Web Services as defined in the WSA.
b. Ways to implement RPC nor RMI.
c. Exposed Methods or Objects.
d. Defined by the process they execute
e. Necessarily Stateless (YES, they should not necessarily be Stateless).

3. Principles.
Whenever there is the opportunity when working with SOA, people should
a. Favor Business Value exposed in the business domain OVER Business Value exposed with IT domain.
b. Favor Composability OVER Integration through Interoperability, AND favor Integration through Interoperability INSTEAD of Distribution.
c. Favor Metaphor rules INSTEAD of Tool rules
d. Favor Pure Document Style INSTEAD of RPC/RMI style. (NO WRAP TRICKS ALLOWED)
e. Favor Messaging OVER Other communication options.
f. Favor Business needs OVER specific IT practices (like optimization or flexibility)
g. Favor Entity documents OVER commanding documents
h. Favor Versionable documents OVER Structurally static documents.

So it is written.

William Martinez Pomares