Notes
Stefan Tilkov has been running an excellent series of “unedited notes” from the talks he has been attending at QCon. Within the notes are a number of excellent talking points about REST, SOA, etc. Reading through them, I couldn’t help but feel inspired to write up a few of my own thoughts on the various topics.
The first set of notes tracked former IBM colleague Sanjiva Weerawarana’s talk on WS-* vs. REST.
- Sanjiva claiming he’s unbiased, and if he’s biased, it’s because I told him to be :-)
- to be loved and hated for creating WSDL 1.1
- started Apache SOAP, WSIF
- co-authored lots of other stuff in the WS-* space
Sanjiva and his team were instrumental in the early days of WS-*. I still remember the email exchanges I had with Sanjiva back before WSDL 1.0 was developed (before I hired on with IBM). Microsoft and IBM had each come up with their own ways of describing Web service endpoints; I had also developed my own alternative description format that managed to get a surprising amount of attention. When WSDL came out, I decided to abandon my format and go with WSDL 1.0 largely because it was “Good Enough”. That event was one of a series of things that led me to joining IBM.
- world is not all about HTTP, XML, XML Schema
- not all interactions are request/response
- composability of features is key (security, reliability, …, needed, but not al the time)
- SOAP 1.1 section 5 created because XSD was not yet a standard
There should be absolutely no doubt that back in 2000, SOAP and WSDL were critically needed. At the time, many pieces of the right solution were present (e.g. I was writing AJAX apps back in 1998-1999) but no one had put them together in just the right way. The movement towards WS-* began an evolution in the way the industry approached services on the Internet. The lessons we’ve learned over the past eight years have been transformative. When I consider SOA and REST today, I do not see two competing visions, I see an evolution that has brought the industry back to a core set of fundamental design principles that we seemed to have lost sight of for a while.
- WS-* is just overhead unless you have something in your SOAP headers (shows SOAP message without headers)
- Lie: always do SOAP
- if HTTP(S) + XML is enough for the problem, more power to you
Those who are familiar with my history with IBM should know that I was once a *major* proponent of the WS-* approach. I was one of the original members of the IBM Emerging Technologies Toolkit team, I wrote so many articles on the subject during my first year with IBM that I was able to pay a down payment on my house without touching a dime of savings or regular paycheck, and I was involved in most of the internal efforts to design and prototype nearly all of the WS-* specifications. However, over the last two years I haven’t written a single line of code that has anything to do with WS-*. The reason for this change is simple: when I was working on WS-*, I never once worked on an application that solved a real business need. Everything I wrote back then were demos. Now that I’m working for IBM’s WebAhead group, building and supporting applications that are being used by tens of thousands of my fellow IBMers, I haven’t come across a single use case where WS-* would be a suitable fit. In contrast, during that same period of time, I’ve implemented no fewer than 10 Atom Publishing Protocol implementations, have helped a number of IBM products implement Atom and Atompub support, published thousands of Atom feeds within the firewall, etc. In every application we’re working on, there is an obvious need to apply the fundamental principles of the REST architectural style. The applications I build today are fundamentally based on HTTP, XML, Atom, JSON and XHTML.
- REST-* is on its way - ARGH!
- HTTP-R, anyone?
- no commonly accepted, a.k.a. interoperable REST model for message signing, non-repudiation, reliable messaging
I’ve always found these kinds of arguments humorous. No rational person would ever claim that REST is all we need. The more folks try to do really interesting and sophisticated things, the more extension and standardization is going to be necessary. That’s a fundamental truth. The trick is in designing those extensions and standards so that the core constraints and principles of the architectural style are not compromised and so the overall architecture remains consistent throughout. It is also critical that tool vendors not try to push solutions to problems we don’t have (ahem, UDDI) or don’t have enough collective experience with to standardize.
- Lie: WS-* is complex
- reference to Tim Bray’s slide
- reference to our poster
- is WS-* really complex?
- for the middleware implementor - yes quite
- for the app developer: no
Wrong. All of this is complex. Technology is complex. It does nobody any good to argue that one particular way of building applications is less complicated than any other way of building applications. WS-* is complex. REST is complex. It’s all complex. Complexity is not the issue. The real issues are things like utility, consistency, accessibility, etc.
- Sanjiva: most of the problems come from databinding
This is true of ANY application architecture.
- REST is simple
- not true: increasing abstraction, comparison to UML/MOF (MDA was supposed to rule, and now is dead)
Again, this is entirely not the point. None of this is simple.
- are average developers & architects able to design RESTful systems correctly
Are average developers and architects able to design ANY system correctly? I think if you look at the history of software development as a whole, you’d really have to stop and wonder about the answer to this question. The fundamental challenge comes down to this: developers get paid for coming up with solutions that work; doing so means learning just enough about a technology so address the immediate need so they can move on to the next line item in their list in order to meet the deadline; this will quite often mean that average developers and architects aren’t even going to bother designing and implementing solutions “correctly”; nor should we ever actually think that they will. The best we can do as tool providers is educate users better and provide excellent tooling that makes it easier to do the right thing. Most of the time the tool developers can’t even get it right tho.
- Lie: REST doesn’t need a description language
- Truth: you need to have tooling, thus you need a description language
- Stu: the real debate is about whether we should have a generic description language or specific MIME types
REST doesn’t need a description language. Certains types of RESTful applications will need a description language; Atompub for example. OpenSearch is another example. Here the issue is not whether a description language is needed, it’s whether we need a single generic description language capable of handling are broad a range of use cases as possible or whether simple, targeted solutions that focus on discreet problems are best; i.e. URI Templates.
- Myth: HTTP, the one true protocol
- Enterprisey: JMS, SMTP, TCP, IIOP, MQSeries
- Cool: Jabber/XMPP, YahooIM, SIP
This is another one of those arguments I find extremely silly. The fact is that there will always be multiple technologies. We use the technologies that best suit the problem we’re trying to solve. It just so happens that HTTP is well suited to address a hell of a lot of problems.
- HTTP’s uniform interface is the greated - untul we need just a tad more
- WebDAV and DeltaV: a whole bunch more
- PATCH: just one more to get it right
- Stu: There’s nothing wrong with having an extensible uniform interface
Anyone who believes that four verbs is enough is living in a fantasy world. The issue is not the number of verbs; but the consistent application of a single set of verbs to multiple kinds of resources.
- REST is HOT, WS-* is NOT
- REST is at the top of the hype curve, WS-*
Who cares? We need to be focusing on building solutions that solve real problems, not arguing about popularity. Anyone who picks a technology just because it’s popular in 2007 is an idiot.
The next post in Stefan’s series covers Steve Vinoski’s talk,
- As SOA has no constraints, it doesn’t help you there
- You want to make tradeoffs and impose constraints that induce properties that you want
I don’t agree that SOA has no constraints. I just don’t think the constraints have been well documented or are all that well understood. This stems directly from the fact that SOA has been developed to be as generalizable as possible so that SOA can be seen as solving as many problems as possible. Problem with that is that it is subject to a natural contradiction: The more effort that is put into generalizing a solution, the less applicable it is to any given problem. HTTP, on the other hand, demonstrates exactly the opposite behavior: HTTP was designed primarily to address a single type of solution (publishing hypertext) but has been found to be applicable to an extremely broad range of problems.
- Sanjiva & Glen: HTTP can be mis-used, governance needed there as well
Governance will always be necessary. That’s not the point.
- SOA guy: a uniform interface can’t possibly work? my services all work differently
These kinds of arguments are funny. Consider: pretty much every piece of software in existence in the world uses the fundamental concept of a byte — a series of eight zero’s and one’s. Nearly every single computing operation that occurs in every computer program ever written deals with manipulating different combinations of those zero’s and one’s around to accomplish different tasks. With that in mind re-read that sentence from the SOA guy: “a uniform interface can’t possible work”. What people need to understand is that it’s not a question about what your application needs to do; it’s a question about how to make it do it. SOA and REST can be applied to the exact same set of problems; the issue is which is best suited for any given problem.
- Interfaces and Scalability: specialized interfaces inhibit scalability - the likelihood of serendipitous reuse goes down with specialized interfaces
- every interface is a new protocol
- Sanjiva: Aren’t specialized resources a specialized protocol, too? Steve: Not every client is generic and can talk to everything, but some can.
- Q. Are you saying “an interface is a protocol” or “an interface has a protocol”? Steve: A bit of both - it tells you how to use the thing at the other end, it’s just a different layer that the protocol that tells you how to move the bits
Sanjiva is right. Specialized resources are a form of specialized protocol. Every time an application uses an application specific set of methods or an application specific data format, serendipitous reuse goes down. Don’t believe me? Look at the war between ODF and OOXML. Look at the difference between encoding data as XML versus encoding in a proprietary data format. Look at the hell that is character set encoding. REST is no less resistent to tight coupling as any other architectural style.
Consider GData as an example. I really appreciate the work that Google has done with regards to Atom and the advances they’ve made, but GData relies heavily on vendor specific extensions. When we held our last face-to-face interop session, everyone had a very hard time interoperating with the GData services *because* of their vendor specific extensions. It happens. The trick is in making it happen less frequently. Standards that are developed honestly, openly and patiently are key.
REST does not solve The Interoperability Problem; it merely places a number of constraints that reduce the likelihood of a specific set of common interop problems.
- The fact that del.icio.us and Flickr use a GET to delete something is just an abuse of the system
No, it’s reality. Developers do what they feel is necessary based on what they understand and the constraints they’re operating under.
The next set of notes came from Pete Lacey’s talk,
- Agrees with everything Steve said, disagrees with almost everything Sanjiva said
- Degrees of RESTfulness - adding and removing constraints is perfectly fine
- for every constraint, there is a trade-off
- table showing constrains, properties, trade-offs
- topic: accessibility, a new “ility”, the sum of: simplicity, scalability/2 and modifiability/2
- scalability: both at runtime (1 million simultaneous clients) and ability to connect (500,000 to begin with)
- the ability to manipulate information as needed
- also a topic: modifiability (extensibility, evolvability, configurability, reusability, customizability)
The *ilities of REST based applications are the fundamental reason is it so broadly applicable. Interestingly, we achieve these not by designing a solution to be as general as possible, but by working to solve one set of very distinct problems very well.
- media types should be selected so that the maximum number of clients can participate
This is where I stop and say that I really really like Pete. He has the admirable quality of saying really really smart things.
- Question/Comment: you can use cookies and still be RESTful, if you don’t use sessions
- Answer: the hard thing is cookies can have any meaning, because of this intermediaries might decide to not cache
- Suggestion to avoid cookies, rely on HTTP authentication
If only reality were that simple. :-)
- shows demo curl command line. (Weak, he had it in a script. I always actually type them in :-))
Many many years ago I saw a t-shirt that said “Real programmers use vi”. We need to modernize that: Real programmers use curl.
The final set of notes comes from fellow Abdera committer Dan Diephouse’s talk on the Atom Publishing Protocol,
- Glen Daniels asks why collections don’t simply contain collections
Simple answer: hierarchy is hard. The WG collectively did not have enough experience to do it right so it didn’t do it at all. Once we all get more experience, we can revisit it. Solutions to this have already been deployed.
- Sanjiva: doesn’t the service document introduce coupling to the content types accepted? Dan: you can keep it general, or accept more. Again:
Yes it does. That’s the point. The Service Document exists because it is needed.
- Question: there is no way to specify binary and metadata in a single request. Answer: no.
- Multipart is perfectly feasible (as a private extension to the protocol)
- why use Atom?
Yes, there is a way of specifying binary and metadata in a single request, it’s just not part of the standard. This is another thing that the WG did not have enough collective experience to standardize. Solutions to this have already been deployed.
- AtomPub guarantees that you’ll follow RESTful best practices
- avoid writing your own protocol
Not quite. It is still possible to mess it up. For instance, Atompub + Required Vendor Specific Extensions == New Protocol
- many frameworks are popping up
Another shameless plug.
- some quick notes about GData: “simple standard protocol for reading and writing data”
- not that simple from Dan’s point of view
I repeat, “Atompub + Required Vendor Specific Extensions == New Protocol”
- APP doesn’t do batch - GData does, but hasn’t received warm reception
- feed with “operation”-specific operations in the feed
- while there’ve been many complaints, there hasn’t been an alternative suggestion
- some discussion about the right batch model - suggestions from Stu: either pick a super-resouce and update that, or use a single connection and pipe through it
- Mike Herrick points to James Snell’s suggestion to use multi-part
- Stu: reasons against pipelining include broken intermediaries
- Suggestion from Patrick Logan: nothing wrong with submitting many things at once and retrieve results in a feed
For the record, this is a really really really really hard problem to solve. However, it’s not hard because of any single technological issue; it’s hard because no one can seem to agree on the right way to do it.
- Thoughts on GData: does it expose a weakness (AtomPub is not enough) or a strength (can be extended)? Dan thinks the latter
Both. Note this well: Atompub was not designed to be the end all, be all of protocols. There are some things it cannot do that well and lots it cannot do at all. Much of what you see in GData is based on perceived gaps in what Atompub can do and what the developers felt was necessary. Like I’ve said before, developers will do what they think is necessary.
- Hierarchies not easily modelled
- Standard suggestion: put a link to the collection into an entry (in a collection element) - possibly adding rel attributes if there’s more than one collection
- Client needs to know the collection element is there, not a required part of Atom
For the record, this suggestion has proven to be extremely useful.
Anyway, that’s all. Time for me to head to the gym.