Gordon Sim is a senior software architect and developer at Apache Software Foundation and employed at Red Hat. Gordon lives in Scotland. He is one of the main contributors to the Apache Qpid project. Gordon replied on the Qpid users mail list (Aug 29th 2012) to my post The Journey to AMQP 1.0 with Apache Qpid and he was kind enough to give me permission to post it here. Thank you Gordon.
From Gordon Sim:
post, I particularly like the title! It has indeed been a journey and
the Qpid community - all the users who have raised JIRAs, highlighted
limitations of the software and earlier protocols, all the developers
who have contributed features, patches and ideas - has
been instrumental in getting AMQP to this very exciting
Change is unsettling, but it can also bring great benefits. I'm grateful that you persevered and glad you can now appreciate the greater power, flexibility and reach of the AMQP 1.0 model.
I was more ambivalent about exchanges than you. I felt that because they were the most visible and distinctive feature of early AMQP, they got too much attention at the expense of other important aspects of messaging.
In your post you write:
"The real power of 0.8 was NOT in the exchanges but instead it was in the bindings - the routing [...] the power of AMQP is that it is a really powerful router of messages"
I would argue that routing is one of the benefits of messaging intermediaries in general (along with store-and-forward capabilities). The power of AMQP is that it enables interoperability between different intermediaries and/or applications. This lets you construct your messaging network from diverse components which gives the system designer a richer more flexible set of tools.
I don't think we are disagreeing there, just providing slightly different emphasis.
"The broker of 0.8 and it's exchanges, queues and bindings provided a brilliant and dynamic routing capability mainly through decoupling the sender from the endpoint."
The routing capabilities of early AMQP were actually fairly rudimentary and inflexible in my opinion. They also fell a little short in the decoupling since the destination you sent to dictated the only possible routing algorithm you could use for those messages. The shortcomings could be addressed by extensions, but I think 1.0 provides a far better foundation, standardising the simple patterns while allowing freedom for richer patterns to emerge.
I do agree that the addressing syntax as supported by the AMQP 0-10 implementation of the qpid::messaging API is at times a little complex and awkward. I think that was largely due to the fact that it was the control point for all bridging between the conceptual model of AMQP 1.0 and the 0-10 protocol which is actually used underneath.
Again, this was a stage in the journey we have travelled. We wanted simple things to be simple, more complex things to be possible and transition to 1.0 to be as painless as possible.
I am confident the mapping onto 1.0 will be much more direct, transparent and intuitive. I'm working on a 1.0 implementation of the API right now and will post more detail on that for comment as I make progress.
"Proton is a high performance, lightweight messaging library. It can be used in the widest range of messaging applications including brokers, client libraries, routers, bridges, proxies, and more. Proton is based on the AMQP 1.0 messaging standard."
It's been way too long since I blogged. So long, in fact, that I couldn't log back in. With some help from Moveable Type Forums I managed to get logged back in today.
Last week I was at Red Hat Summit 2011 in Boston. What a buzz. Red Hat's new PaaS offering, OpenShift, was announced and demonstrated. Really sweet. I contributed a very small amount of code to the OpenShift Express offering. It was an AMQP plug-in for the management framework MCollective using MRG Messaging with the Ruby/C++ client. The OpenShift team are a pleasure to work with. I look forward to contributing some more.
I had the privilege of speaking at Summit with Lans Carstensen from DreamWorks Animation. We did a session called Red Hat Enterprise MRG at DreamWorks: Grid at Massive Scale, Utilization, and Throughput. You can download the presentation from the Summit Presentation here. All presentations are on the Red Hat Summit 2011 Presentation Page. That version of the presentation contains all the meat but lacks all the fun stuff, including Kung-Fu Panda 2 characters on some of the slides. We had a lot of fun, as can be gauged from the first of many questions we had, "how many Red Bulls did you guys have?" He had only been drinking coffee but we were really pumped about the presentation.
The low-latency feature uses AMQP (MRG's Messaging) to deliver job workloads to a grid's execution nodes. This bypasses the job scheduler (Condor's schedd). Instead a special daemon on the execute node consume jobs off an AMQP queue. It's a pull versus push model.
Why is this useful? Well originally the intent was for sub-second jobs in the financial services industry. Consider a grid of index calculation applications. Each calculation may take less than a second to perform. The applications can't wait for a job scheduler to decide that their execution nodes are available, then schedule a job onto the node, then push the job out there. Instead jobs are placed on AMQP queues. As soon as an execute node is free to perform work it pulls the next job form the queue. There is barely any latency between jobs.
Of course as soon as you do this for one specific use, sub-second jobs, then others see the advantages too. This feature doesn't need to be sub-second jobs. MRG customers from various industries now see the advantage of this feature.
There are some considerations. For example, should the entire grid be untilized this way or should a specific portion of the grid be carved off for low-latency workloads? If the jobs are sub-second and high volume, should they be reported to a management console (this could cause quite a bit of clutter) or just logged? How should failed jobs be managed under different scenarios? e.g. in sub-second transactions a failed job may have missed its window of being useful and therefore there is little point in resubmitting. The answers to these quesitons will depend on the type of low-latency workload.
If you are interested in more information on this topic please remember to check out Matt's post.