So, you've been following ZapThink long enough to know that beginning a service-oriented architecture (SOA) project by purchasing an enterprise service bus (ESB) is starting at the wrong end of the initiative. Purchasing any technology, especially an ESB, at the beginning of an architecture project is a recipe for failure, you've been telling anyone who'll listen. But for whatever reason, your organization didn't pay attention to you, and now they've dropped a bundle on an ESB. Maybe your boss golfs with the vendor sales rep, or maybe the powers-that-be are listening to the wrong analyst firm, who knows. But in any case, you're stuck with that decision, and you're now expected to implement SOA with it.
Fortunately, while purchasing an ESB too early in a SOA project does substantially increase your risk of failure, all is not lost. After all, you're not alone; this mistake is one of the most prevalent SOA snafus in IT shops around the world today, and not all of those projects end up as failures. Many of today's ESBs are now mature products, and can be an important part of a fully functional SOA implementation. Understanding the risks that buying an ESB too early in a SOA initiative presents, and dealing with those risks proactively, can turn a bad situation around and get your SOA initiative back on the right track.
Understanding the risks
Fundamentally, the problem with buying the ESB first is that you might fall into the trap of doing things the way the ESB would like you to do them, in light of the fact that many ESBs are in many ways traditional middleware under the covers. After all, if middleware solved all your problems, then you wouldn't be considering SOA in the first place -- and adding Service capabilities to your middleware doesn't change this fundamental fact.
In fact, the pitfalls that the ESB-first approach introduce fall into three broad categories:
- Taking an overly integration-centric perspective of the project -- Most ESBs are generally really good at connecting things -- in other words, most ESBs are quite capable integration middleware solutions. The problem is, SOA isn't about connecting things, it's about building loosely-coupled services the business can leverage to meet changing process needs. We want to get away from the "connecting things" approach to distributed computing, and instead move to a "composing Services" paradigm, where integration becomes a byproduct of composition.
- The "middleware for your middleware" problem -- if it were practical (or even possible) to take a single piece of middleware and put it all by itself in the middle of your IT infrastructure, that would be one thing, but for most large (and many midsize) organizations, the vision of relying upon one piece of middleware to solve all integration problems is an unrealistic fantasy. In reality, organizations tend to have several different pieces of middleware, of different vintages and for different purposes. Introducing one or more ESBs into the mix means that now you have to integrate your ESBs with existing middleware as well as with each other, leading to the requirement of middleware for your middleware. Where will it ever end?
- The "good money after bad" fallacy -- The "good money after bad" fallacy is actually much broader than IT. People would rather throw money at an approach that's already cost a bundle than to switch approaches to a less expensive, but more effective alternative. If you've been buying middleware from a vendor for years, and now they tell you that you need an ESB, you're likely to take that advice, even if an alternative is lower cost and lower risk, simply because you've already spent so much with that vendor.
ESB-first SOA best practices
Now that you've steered your bus past the pitfalls, let's see if we can point it in the right direction moving forward. The most important thing to remember is that your architecture should drive the technology, not the other way around. Remember that ESBs, like any mature solution, come with a boatload of features -- many of which may not be appropriate for your situation. It is often figuring out which features not to use rather than the capabilities you should actually use that is the key to being successful with a product like an ESB.
In particular, it is essential to take a process-driven approach to your infrastructure, instead of an integration-centric approach. Remember that building service compositions that implement processes typically compose capabilities across multiple execution environments. Furthermore, those compositions are both dynamic and unpredictable -- the business process specialist in charge of the compositions may change them around long after you've deployed the services. Governance becomes the key to managing that unpredictability, rather than pre-defined integrations.
As a result, you shouldn't rely upon any one execution environment for your service implementations, or any one process management environment either, for that matter. ESBs can offer an effective, managed execution environment for some of your service interfaces, but you rarely if ever want to rely upon any one runtime environment for all of your services. In other words, you should balance the advantages of running your services "on the bus" with the fact that SOA allows you to leverage heterogeneity both on and off the bus.
One essential point here is that SOA leverages interoperability more so than portability. In a virtual machine-based "write once, run anywhere" environment, whether Java or Microsoft Common Language Runtime (CLR)-based, distributed computing relies upon the portability of code. SOA, however, leverages the interoperability of the service interfaces so that you don't need to move the underlying service implementations around. As a result, running all your services on the ESB can actually impede your SOA implementation, rather than support it.
So, if you shouldn't think of your ESB as either integration middleware or as a universal service execution environment, then what role should your ESB play? The answer is a service intermediary. Transformations and content-based routing are the essential capabilities a Service intermediary should deliver, in conjunction with robust security and management. Building the business service abstraction depends upon transformations and content-based routing, and fortunately, most ESBs offer these capabilities. So, only use the traditional messaging middleware capabilities of your ESB if you really need them, and only leverage the service runtime your ESB provides when convenient, but configure your ESB as an intermediary to get full value out of it as part of your SOA infrastructure.
Not only does using an ESB as an intermediary enable you to architect the business service abstraction, it also resolves the "middleware for your middleware" problem, because intermediaries can intermediate between disparate integration technologies just as well as they can intermediate between service providers and consumers. If you feel you need to use your ESB's message queuing technology, for example, just because it's there, however, then you won't get this benefit.
The ZapThink take
Yes, you need security, governance, quality, and management, in addition to the transformation and content-based routing capabilities of service intermediaries, in order to build an effective SOA infrastructure. But remember, ESBs aren't the be-all and end-all of SOA infrastructure -- many ESBs on the market include most of the above capabilities, but rarely if ever offer everything an organization requires. In fact, XML appliances are likely a better approach to security and policy enforcement, a registry/repository combined with a full-lifecycle SOA quality solution might serve as your design time and run time governance tools, while a robust SOA management solution might be a critical part of your infrastructure as well. In fact, many organizations leverage such products in conjunction with existing middleware to build out their SOA infrastructure without having to buy an ESB at all.
The bottom line is to always remember that the business drives the architecture, and the architecture drives the technology. Don't let the technology drive the architecture! SOA is particularly adept at abstracting existing technology, which can include recently purchased products in addition to your legacy environment. But knowing which of your existing capabilities to leverage -- and which to forego -- can make or break your SOA initiative.
