Business Making Progress

Does your organization shape your SOA or is it the other way around?

I had the privilege to speak at the IDC Event in Buenos Aries last week on Socially Oriented Architecture and then visit some companies who have or are about to invest in SOA infrastructure - in one form or another.  While the speaking engagement was fun and there were loads of questions, it was the visits to the companies that were enlightening because the questions in Argentina are very similar to the ones that I hear in the US.

As I was visiting the various companies, I noticed how many questions were NOT about the details of the technology - in this case the ESB - but about how the deployment maps into their existing organizational structure.  This means that many people use ESBs/SOA to essentially optimize what they are doing today, and as such the benefits of this must be limited by the organization that owns and implements the SOA. What I did not hear is how SOA changed their organization(s); in fact I only heard this question asked by a handful of people in all my years working with SOA technology.

This got me thinking...  When will we really know that a successful SOA implementation is achieved?  Sure, if the SOA matches the current organization, then essentially the existing systems and processes can be optimized, but the maximum potential appears to end there.  If, however, SOA changes an organization in a bigger, more fundamental way, the chance for even greater benefits and success can be achieved.

Perhaps most successful SOA implementations come as part of an organizational shift as well as a technology shift. This obviously means that the biggest reward comes with the biggest risk, and I guess more chance of failure is also possible.

SOA What?  Does your SOA project change the world or just your immediate surroundings?  And, if the organizational scope creep happens, can you and your SOA cope? Because, lets face it, scope creep will happen...

Loose coupling versus decoupling ...

Last week Jeremy Suratt made an interesting point on the Infor Tech Talk blog.  He says that event-driven architecture (EDA) is different from traditional service-oriented architecture (SOA) in the manner in which applications are related - or not related.  In particular he says "key to event-driven SOA is the ability for decoupled, asynchronous operations."  I agree wholeheartedly with Jeremy and think this point is one of the least well understood aspects of modern enterprise architectures and the complexity inherent in large integration projects. 

There is a great book on this topic entitled "Software Fortresses" by Roger Sessions.  Though the book doesn't mention the term SOA (it largely predates SOA and the current hype), it does point to the fundamental need for decoupling of systems through asynchronous messaging.  In the book, one of the golden rules is "never let transactions cross software fortresses."  I may be misquoting this a bit ... but the essential point is that between application domains, which are referred to as fortresses, you should allow for no dependencies which might block a transaction's completion.  This rule flies in the face of what most SOA infrastructure advocates today suggest is the very purpose of this new architecture.  Aren't we to just make a Web Service call when we want to leverage components in other systems?  Roger says - just because you can doesn't mean you should.  Once you grok these simple architectural best practices and the power it brings through "isolation", you're fast on the road to understanding the difference between traditional SOA and event-driven SOA.  The latter abides by the golden rule by inverting the control of intra fortress communications - using an event publishing rather than a service invocation model.  The former decouples the applications and the latter couples them - albeit loosely through Web services. 

Perhaps this is a long winded "dah" moment for you, but for many organizations it's still something they wrestle with daily.   Progress chose this decoupled model for the design of our integration backbone, Sonic ESB.   As a result, applications integrated with Sonic ESB are naturally decoupled.  This means you can shut down any application without impact to the overall operation of the other applications.  Unlike systems that link through traditional SOA request/reply invocation... Sonic ensures that messages get through when the application comes back online.  Now this is very different from other ESBs on the market or available through open source today.  These ESBs employ a "traditional SOA" architecture - meaning that applications connected through them are coupled via service-proxy rather being decoupled through asynchronous message forwarding.  I don't want to belabor the point, but a loose coupling of Web services is an insufficient architecture for large integration projects.   If you're not familiar with this type of ESB, I encourage you to download our latest Sonic ESB 7.6 version and go through the tutorial on "batch to real-time" processing - which shows the power of event-driven integration in the form of a continuous pipeline processing example. 

Marrying Best-of-Breed Tools for SOA Governance

1+1=3: Marrying Best-of-Breed Tools for Business Process Governance
Tuesday, April 29, 2008 at 11:00 am EST
Register to participate >

Communication is the basis of any successful relationship, including the communications between business people and technologists. Bringing together BPM and SOA Governance enables business drivers to be translated quickly to SOA implementation, with full accountability, lower cost of ownership, and business-level SOA optimization.

Join David Bressler, Actional Product Evangelist, and Marc Smith, Director of Product Marketing, Lombardi Software, tomorrow, April 29th, for a live webinar entitled, 1+1=3: Marrying Best-of-Breed Tools for Business Process Governance .  Listen to what David and Marc believe are SOA best practices for company's looking to ensure consistent enforcement of policies across all teams building SOA applications.

Other upcoming SOA infrastructure webinars you may be interested in, include:

• Visibility and Tracking for Critical Applications – Optimize Application Performance and Increase Revenue
  Tuesday, May 6, 2008 at 11:00 am EST
  Presented by: Julianna Cammarano, Sr. Product Marketing Manager, Progress Software

SOA Ironic

If you were listening to the webinar I gave a few days ago, you probably remember the phone system going haywire with feedback (don't worry, if you listen to the recording we're posting, this will have been edited out).

One of the topics I was talking about right at the time the feedback occurred was the "deer in the headlights" problem that IT faces when a business user tells them something is wrong with the applications since they often have no idea that a) that there is a problem, b) where the problem is, and c) what the problem is.  If you remember my theme, it was that good visibility is the solution.

What you didn't get to hear was that, after the webinar was finished, we talked to the vendor that ran the webinar about the feedback problem.  What was their answer? I quote, "We don't know where the problem is."  They, of course, promised us they would have their quality assurance specialist look into it.  And their answer was... "We're not sure what happened."

SOA What? Well, the moral of the story is that business service failures don't only occur with SOA infrastructure, but the same approach can be used to address them with SOA and without: visibility.

Before you start writing you own ESB ...

Vincent Partington recently wrote on his blog "before you start writing your own services and your own ESB, have a look at what's available."  Vincent point out the challenges of migrating from a home-brew ESBs and recommends against a "not invented here" attidute.  Reminds me of a dinner conversation in Winnipeg, Manitoba several years ago following my presentation of a new SOA Maturity Model to a group of local IT directors and executives.  The gentleman next to me asked a simple question - of all the services in the world (SOA) which ones are the most important to do first?  I was a bit set back by the question since I'd never had someone ask such a straight forward and obvious question.  After chewing a bit on my dinner I admitted how profound it was and then suggested - file parsing, transformation, routing, and file writing were probably the services that everyone should invest in first.  These are precicely the same set of services now offered out of the box - in both commerical and open source ESB (enterprise service bus) implementations. 

At Progress we've gone a step further to document larger enterprise integration patterns such as "batch to real-time" and "remote data distribution" on our developers network, and we also offer public code share libraries for users of Sonic ESB.  So, long and short - consider downloading Sonic 7.6 and leveraging some of these prebuild integration services, code shares and higher level EAI patterns.  I think you'll see that Vincent was right on the money with his advice, "...have a look at what's available." 

WSOA: What is a Common Model?

And how does it fit into SOA infrastructure? Listen to the podcast, What is a Common Model?, and hear Ken Rugg, VP of Products, DataXtend & ObjectStore, Progress Software, answer questions about a common model, including how to get one, the benefits of using one, why companies resist using one and more.

Listen to the podcast: What is a Common Model?

Live Webinar with Dan Foody

The Risks, Strategies and Payoff: SOA Runtime Governance
Tuesday, April 22, 2008 at 11:00 am EST
Register to participate >

Cut through all of the confusion around SOA governance to learn the ins-and-outs of the two key aspects of SOA runtime governance: visibility and policy enforcement. Hear about the challenges and strategies for addressing them. By learning how SOA runtime governance can directly impact your business, you will have the key information needed to build a business case to justify the investment in SOA solutions.

Join Dan Foody tomorrow, April 22nd, for a live webinar entitled, The Risks, Strategies and Payoff: SOA Runtime Governance.  Listen to what Dan believes are SOA best practices for company's looking to gain complete visibility into all the services that their applications consume. He'll also talk about how system and business professionals can easily monitor and ensure quality of service (QoS) that their business-critical applications - and customers - demand.

Other upcoming SOA infrastructure webinars you may be interested in, include:

• 1+1=3: Marrying Best-of-Breed Tools for Business Process Governance
  Tuesday, April 29, 2008 at 11:00 am EST
  Presented by: Marc Smith, Lombardi Software, Director of Product Marketing and David Bressler, Actional Product Evangelist, Progress Software
• Visibility and Tracking for Critical Applications – Optimize Application Performance and Increase Revenue
  Tuesday, May 6, 2008 at 11:00 am EST
  Presented by: Julianna Cammarano, Sr. Product Marketing Manager, Progress Software

Where is the next opportunity?

Whatever disaster occurs next, there is always someone able to turn it into an opportunity.  Look no further than the last dip in the economy and the associated Enron scandal.  Here was a company that lost millions of dollars overnight, causing distress to investors and employees alike.  However, from Enron came Sarbanes Oxley and as such a number of companies were created or changed direction to support SOX compliance and then went on to make money on the back of that disaster.

Now as we watch the latest downturn happening, we should be wondering what opportunities there will be and whether IT is ready for these changes.  Personally I can see a couple of things that are going to happen...  In the banking sector there is going to be new products that are going to appeal to buyers that are being cut off now.  For this, look no further than the 100% mortgages that were common in the UK just a few months ago and that have gone the way of the dinosaur. What is going to replace them when these banks must look to open up other revenue streams?  Governance is probably going to be one of the main areas of opportunity so that repeats of Bear Stearns are avoided at all costs.

From a technology perspective, I believe the following will be important: For those that have invested in SOA infrastructure, this is your moment to shine and to continue seeing the benefits.  Events will become important, and the ability to extract and view events, and then correlate them to monitor and also to drive new business processes will have a leading edge.  The other thing that is probably going to happen is M&A activity as values of companies go up and down. And as such those that have invested in agile, distributed and heterogeneous SOAs are going to show the wisdom of their investment as they give the business an opportunity to recoup their investment many times over.

SOA What? You should ensure that your SOA is ready for the new opportunities that will come.  What does that mean?  It means that the future is going to be a test of how quickly your SOA can change. It means that the rules may need be rewritten every month to support the constant change that is going to happen.  Will your SOA cope?

Web 2.0 and RIAs is Driving SOA Middleware Growth

An interview with Gordon Van Huizen, Vice President, Products, at Progress Software

If if haven't had a chance to read the interview by Jeremy Geelan, Sr. Vice-President of SYS-CON Media & Events, take a moment to read it.

Middleware is one of the fastest growing segments of the software industry today because of SOA. Gordon talks about why and how he believes that the increased interest in Web 2.0 and rich Internet applications (RIA) will further accelerate this growth. He also explains what he believes are the current barriers to SOA adoption and that IT infrastructures need to move toward normalization, as opposed to homogenization, and that the adoption of SOA infrastructure such as enterprise service bus (ESB) and SOA management will help normalize infrastructure interactions with backend systems.

Gordon also talks about the Progress Software approach and how he believes 2008 will be the "year of the events" and that events are the decoupling agent in modern enterprise computing architectures and are as important as services and processes.

A Problem Common to SaaS and SOA

I read a blog posting from the other day by a frustrated software as a service (SaaS) vendor venting that:

• They need rapid release cycles... but,
• They can't find all issues in their pre-production environments

For example, the author said "There are multiple points of failure and I can’t seem to think of how we could avoid these types of issues going forward."

Welcome to the world of running a service.  Whether the service is SaaS, a service you provide inside your organization (e.g. a SOA service), or a traditional business service (e.g. like those provided by Reuters, Bloomberg, and Thomson), the issues are the same: You can never anticipate all the ways your consumers will use the services in advance.  This means the risk of consumers experiencing problems is much higher than in other situations.

It also means you can't operate in a "let's get it right, ship it, and go on to the next project" traditional development-shop mentality.  You need to think about solving the problem in a fundamentally different way.

The first step to addressing this problem is being able to insulate consumers from one another (even when they use the same instances of your service).  The most important thing is to ensure that the actions of one consumer can't cause problems for another consumer.  For example, if a consumer goes into an endless loop making requests of your service, you need to ensure this doesn't impact your other consumers (for example, using infrastructure to implement request throttling controls).  Basically you're controlling the ripple effect.

Once you've figured out how to insulate consumers from each other, you can now think about "onboarding" each consumer as a separate project (even though you are in production your consumers aren't).  What assistance do you need to provide to them so they can successfully complete the project of going live with your infrastructure?  If, for example, they use REST or SOAP services you provide, how do you help them to ensure that they are using them correctly and appropriately?

Thirdly (and really this is just an optimization of the past point): How do you empower your consumers to do this for you (so you don't have to do all the work of making them successful).  You'll never be able to fully avoid the responsibility of "owning" consumer satisfaction (and fixing it if it's bad) - but you can put measures in-place so that your consumers can be more successful with less effort.  This can be as simple as providing much better documentation, samples, and tutorials - all the way to providing an interactive environment for the consumer to explore how to best use your service.

SOA on ACID

Tony Baer reported a recent IBM event where Steve Mills talked about how the next stage for SOA was to add ACID reliability and fault tolerance to it.  Am I the only one that doesn't understand this?

If IBM really means ACID (as in formal database-type transactions), then maybe they need a bit of re-education.  It's commonly agreed that ACID is not appropriate for a loosely coupled environment.  Instead, guaranteed messaging and compensation are the appropriate means.

But, if Steve Mills didn't mean ACID - and just means "reliable", then I'm also confused because there are a lot of organizations doing this today.  We have dozens of customers running high volume mission critical applications - where revenue depends on the reliability of their transactions... and they are doing this on SOA.

Now, I suspect that IBM isn't really planning on trying to run every application inside DB2 to get ACID properties (I suspect only Larry Ellison would propose a vision like that).

So, maybe, and this is just a thought, maybe what they mean is that when you stitch together 60+ CDs worth of different products and call it a "unified solution" (after all, they all have the same brand, so it must be a unified solution), it might not be the most robust approach.  If this is the case, I applaud IBM's desire to reduce the complexity of their product lines... but let's not talk about this as a deficiency of SOA infrastructure.

Extreme Volatility

Based on the title, this post can be about either:

1. The stock market
2. My personality

Obviously, there are two different audiences that might be addressed here. What's needed is more context around the description above to decide whether this post is actionable or not.

More context is needed. Where have I heard that before? (SOA What?)

Well, if you were in Europe in 2006 just after the acquisition, you probably heard me talk about capturing context of the messages flowing across your SOA infrastructure. It's one of my favorite topics because the benefit is so powerful.

Imagine having a system that pushes contextual information along with alerts, so that the root cause and business impact can be immediately known? This information drives action, improving user experience, and overall system performance. Cool, right?

Here's the message in my head:

1. Actional is different because we track EVERY SINGLE MESSAGE flowing through the SOA.
2. We do it across multiple protocols and platforms, giving people who care, an end-to-end view.
3. We track every single message, all the time, WITHOUT AFFECTING PERFORMANCE (it's magic).

What does this mean?

1. We have every message, and know the relationships automatically - so we can build flows that help us know where to look when a problem occurs, and where the dependencies are (even if we don't have prior knowledge of those flows ahead of time).
2. Because we're tracking every single message, there's no "inference" involved, no guessing - just the facts.
3. We can gather data about the flows and relate it to the flows to add business meaning to what we observe.

Why is this important?

1. No more urgently responding to alert messages that don't impact the business.
2. Preventative alerts let people know what's going on - improving the user experience.
3. The context provides the ability to act on events, in ways to preserve what's important and prevent further service deterioration - important customers, important orders, important channels of business can be prioritized above those less important.

And, most importantly, no more reading blog posts about SOA when you hope to read something about this crazy stock market.

Oh, yeah. I've got a webinar coming up on April29th entitled 1+1=3: Marrying Best-of-Breed Tools for Business Process Governance, during which I'll be using some customer examples of our partnership with Lombardi. If you remember, we shipped integrated SOA Governance with Teamworks, the Business Process Management platform from Lombardi, when we shipped Actional v7.1. Join us for the live event and hear about our success.