Business Making Progress

SOA. It’s back… but it’s got more sparkle.

I’ve been working in high-tech marketing roles for over 20 years now and every time some new marketing collateral comes across my desk and “I get it”, I get excited. As a matter of fact, my first job in a high tech company was as a marketing assistant to an R&D group at BBN Systems and Technologies. They were introducing a very innovative product called BBN Slate which was a multi-media editor for Unix. Because my previous jobs involved word processing (Wang), spreadsheet analysis (Lotus), and creating pretty charts (DEC/VMS – yes, really), “I got it” and I was committed to championing it. I loved every minute of it. Well, in 2007 I was tasked to optimize the Progress Software website for SOA. We already had great traction for the enterprise service bus (ESB), but not specifically for service-oriented architecture (SOA). Well, “I got it”, got going and started optimizing for SOA Infrastructure – a popular long tail term at the time - less than two months later. As a matter of fact, this blog was once the SOA Infrastructure blog.

In January of 2009, Anne Thomas Manes of the Burton group published the blog post SOA is Dead; Long Live Services. The industry had a lot of fun with that. Now, I don’t know if it was a coincidence, but suddenly Progress stopped talking about SOA and they asked me to change the name of this blog. I was sad, I admit it.

This responsive business integration diagram was taken from a presentation by Hub Vandervoort, CTO, Enterprise Infrastructure. Listen to the archive.

Well yesterday I saw some sparkle. We announced our new Responsive Business Integration (RBI) suite. Truth be told, I’ve known about it for a few months because I had to post the content and work on my SEO plan. I know it’s not nice to play favorites, but I was delighted to see my favorite Progress technology/products back in the game – Actional, DataXtend SI and Sonic. And, I’ve concluded that the RBI suite is SOA pimped up! It’s taking the existing service and application foundation most enterprises already have (or are building) and it’s enhancing it with semantics, policy management and mediation. The Progress RBI suite is going to allow businesses to decouple systems which will give them improved visibility, agility and the ability to change.

Read more about Progress Responsive Business Integration. And check out a great presentation by Progress customer Southern Union Company. They built their enterprise integration strategy around RBI and didn't even know it.

For the lack of a better term, do you think RBI it is our next-generation SOA? Give us your comments!

Multi-tenant Distributed Process Environment

As part of the Internals of a Middleware Systems course (one of the two courses I teach at IIIT-Hyderabad), I wanted to dabble in multi-tenant service oriented architecture (SOA) as a course project. About two months back when I first thought of this as the project, I assumed it was going to be an academic exercise where the class would build some interesting multi-tenant SOA platform (building on Camel, CXF on JMS, with a home-grown approach to partitioning the tenant domains for same set of services & processes (camel routes) available in the environment). But as I started getting more into it, I began to realize that there is probably more to it than an academic angle!

We actually started looking at business processes as being a more integral part of an application infrastructure, and not necessarily just part of the integration layer. Our thought process even brought us above and beyond our recently introduced OpenEdge BPM solution that enables our Application Partners to have key parts of their business logic defined as BPM processes, thereby allowing easier customizations for unique business flows and requirements. (Many years back had proposed embedded buziness preocesses (in Java applications) as a theoretical possibility in application architectures. At that time didnt realise there will actually be a simple platforms that brings both BPM & application logic together as well as our OE-BPM now does). With BPM and application logic now part of an integrated platform, it is only question of time before this architecture extends to a more distributed deployments with application logic composed as distributed services that is then orchestrated in the BPM layer to enable first level business capabilities and flows. Thus bringing in SOA also into the mix. And the moment this solution goes on the cloud in a SaaS model, the whole platform- Application logic, SOA & BPM- will now need to be multi-tenant capable.

Most of multi-tenant SOA seemed common sense.Still, when I was trying to find some academic basis, I came across this comprehensive overview in an IEEE article, Multi-tenant SOA Middleware for Cloud Computing. This article summarizes the various aspects of multi-tenancies - from the well known models for database centric solutions to enabling multi-tenancies (one DB per tenant, one schema per tenant or the more advanced tenants across multiple schema's (each shared)), to the more complex problem of actual application multi-tenancy. The application multi-tenancy also had a sound approach presented in this paper - Architecture Strategies for Catching the Long Tail - where they present a few levels, from the basic custom instance per tenant, to the next level where it is still one instance per tenant but the application is architected for easy configurabation. This paper further presents the next level where there is a shared instance that can service multiple tenants, to the most advanced level that it is a shared instance across tenants that can also be clustered & made highly-available. All great stuff. But what does this mean for SOA and BPM environments?

The more fundamental question that begs to be answered is what is the play for SOA in the cloud? Is there a SaaS play at all involving SOA? In the past few years that I've been dabbling in cloud  computing and talking about cloud environments at seminars here in India,  I never really thought SOA had a major angle. But surely in today's enterprise IT landscape, any enterprise integration solution will need to also access solutions in the cloud either by running a SaaS platform, custom built on a PaaS platform, or in a cloud-based virtual infrastructure platform. As SOA adoption matures, the other aspect that is rapidly emerging is a new class of business applications that use business process management (BPM) and service abstractions as a more integral part of solutions. All the simplicity, rapid development, composability and maintainability value that SOA offered for enterprise integration (in terms of atomic coarse grained business functions and processes that are composed off of these services, and that are easy to define and modify), now makes sense even for individual applications. Who would say NO to applications that are easy to build and maintain?

Given this, the use cases emerging for BPM solutions are now very different. And once BPM and SOA get into the application architecture, the need for (and value from) the cloud and multi-tenancy is immediate. We've seen the SaaS model get serious traction from ISV's because it delivers flexibility as well as a low-cost & ease of ownership. Once we go the full hog, then why just business processes? The applications can also have first level business services (aka SOA services) that can be composed into next level business processes and business workflows. A full-fledged SOA platform now forming the basis for applications.

So, in order to multi-tenant enable these BPM + SOA solutions, what are the challenges? ...Stay tuned.

Belgacom and the Case for Business-oriented integration in Communications Providers

It is no secret that the telecommunications industry has been exploring various ways to tackle issues such as declining ARPU (average revenue per user) and increasing customer churn over the last five years. The keys to proactively addressing these issues are agility and responsiveness. For example, responsiveness around customer service – to make customers feel they have a personalized valuable service – which reduces churn; and agility -- around launching new and compelling services rapidly – to increase ARPU.

Despite this quest for the Holy Grail, many communications providers have failed to lay sufficient foundations. While there may be some useful technologies deployed in an attempt to provide an agile and responsive integration platform – these technologies have often failed to deliver what is needed by the business. SOA is a prime case in point. While SOA technology has been successful in many ways, it has also led to a lot of disappointments. Often the business was sold on the promise that SOA would make operations more agile and responsive.  However, what resulted was technology for technologists that looked like plumbing and was daunting and too generic to the business.

Business people want solutions that can deliver visibility to problems and opportunities – such as key business events (e.g., an opportunity to sell a new service, based on context or location) or process failures (e.g. persistent dropped calls for an important customer) – so that problems can be responded to quickly and opportunities taken. The business also wants solutions that can enforce business-level policies and service-level agreements, as well as solutions that can interconnect services at a “semantic level” – not just plumb data together. In other words – the business wants to deal in business-level concepts, and be assured that the underlying complexity will be managed by the system.

Today’s announcement that Belgium operator Belgacom is transforming its business and IT integration programme (more information here) represents a huge step forward for the telecommunications industry. Belgacom is clearly focused around being more responsive to the needs of their customers in order to tackle issues such as churn through improved integration. By selecting the right business integration foundation to support their IT strategy, Belgacom has taken a critical step forward in a long-term strategy to become more operationally responsive to their customers.

Due to the nature of the economic times we live in, it is vital that the telecommunications industry as a whole ensures that they have the best possible integration technology in place. Only then will operators be able to enhance their customer experience management offerings to attract and retain their business customers, differentiate their offerings, and lower their service costs.

 

When it comes to Integration, Contracts Make all the Difference.

It may seem odd to talk about contracts and SOA in the same sentence, but without them experienced enterprise architects and integration developers understand the negative consequences that will ripple through the business.

A service-oriented architecture (SOA) requires establishing contracts between its participants. The first or most basic SOA contract provision concerns what formats and protocols are used. When two service participants, a provider and a consumer, establish a contract, it must start out with some protocol and format artifact (or transport and data-level semantic), such as an XSD and the associated WSDL describing the interface between that consumer and producer. These decisions establish the beginnings of interoperability, specifically how the first-level pin-outs get wired together.

Beyond this basic level, IT professionals tend to think about contracts in two broad areas: the service-level agreement (SLA) and security. For IT, an SLA usually specifies scale, response time, and availability: for example, handling 10,000 requests per second with a response time of less than 500 milliseconds, and availability to four 9s, or 99.99 percent uptime. From a security standpoint, a contract covers four or five areas.

In installment #7 of the Enterprise Integration Whiteboard Series, Hub Vandervoort delves into the sixth point of mediation - Quality of Service (QoS) and Quality of Privacy/Protection. But rather than providing a comprehensive account of the idea of QoS and QoP, the video podcast and technical paper offer a high-level overview of some capabilities within Sonic ESB by itself and in association with Actional, which is largely built into the ESB infrastructure, and how those come together to create the foundation of QoS\QoP—trust and commitment —required for effective contracts.

---

Check out all of the Enterprise Integration Whiteboard Series white papers and videos here!

Are you a sitting duck or one that will respond immediately to threats?

While many organisations are being ‘cautiously optimistic’ about what the future holds, the realities of today’s tough business environment could leave them as sitting ducks, according to Rick Reidy, CEO at Progress Software. They might take consolation that they’re in the same pond, but when interest rates in Japan hit near-zero, banks continue to fail and mistakes can lead to a ‘flash crash’, the pond is not a safe place to be. Businesses may have money, but fear and uncertainty is holding back decision-making – we await further regulation and want to know the consequences of recent government changes.

 
Listening to Rick’s keynote at our UK business summit (#progresswsummit, if you want to follow on twitter), in the impressive surrounding of Chelsea Football Club’s ground, London, it seems most of the audience agrees – it’s not good enough to sit around and wait to see if growth returns, and you cannot grow simply by cutting costs. You have to take control of your own ‘growth agenda’, as Rick put it. Businesses that want to survive the next five years need better visibility, through putting processes in place that enable them to react quickly to meet customer demands, adapt to market changes and take advantage of new opportunities. As Rick has advised, businesses need to act on up to the minute information so that leaders can make decisions based on foresight, not hindsight.
 
If you’re a regular reader of this blog you’ll already know that we call this ‘operational responsiveness’: the ability to sense and respond to customer and market changes so that organisations can move quickly to meet challenges and take advantage of new opportunities. 
 
Rick has talked about what this means in the airline industry: the notion of irregular operations has become a weekly reality as companies face intense market pressure, striking staff and disruption from natural phenomenon. ‘Swivel chair’ communication between operational areas is no longer good enough. To react quickly enough, they need responsive processes in place that can help them maintain services and inform customers, almost as-it-happens. If they don’t, they will face massive fines, lost custom and damaged reputation – risks no company can afford at present.
 
We’ll be hearing more from Gordon Penfold, CTO at British Airways, about their approach to becoming operationally responsive to meet the challenges of today and tomorrow. Watch this space for my take on his talk…

 

Stamford Bridge, here we come

Tomorrow sees Progress Software taking over Stamford Bridge, home ground to the world-famous Chelsea Football Club. We’re not just there to check out the players’ dressing rooms – we are being joined by James Caan, of Dragons' Den fame, as well as the great and the good of the UK business community, to discuss how businesses can start to make decisions based on foresight, not hindsight, in their operations.

Gordon Penfold, Chief Technology Officer at British Airways, will be sharing his insight on ‘operational foresight’, revealing how the organization has set itself up to better deal with the irregular operations that have become a fact of life in the last year. And Mike Gualtieri, senior analyst at Forrester Research, will be sharing his views on where the next wave of truly responsive business management is coming from, and which trends to watch for. And Progress' own Chief Executive Officer, Rick Reidy, will be giving a keynote too.

I'll be there, speaking in one session but also blogging and tweeting from the event. So watch this space for the latest updates.

For those of you attending, I look forward to seeing you there.

www.progresssoftwaresummit.com

 

Who's Job is it (to Sequence and Process Errors) Anyway?

It is a common software and integration architectural principle that the more dependencies built into a service, the less reusable that service becomes.  So then, why do some many vendors and enterprises continually create tightly coupled services and therefore lose re-usability?

The answer likely has something to do with mediation, more specifically sequencing and error recovery mediation. Sequencing and Error Recovery mediation are the two topics covered in a new webcast and technical paper posted today on Progress.com.  Both discuss why and how you should delegate sequencing and error recovery completely away from services, how this makes services unaware of what order they’re called in and the order the processes execute in, and ultimately how this makes services maximally reusable and your business more operationally responsive. 

The paper also explains how the Sonic ESB is designed to perform sequencing and, with multiple output paths on each endpoint, enable error recovery processes that are separate from the “happy” path. Just as important, Sonic offers the unique benefit of enabling you to orchestrate services using BEPL or itineraries—whichever is optimal for your process scenario.

Be sure to watch the Sequencing and Error-Recovery webcast and download the technical brief to learn more about the Seven Points of Mediation and the importance of each in relation to supporting a truly agile and responsive business application infrastructure.

---

Check out all of the Enterprise Integration Whiteboard Series white papers and videos here!

The Seven Points of an ESB

When discussing an ESB, many people keep it simple, sticking to the generalities of SOA and sharing services to build composite business applications.  Certainly, these are incredibly important benefits of an ESB but what do these dicussions really mean at the architecture level? 

Another word for integration when it comes to ESBs is mediation.  After all, that is truly what an ESB does - mediates the differences between application components (services) that were not intended to work together.  So then, in context of mediation, how should an ESB be examined?  Here at Progress, we view there to be seven points of mediation that serve as a set of key principles that apply to any integration infrastructure. They are critical because adherence to all seven is necessary to achieve the ultimate goals of SOA: reuse and agility.

Any one point that is not mediated becomes a point of brittleness. Put another way, interoperability needs to be tested along seven key points, which, briefly, are:

7 POINTS OF MEDIATION
Transport/Protocol
Location/Destination
Semantic/Format
Sequencing
Error Recovery
Quality of Service/Quality of Protection
Interaction Model

To help examine these points in-depth Progress has created a series of technical briefs (white papers).  These papers use the 'Seven Points of Mediation' as a way of describing how an ESB is, in fact, a comprehensive infrastructure onto which services can delegate these points. The overriding idea is that if a service delegates each and every point of mediation, it becomes more reusable in more contexts. These papers explain how Sonic ESB enables a service to free itself of these considerations, ultimately, so that it can provide the agility that SOA promises.

Be sure to watch the Transport, Location, and Semantics podcast and download the technical brief to learn more about the Seven Points of Mediation and the importance of each in relation to supporting a truly agile and responsive business application infrastructure.

---

Check out all of the Enterprise Integration Whiteboard Series white papers and videos here!

Just how important are SLAs to BSPs in how they support, retain and win customers?

Progress recently commissioned independent research company Vanson Bourne to find out and here's a link to the results:

Survey Results: SLAS: Lost In The Cloud?

Read about the increasing importance of service-level agreements and the difficulties in achieving them. Whether you're a Business Service Provider (BSP) or a client of a BSP, Service Level Agreements (SLAs) deserve your attention. This report will give you some information about the changing nature of SLAs and how you can use them to your advantage. 

You can also learn more about how Progress® Actional® will help you manage your service-level agreements by enabling companies to meet SLAs and manage services to support business goals. With policy-based alerting tools, users can set service alert thresholds and define service behavior for dynamic IT management. Actional also provides business analytics about runtime operations and controls for changing process behavior, for example, to give high-value customers priority service.

Dynamic Routing Architecture - Strength Comes from the Core

This week's deep dive into the gears of Enterprise Integration take us to the Dynamic Routing Architecture or DRA.  SonicMQ's Dynamic Routing Architecture (DRA) technology allows the delivery of messages between applications regardless of the cluster that the application is connected to. In case of a connection failure, (e.g. between regional offices), DRA will route messages via alternative operational paths, and facilitate expansion without incurring significant administrative overhead. Clusters may connect to other clusters as needed, creating highly distributed deployments across loosely-coupled locations.

As mentioned in the first series brief, “Messaging Architecture,” every Sonic broker has a set of acceptors that define it’s connectivity to the rest of your messaging architecture. These can play three roles: as a client link, as an inter-broker link (in a cluster) and as a DRA link enabling two clusters to communicate across a WAN. In addition, because Sonic acceptors can handle multiple protocols, the DRA link can work over other various types of connections as well.

But, DRA goes beyond the Sonic clustering architecture. It optimizes communications in many complex, distributed topologies while providing administrative transparency, or single-entry administration. As a result, DRA enables easier scaling and greater agility in changing the IT environment to meet changing business needs and market
conditions.

Be sure to watch the DRA video podcast and download the technical brief to learn more about how DRA provides unmatched high-availability and delivers transparency and administrative efficiencies that truly ripple to both the bottom and top-line of business activity.

---

The forth brief and whiteboard video in the Enterprise Integration Whiteboard Series, Dynamic Routing Architecture,  explains how the Progress enterprise integration solutions work. This brief focuses on the Progress® Sonic® Dynamic Routing Architecture® (DRA), its purpose and capabilities over and above the Sonic clustering architecture as well as some special use cases..

Join Us At Progress Exchange 2010

Progress Exchange Online Conference is coming to a computer near you, September 14 – 16, 2010. This free virtual forum is the place for sharing ideas, tips and best practices on how to benefit fully from Progress OpenEdge in the cloud. You’ll be joined by OpenEdge users from around the world to explore hot topics like:

• Modernizing OpenEdge applications using GUI for .Net
• Combining Microsoft Silverlight and Progress OpenEdge
• Leveraging the latest and greatest of OpenEdge Architect
• Previewing OpenEdge 11 and the NEW multi-tenant database
• Enhancing OpenEdge performance
• Deploying OpenEdge in the Cloud
• Understanding how Savvion (BPM) and OpenEdge can work together

Register now and get ready to choose from over 36 interactive sessions in 6 tracks, including:

• Integration and Process Management
• Best Practices and Application Modernization
• Developer Tools & Productivity
• Operational Efficiency
• Software as a Service/Cloud Computing
• UI Flexibility

If you register before August 30th, you’ll be entered to win an Apple iPad. For more information and to register, please visit www.progress.com/exchange2010.

Integration & Performance - It's all about the Clusters

As we continue our series on Enterprise Integration as a foundational element for Operational Responsiveness we venture into the world of Performance, Scalability and Reliability.

These characteristics are vital to distributed networks, and in large part, depend on transparency in the cluster to be fully realized. A cluster is a number of independent machines that behave as though they were one single logical entity. The rationale for clustering is that this single logical entity will easily scale and provide uniform performance: if one member fails or goes offline, messaging can continue. There are three dimensions of transparency that can be examined to determine the efficiency and effectiveness of an enterprise messaging solution in delivering this vision. These are administrative transparency, functional transparency, and high-availability and load-balancing transparency.

Check out the complete discussion on this topic by viewing the video podcast whiteboard session or downloading the transcript as a technical brief.

The second brief and whiteboard video in the Enterprise Integration Whiteboard Series, Clusters and Technologies, examines how the Progress enterprise integration solutions work. This brief focuses on administrative and functional transparency in the cluster architecture of Progress® Sonic® versus its competitors, showing why Sonic provides greater operational responsiveness at lower cost.

Integration and the Responsive Enterprise

Over the last year you’ve heard a lot about Operational Responsiveness from Progress and what it means to and for your business - the ability to:

• Respond to changing business conditions as they occur;
• Capitalize on more opportunities, to drive greater efficiencies;
• Make real-time course corrections;
• Reduce risk.

Every business and enterprise wants to be operationally responsive, but getting there is the challenge. Certainly, our recently announced Progress® Responsive Process Management (RPM) suite makes this challenge easier by enabling business users to gain visibility into critical processes, immediately respond to events, and continuously improve business performance without disrupting existing infrastructure. But what about that infrastructure? What role does that play in responsive process management and Operational Responsiveness?

We believe it plays a foundational role and that is why we’ve created a completely new set of educational and informative videos, whiteboard sessions, podcasts and technical papers that services and application platform architectures in the context of Operational Responsiveness.

Beginning with four podcasts and technical papers on messaging, the Enterprise Integration Whiteboard Series will explore services integration, data connectivity, transformation and life cycles, legacy modernization, and management as core, interdependent components of every modern enterprise application integration infrastructure. In total the series will consist of 10 topics with a new topic posted every two weeks through September 2010.

The first brief and whiteboard video, Messaging Architecture, examines the four basic components of the Progress® Sonic messaging system --brokers, acceptors, clusters, and the underlying protocols—to show how they provide not only robust integration, but business and IT benefits as well.

Read the paper and watch the video and let us know what you think by commenting to this post. Share your enterprise integration experiences.

Tomorrow - Webinar Discusses How Smart Grids Are Helping Utility Companies

Paving the Way to Smart Grid Modernization: A Standards-Based Reference Architecture
Wednesday, February 17, 2010 1:00 pm (EST)
> Join us for the event: http://bit.ly/smart-grid-webinar

During this live event, presenters - Terry Nielsen, VP at UISOL, and Conrad Chuang, Product Marketing and Mark Brooks, Principal Solution Architect, at Progress Software - unveil our unique, standards-based reference architecture that uses a product integration bus to simplify technology deployment and upgrades. By leveraging their domain, technology and integration expertise, they'll provide insights on:

• Market and regulatory conditions.
• The readiness of legacy applications to support real-time smart metering data.
• he cost and difficulties of integrating disparate utilities systems
• Steps for moving to a smart grid-compatible environment, cost-effectively and non-intrusively.

Pre-register or join us tomorrow!
Wednesday, February 17, 2010 1:00 pm (EST)
Wednesday, February 17, 2010 10:00 am (PST)

View the archive of this webinar >

Reduce Order Fallout and Improve Your Customers' Experience

A New Video / Flash Presentation

Our audio presentation, Lost in Transactions – A Day in the Life of Application Failure, presents the fictitious story of one company struggling to find the root cause of application performance problems. If your enterprise identifies with their pain, take the next step by learning more about business transaction assurance with Actional.

Visit our website to view the Flash, or visit the Progress Software YouTube channel to watch the video.

Enjoy!

[Note: The voice of Karen is me, and the voice of Chuck is our own David Bressler.]

SOA? What do you call it?

Some of you may have noticed that we renamed our blog from SOA Infrastructure to Integrated Infrastructure. This came about when our product marketing team wanted to create a new “Open Integration” blog – same authors, same topics. Admittedly, I wasn’t a proponent of it because I wanted more blogging activity here, and I wanted to leverage the audience (and SEO truth be told) that we've built up over the past two years. So, I recommended that we re-brand it and we did.

I have no doubt that one of the reasons we wanted to move away from SOA was due to last January’s post by Ann Thomas Manes, SOA Is Dead; Long Live Services. Lots of SOA evangelists commented about the post, including our VP of Products Dan Foody who agreed with Anne’s perspective. Me? I personally think that SOA in itself is just a marketing term for a number of fairly distinct things, including enterprise integration.

With that said, this wouldn’t be a post by me if I didn’t offer or promote something, so I’m going to let you fill in the blanks.

Our new white paper The Foundation of _________ Quality is now available! What does _________ quality really mean? This white paper not only answers that question but it also examines the many facets of _________ quality. Read it and learn how you can ensure that your _________ initiative, such as Web 2.0, cloud computing, and BPM, can deliver the visibility and operational responsiveness that your enterprise demands. Get the white paper.

If you’d like to learn more about how your enterprise can achieve operational responsiveness, visit the Progress Software website.

Adding Leading BPM (Business Process Management) Solution to Our Portfolio

Our integrated infrastructure (or SOA infrastructure) portfolio just got broader and better! On Monday Progress Software announced the acquisition of Savvion, Inc.  Savvion offers a comprehensive, standards-based BPM suite that helps more than 300 of the world’s top-performing companies – including 24 of the ‘Fortune 100’ – automate and continuously improve critical business processes. Dr. John Bates, Progress Software’s CTO and Head of Corporate Development, says, “The Savvion BPM suite is a perfect fit for Progress because it offers leading capabilities for business process modeling and execution. The suite also uniquely includes other integrated key capabilities, including business rules management, document management, an event engine and an analytics engine.”

Progress Software made the announcement during our Global Field Operations Conference in Orlando, FL, which is being held this week. Those lucky enough to attend were able to hear David Bressler deliver a great sales pitch that really communicated the benefit of having the industries best-in-class BPM technology in our briefcase. The combination of our Business Event Processing (BEP), Business Transaction Assurance (BTA) and Integration portfolio, coupled with Savvion's BPM suite, will enable enterprises to achieve the highest levels of operational responsiveness.

To learn more about this announcement, visit our Apama Event Processing blog and read two posts by Dr. John Bates:

• Why businesses must evolve their business processes to be highly responsive, dynamic and predictive – or they will cease to be competitive
  
  
• 10 Reasons Why Progress Chose Savvion

Welcome Savvion to the Progress family, and stay tuned for more details.

Progress Software Announces Q4 results - Here are some highlights

Progress Software (NASDAQ: PRGS) just announced their Q4 earnings release. To summarize, it says "Earnings Up in Q4; Progress® Actional® Revenue Up with Triple-Digit Growth; Progress® Apama® Revenue Up with Double-Digit Growth." What I really thought was interesting was the Q4 highlights. The majority of wins involve building or enhancing an integrated infrastructure, and application modernization - both topics we cover in this blog. In case you missed the release, I've included these highlights below. Enjoy!

Q4 Highlights

• Progress Software announced that the Progress® Sonic® ESB (enterprise service bus) is deployed and operational at British Airport Authority’s (BAA) Heathrow Airport Terminal 5.
  
  The Progress solution enables BAA to provide airport integration capabilities using the Sonic ESB product. This includes the creation of reusable integration services for new Terminal 5 systems and of specialist adaptors for the integration of existing key operational BAA systems, such as the Airport Operational Database Integration. (Tag: Application Integration)
  
  
• Progress Software has successfully enabled more than 250 Independent Software Vendors (ISVs) to deploy thousands of on-demand, SaaS applications over the past five years.  These ISVs use the Progress® OpenEdge® SaaS platform to build applications that are used in some of the most demanding and diverse business environments in the world. (Tag: Cloud Computing)
  
  
• British Airways selected Progress Software SOA Solutions to upgrade their travel experience.  The UK’s largest international airline, British Airways (BA), will use the Progress portfolio of SOA solutions as a key part of its travel program to upgrade its IT systems by integrating over 600 different electronic systems and processes involved in getting BA passengers in the air. The flexibility of the Progress SOA portfolio allows BA to extend the features of its e-commerce site right through to its airports, by allowing greater self-service functionality and 'plug and play' capability. (Tag: SOA Success)
  
• match2blue stands out from the crowd with the Progress® Apama® Business Event Processing (BEP) platform by adding real-time capability to next-generation social networking. Enterprise platform enabler for mobile solutions, match2blue (www.match2blue.com), has selected the Apama platform to empower its social networking platform with real-time information on location, ideas, news and trends.  The Apama BEP platform will form a crucial part of match2blue’s back-end infrastructure, providing the performance and scalability needed, as well as supporting its business partners, who will be operating the location-based services to control and monitor their operations through dashboards. (Tag: Complex Event Processing)
  
  
• Alphameric Solutions Ltd, the leading solutions provider to the gaming industry, selected the Sonic ESB to revolutionize the way it handles content and messages across its network. Relying on highly complex and automated processes to deliver odds, prices, race information and documents across a distributed architecture – most needing to be handled in a sub-hundred millisecond timeframe – Alphameric needed a simpler way to incorporate new or updated information in real-time. (Tag: SOA Best Practices)
  
• West Bend Mutual Insurance Company has selected the Sonic ESB (enterprise service bus) and Actional products to underpin a service-oriented architecture (SOA) based IT infrastructure.   West Bend Mutual Insurance, a property and casualty insurance carrier, is pulling together dozens of disparate internal policy administration applications into a single integrated insurance portal. (Tag: Distributed SOA)
  
• Progress Software announced the availability of the Apama 4.2 Event Processing Platform.  The Apama 4.2 release extends the capabilities of the previously announced Apama Parallel Correlator, and introduces significant new developer productivity features that accelerate the deployment of event processing applications. The Apama Parallel Correlator leverages multi-core, multi-processor hardware to deliver high throughput, low latency execution that has achieved seven-fold performance improvements, as benchmarked with real-world customer applications. (Tag: Event Driven SOA)
  
• Slumberland, a leading furniture retailer, is now using standards-based data connectivity products from Progress® DataDirect® for reliable, high-performance support for all their major databases and 64-bit operating systems, for reliable connectivity to their Oracle applications, and streamlined reporting to improve fulfillment and customer satisfaction. (Tag: Semantic Data Integration)
  
• Progress unveiled the industry's first mainframe SQL engine for non-relational data, which can leverage zIIP specialty processors for lowering a mainframe’s total cost of ownership (TCO), with the announcement of its DataDirect Shadow Release 7.2.1.  The DataDirect Shadow release includes ANSI SQL-92 to Non-Relational Data with zIIP Offload and new capabilities that lower costs and attract new process-intense workloads to the mainframe.

Now That's a Real Forklift Upgrade

I have to admit... I don’t really know how our customers use OpenEdge. I do know there are a ton of customers - over 65,000. And, if that weren’t enough, there are over 1,500 partners too. What's more, many of them are in-production with SaaS offerings.

Damn, that’s a lot.

(If any analysts are reading... just think about the opportunity of selling Actional into that installed base even if we never got another “new logo” sale.)

This week’s press release follows on from several months of a beta period where about 20 or 30 OpenEdge customers tested the newly released Actional integration.

As TVH Forklift Parts realized, knowing what’s happening in their integrated infrastructure, and being able to assure a consistent level of service has tremendous value to a distributed and shared infrastructure.

Why is this important?

It’s about the business context. Without that context, solutions are just technology (we have good technology too… but that’s not enough).

That’s the difference between assurance and management. Assurance implies business-technology coordination to achieve a business result. Management implies your technical components are up and running. Big whoop. Just today I spent 2 hours on the phone with T-Mobile. All the technical components were up, but it still wasn’t working. I know you can relate.

Colleen points out that our partners are being viewed more and more as business partners, not just technology providers. Simply put, our partners need technology to understand the business impact of “events” within their infrastructure.

Understanding the business impact means that we (technology infrastructure providers) need to provide an awareness of the business context when problems occur. The only way to do that is to track business context all the time.

I’ve heard a few times recently of prospects who have a “competitive” solution in place to track business assurance… but when I probe, it seems they don’t run it all the time because (pick one):

1. It impacts performance of my applications. (it doesn’t scale)
2. It collects too much information. (it doesn’t scale)
3. It requires too much CPU on my app servers. (it doesn’t scale)

I don’t understand how people think a solution that doesn’t run all the time can do the job.

Let me rephrase.

If it’s not running all the time and collecting context of your business, how are you using the context of the business to make better run-time decisions?

Simply put, you’re not.

I’m glad to welcome TVH Forklift Parts to the Actional family. And, if you’re reading, thanks for sharing your story.

Meet Progress Software On The Road Next Week

Before we jump into the swing of holiday shopping, celebrations and announcing year end numbers, we're hitting the road. If you are attending Gartner AADI in Las Vegas or Management World Americas in Orlando, make sure to take time visit us. Here's where Progress Software will be:

7 - 9 December 2009
Las Vegas, NV
Gartner Application Architecture, Development & Integration Summit
This is the year's MUST ATTEND conference on Cloud Computing, SOA and Applications. Regardless of your challenge, you'll find answers that fit your needs in today's changed environment. And the best part... Progress Software is going to be there, at Booth #3. As a platinum sponsor, we'll be speaking about how our customer, Sallie Mae, used event-driven architecture to transform their IT department into a strategic partner for its business.

> Learn more!

8 - 10 December 2009
Orlando, FL
Management World Americas
In addition to visiting us at Management World Americas in Booth #112 during exhibition hours, you can see how DataXtend SI and Apama are being used in three TM Forum Catalyst projects. As part of the show's Forumville event, they are featuring five Themed Zones that focus on key areas for customers, services and networks – all driving towards optimizing business performance. See our products in action!

> Reserve your FREE book!

HOPE TO SEE YOU!

SOA Upgrade to British Airways

British Airways (BA) is turning to service-oriented architecture (SOA) and Progress Software to connect over 600 different electronic systems and processes involved in getting BA passengers in the air.

BA has more than 250 key applications distributed over 300 locations around the globe, which explains why they chose Progress. From an integration perspective, Progress® Sonic ESB® excels in those scenarios that require the integration and management of hundreds or even thousands of systems. One Progress customer was able to deploy its integration backbone out to 25 locations a day (and can update them in a fraction of that time).

One can only guess how many of those 250 systems are touched when you travel BA. Everything from choosing your seat assignment to charging your credit card (once only please), to selecting a vegan meal is managed via systems. And let's hope those systems succeed in keeping track of your luggage.

The real-time data synchronization of Progress® DataXtend® Semantic Integrator (SI) allows BA to significantly improves information quality while reducing costs associated with data replication. And Progress® Actional® SOA Management provides BA with the visibility they need to make sure your reservation gets booked and you make it to your destination in a timely manner. Weather permitting, of course. ;)

Thanks to SOA, BA is able to extend the features of its e-commerce site right through to its airports, allowing greater self-service functionality and 'plug and play' capability to over 25,000 users.

"Moving this to a highly automated environment is a challenge, but SOA quickly proved itself to be the right approach to achieving our goal of a fully agile environment."

So the next time you take your family to London (or Disney - that's my family's destination of choice at the moment), think about how many transactions went right along the way, automatically.

I love when technology makes a tangible difference. Pretty cool stuff.

British Airways Selects Progress SOA to Upgrade the Travel Experience

British Airways announced yesterday that they had selected Progress Software for a revolutionary project that is integrating more than 600 different electronic systems and processes which are involved in getting BA passengers in the air.  This new highly automated infrastructure will bring increased agility to the airline. Rollouts will be easier, and associated cost and time will be reduced.

As is often the case with these kinds of announcements, however, it wasn’t really news to an “insider” like me.  I had visited BA earlier in the year and met with numerous people in the organization from the CTO to Architects and Developers on a number of the projects where they are implementing our technologies.  The thing that was most gratifying to me in those meetings wasn’t the scope of this cutting edge project or even that they were implementing using nearly our entire portfolio of SOA infrastructure products, namely Progress® Sonic® ESB, Progress® Actional® for SOA Management and Progress® DataXtend® Semantic Integrator (SI).  It was that, at every level in the organization, they expressed how pleased they were in the selection of Progress.  

At the senior executive level, they were discussing the partnership that they had developed with Progress and the vision we had provided to address real business challenges, for example, improving customer service by extending the features of BA’s e-commerce website into airports.  At the architecture level, there was a great sense of partnership on how the products we provided could be brought together in a coherent SOA based approach to their infrastructure that increases agility and operational responsiveness.  The developers were just happy that it really was “best of breed” technology that “just works”.  

At a time when most airlines are cutting back, it’s great to see British Airways taking advantage of what SOA has to offer and at every level of the organization they can count on Progress as a trusted partner to help.

Alphameric Beats the Odds with Progress Software

It’s always great when a company is willing to talk publicly about your products and services, even when the public statement incorporates a bad pun. In this case, Alphameric Solutions Ltd., the leading technology provider to the bookmaking marketplace in the UK and Ireland, wasn’t holding their cards close to their chest when they decided to collaborate with us on a press release. Alphameric selected the Progress® Sonic® ESB to revolutionize the way they handle content and messages across their network because they couldn’t roll the dice with unreliable infrastructure. Sonic’s continuous availability architecture made this a sure bet.

With new virtual games such as poker and fantasy football being released every day, Alphameric was struggling to manually keep all the feeds and databases up to date. The new SOA-based approach makes accomplishing those tasks a lead pipe lock. Alphameric can now change data more quickly, easily bring new feeds online, without gambling with the accuracy through automation.

Using Progress, Alphameric is achieving operational responsiveness, increasing revenue and improving customer service to finish in the money. You can’t beat that. I’d add that this decision puts them clearly ahead of the game, but Matt Smith, our Senior Enterprise Architect quoted in the press release, beat me to the finish on that one.

OK, I think I got that out of my system now. Sorry about that...

CORBA is now Actional-ized!

This is really cool, and I can finally speak about it. Wow, gag orders just don't work very well for me. I mean, I can keep a secret, it's just that the really juicy ones are harder to keep than the others! And, this one's juicy. Ready...

Today we will release Orbix 6.3.4 with support for Actional.

Whoa! I bet at this point a lot of you are like SOA What? (yeah, remember that blog-ism we did here back when SOA was cool?)

Well, let me give you end-to-end visibility into the import of that sentence above.

As with all the other technologies we cover, including the recently announced SAP ABAP integration, Actional integration with Orbix gives customers:

1. End-to-end message flow visibility across an entire CORBA environment, in run-time, without any impact on performance or scalability. That same visibility extends beyond CORBA, and does so consistently to provide a unified view of message flows and business transactions in their environment.
2. The ability to centralize run-time policy creation, allow for distributed enforcement across multiple environments, and to ensure compliance with corporate and regulatory policies across the entire messaging infrastructure.
3. The same tools our current customers have that help them achieve an 80% reduction in time to resolving critical production incidents and a 20% reduction in major production incidents per year by simplifying root-cause analysis and problem resolution.

Now, you might think with all these great benefits, there would be a high cost. Nuh uh!!!

• No additional coding changes to get this up and running. Instrumentation of all CORBA services is automatic.
• No architecture changes, Actional won't impact scalability.
• No extra capacity required, Actional won't impact performance.
• No additional staff are required, dependencies are dynamically discovered and maintained to help keep the cost of ownership low. In fact, case studies independently verified by Forrester Research of our customers who are in production showed a rather quick payback of the investment due, in part, to the low cost of ownership.

Of course, if you follow Actional and Progress at all, awesome technology is old news. Actional has been working on non-SOA distributed applications since early in the development of our product and anytime we add a new technology, protocol, or platform, we've always added the same rich features, while maintaining the enterprise-class performance. Let me say that again, because I've accidentally brought up a very important point.

The single Actional Agent adds all of the functionality, across all of the protocols and platforms, and provides the same outstanding performance you've come to expect from us. In contrast, many other vendors can support all the platforms/protocols, and all the business-transaction-management features, and do so non-intrusively... but THEY CAN'T DO IT ALL AT THE SAME TIME!

Back on track, sorry for the diversion. I'm almost done here, I promise.

There are other equally interesting implications from this release that I'd like to share (in no particular order).

1. This helps to, in part, validate the strategy that lead us to acquire IONA in the first place. There are a lot of cross-product synergies.
2. This continues to prove the applicability of our technology far beyond the common understanding of SOA infrastructure (SOA and HTTP/JMS). We have successfully broken out of the "SOA management" niche, and are providing real value to customers across the entire breadth of business transaction management needs.
3. In this "recovering" economy, Progress can help customers gain more value from their legacy technology investments by upgrading the capabilities in mission critical platforms like Orbix/CORBA. Whether it's from the operational perspective of lowering their operating costs while providing higher-levels of service, or from the business perspective of preventing "revenue leakage", Actional solves real-world problems without impacting the architecture or requiring coding changes to existing applications.

In closing, I wish I could share some of the early adopter customer quotes here. I can't, but the feedback from the (former) IONA field was equally funny. The IONA field is (relatively) new to Progress and hasn't necessarily developed the instant-automatic love for each-and-every Progress product because we all share a logo color scheme. We actually had to prove ourselves to these guys and gals (a very capable team I might add). Once the engineering team was finished, they took the early software and implemented it at some rather large (and equally skeptical) customers. After running it through it's paces, people were actually smiling. When was the last time you saw beta users smiling at performance results? To quote one guy working at a large airline... "it simply worked the way it should."

Beat that!

Two Progress Customers Honored in Insurance & Technology's Elite 8 for 2009

Progress Software is pleased to report two of our customers were named to Insurance and Technology’s Elite 8 for 2009. To paraphrase the editor-in-chief of Insurance and Technology, this annual award identifies outstanding insurance technology leaders, who are managing IT in the new highly scrutinized world of financial services, with enthusiasm and determination.

In this article and video Keith Sievers from Unitrin describes how they improved their operational responsiveness by implementing an ACORD-Standards based architecture “for reduced costs, better quality, faster time to market and greater reliability and performance." Unitrin is using Progress Sonic ESB with Progress DataXtend SI to “federate and share technology across the three Unitrin companies through shared services.”  One interesting fact is that Keith Sievers and his team recognized early on that including ACORD standards in their “enterprise philosophy” would contribute to their success by reducing risk in their implementation.

Unitrin’s successful application of ACORD standards is interesting in light of fellow honoree John Kellington from ACORD. John is another Progress customer, and another of Insurance and Technology’s Elite 8 (article). John was honored for his leadership in advancing the ACORD Framework and Enterprise Architecture. It’s ACORD’s belief (and ours too) that an enterprise architecture which will enable insurance IT organizations to simplify their system portfolios, reducing maintenance burdens and improving development capabilities. John points out that reducing IT ecosystem complexity (often represented by those infamous “Spaghetti charts” or “New York City sewer maps”) eases that “resource strain that seems never-ending.”

Progress is proud to be playing a role in ACORD’s efforts to develop these critical assets for every insurance IT organization. Progress was selected to support the ACORD Standards Framework with unambiguous maps between the new ACORD Information Model and its existing Property/Casualty/Surety, Life/Health/Annuities, and Reinsurance/Large Commercial XML standards. And we have worked closely with ACORD to release the DataXtend Browsers for ACORD with the aim of enabling the ACORD-membership to become even more familiar the XML standards.

You can always learn more by emailing us at sales@progress.com. But, if you have the time, you should also consider attending the ACORD Implementation Forum in Fort Lauderdale, FL Nov 3-6, 2009. it’s a great opportunity to learn and network. This year AIF promises to be an exciting event this year with many interesting programs including:

• Frank Neugabauer (ACORD) and Boris Bulanov (Progress) introducing the ACORD Framework and semantic mapping from 4:00pm-5:00pm on 3 Nov
  
  
• Demonstrations of the DataXtend Browser for ACORD at the “You Drive” Sessions from 12:30-2:30pm on 5 Nov
  
  
• Conversations and networking with experts, like Boris Bulanov, at the Expert Café and Lunch from 12:30 – 1:30pm on 4 Nov
  
  
• And finally, the opportunity to receive your own copy of the ACORD Information Model: A Primer just by visiting Booth #3 at AIF.

The great cloud crash of 2009

I've been following the recent story of how Microsoft's Danger division and the T-Mobile Sidekick.  If you don't know much about Danger, it's basically a cloud service tied to a hardware device (the Sidekick).  All the data is stored in the cloud (email, contacts, etc.) and cached on the device so that when the device resets, it starts up empty and reloads its data from the cloud.  Unfortunately, many customers found out recently that when they reset their device, all their data was gone.  Poof.  No more contacts, no more calendar entries, no more emails.  It turned out they didn't have any real backups and their data redundancy was foiled by human error.  Whoops.  While they have recovered most of people's data now, there are some important lessons to be learned from it.

One of the articles I read on this was titled Don't Blame Cloud Computing for the T-Mobile Mess.  While the author's heart is in the right place (he really likes cloud computing), I can't agree: Cloud computing is absolutely to blame for this fiasco.

Let me explain by starting with an analogy.  Recently we had a little event that some people call the "great market crash of 2008".  The root cause of this is pretty well known now:  A lack of transparency into different financial instruments (e.g. CDOs) made it impossible to accurately assess risk.  And, in the absence of an accurate risk assessment, people assume things will be ok and focus on their short term gain.

It seems to me that we have the exact same situation with cloud computing.  There's essentially no transparency into any cloud provider's integration infrastructure, processes, or planning.   As a result no user can accurately assess the risk of using one cloud provider over another.  Do you think that if sidekick users had know "Danger doesn't do backups" they would have trusted the service with their data?  Of course not, most users assumed everything was OK.  They assume their cloud provider is doing the right thing.  It was only a matter of time before a crash would happen (and this won't be the last one).  In the immortal words of Otter from Animal House, "You f*cked up - you trusted us".

Cloud providers, unfortunately, think that it's not in their best interest to be transparent because, frankly, customers are conditioned to just assume everything is OK so why rock the boat.  When was the last time you walked into a grocery store to buy apples and said, "Can you cut this one open so I can see whether it's OK on the inside?"  No, you probably look at the shiny skin of the apple and assume everything is OK with the inside.

Before cloud computing can become mainstream, users of cloud services must have the ability to accurately assess risk for themselves.  In order to do this, cloud providers must provide transparency. if not users must demand it by speaking with their wallets.  Don't let the cool, shiny UI fool you into assuming everything is OK under the skin.

Smart Integration Infrastructure for Insurance Industry

Those of us who have been entrenched in middleware and SOA infrastructure technologies over the years know the importance of having a smart integration infrastructure. Well, today we released a press release announcing that West Bend Mutual Insurance, a property and casualty insurance carrier, chose Progress Sonic ESB and Progress Actional as core applications for their service-oriented architecture. West Bend Mutual Insurance will use Sonic ESB and Actional to supplement its existing policy administration system, so its insurance agents can conduct business more easily and effectively via a single integrated portal. The insurance portal will also be a critical tool to help them improve their customer retention and acquisition. The best part is that West Bend Mutual will finally be able to enjoy operational responsiveness by being able to respond to changing conditions and react more quickly to business opportunities.

For industries like insurance that need to constantly offer new products and services to remain competitive, creating an infrastructure that is agile and scalable, and one that delivers end-to-end visibility of back-end systems, is essential. SOA is a great fit. Read the complete release.

Recap: Gartner EA Summit in Orlando

This guest post comes courtesy of Larry Fulton. Larry is an independent consultant who spent 14 years as a solutions and enterprise architect at UPS and 3 years consulting on, among other things, strategic integration infrastructure issues and enterprise service bus (ESB) technology as a senior analyst at Forrester  research.

The enterprise architecture community is always surprising to me for its enthusiastic optimism - remember that this is a field where the majority of us spend a lot of our time explaining what we do and why it is valuable, often to our own management. Attendees were very clearly engaged in the session topics, there were plenty of insightful questions, and the attendees I spoke with personally saw a lot of value in the material presented. The focus was on the practice of EA rather than specific technical aspects of modern enterprise architecture, which of course begs the question what EAs are doing to stay on top of the technology landscape.

Gartner sees the influence of EA growing over time, especially in those organizations where EA is successfully involving itself in the business and its processes. Aside from the expected pro-EA and how-to-improve-credibility messages, I heard a number of new and refreshing perspectives on EA and its future:

Gartner's Anne Lapkin confronted the "Is EA an art or a science?" dilemma head-on, and clearly stated that many aspects of EA are in fact an art. She was referring specifically to the real work of fitting and re-fitting EA's mission to the current and evolving needs of real businesses. There may be a lot of well-defined process around the tools of the trade - modeling various aspects of current and future architecture, establishing effective governance processes, and so forth - but it takes real insight based on experience to assess what EA can and should be doing to help the business succeed, and to know when that needs to change as the business itself evolves. This is an area where many experts, have been reluctant to come right out and say, "Look, you need to have the right leaders, and you can't necessarily just pick someone who is skilled in another area and expect to train them to this level of EA perspective". Another way to say this is that skilled solutions architects need to be part of EA's activities, but solutions architects don't necessarily have the perspective to define the EA agenda.

Betsy Burton's sessions illuminated the reality that enterprise architects often must fill the role of counselors - working with disparate teams with different perspectives to find common ground and move forward. On the broader business front, this same theme emerged in her recommendation that enterprise business architectures need to include a model of how people actually work together in an organization.

She also mentioned that EAs need to spend at least five per cent of their time playing so they can remain aware of current technologies. My own opinion is that this is not enough - unless you can commit at least half a day each week to some kind of research, which is to say at least ten per cent of your time, it is very difficult to stay on top of important developments.

I particularly enjoyed Bruce Robertson's session on "architecting for emergence". He talked about "EA light", or ways that EA groups should focus on what matters the most and promote application team innovation elsewhere. The idea of establishing policy and technology "guard rails" that essentially say you can do what you like as long as you conform to these particular things and as long as you don't do these other things is not new. But, looking at an organization in a methodical way to identify exactly those rules that really matter and promoting choice elsewhere is where many EA groups need to be headed. Certainly his advice to EA groups to understand local influences and priorities and their relationship to enterprise influences and priorities is a good idea for any EA group, and especially those operating in large organizations.

One of the keynote speakers, Mark Rashino, represented in my opinion the central message to EAs - IT needs to ask itself, "What new strategic capabilities can I offer?" When CIO's are asking themselves that question but don't have an answer, where will they turn? EA needs to be ready to answer that question, whenever it is asked.

Business Transaction Management with SAP

If you follow Actional, you may have seen that Progress Software announced Actional 8.1 today.  The most interesting part of the release is our new support for SAP.  Yes, we already supported SAP NetWeaver (like most of the other people in our space) so unless you're an SAP aficionado, you probably won't recognize the importance of natively supporting SAP ABAP - which is what we've announced.

SAP supports two main application server environments: one based on Java and one based on ABAP, SAP's own programming language (you might also hear the term "basis" which is another name for the ABAP application server stack).  OK, with me so far?  While SAP support both, almost all of the SAP packaged applications are written using the ABAP stack - Java is primarily used for infrastructure services (things like their portal).

With this new version, we've added the ability to trace business transactions into and through ABAP - so we can detect problems even within the ABAP portion of a business transaction.  This is critical for many SAP customers because - without the ability to do this - SAP packaged application logic is seen as a black box silo.  And, with 100's of millions of lines of code in the SAP packaged applications, that a pretty big area to have a blind spot.

While there are a lot of business transaction management vendors out there that support SAP, they are usually referring to the Java side of SAP. As a result, once a transaction hits ABAP (and almost all of them do) it enters the black box and can't be seen again until it leaves ABAP.

As you might have guessed, we're really excited to extend Actional's patented transaction tracing to SAP's core platform.  If you're an SAP user, and need to ensure the success of business transactions that span SAP and other applications, platforms, and middleware then hopefully you'll get a chance to see whether this unique Actional capability can help you.

RE: The wrong marketing for open source

Matt Asay makes a good point in his article on open source marketing, that people generally choose to use open source software because it is cost effective, not because it is “flexible”.

Put another way, “free as in beer” trumps “free as in speech” as a motivator for people making the choice to use open source. I think that is obvious. On the other hand, I think that it is so obvious that the premise of the article that open source should be marketed on the basis of low cost over freedom of choice is wrong. I think the market widely assumes that open source is “free as in beer” so you get the benefit of that perception just by being open source in the first place.

That said, he does refer to the importance of “low cost utility.” It isn’t good enough that Linux or MySQL or ServiceMix ESB is open source, (i.e. perceived to be “free as in beer”). It also has to be good enough to be useful in helping someone solve a real problem. In other words, the beer has to be drinkable or doesn’t matter that it’s free.

We, Progress Software, are also seeing companies begin to recognize that adopting open source isn’t entirely free (as in beer). They will be more successful getting the utility from the software if they invest in developing the skills to use the software properly and have the backing of a company to support them in that effort. To take the beer analogy a bit further, (perhaps too far?), renting a mug is better than drinking the beer out of your hands.

From an open source marketing perspective, what we need to do is let people know that the beer tastes good and explain how much better the experience will be drinking it from a mug. We don’t need to spend a lot of time explaining why it’s a good thing that the beer is free.

Book Reveals How to Create SOA the Right Way

The book, An Implementor’s Guide to Service Oriented Architecture: Getting it Right, explores SOA related topics ranging from design services, registries and repositories, to runtime management, and organizing for success. The book includes insight offered by AmberPoint, BearingPoint, Composite Software, MomentumSI, and Progress Software. Purchase your copy from Amazon.com.

If you are interested in a preview of the book, download Chapter 4 entitled, Enterprise Service Buses. This chapter was written by Hub Vandervoort, CTO, Progress Software. In this chapter, Hub talks about the ESB, a messaging-based communications backbone for SOA infrastructure. He also shares his collective experience of working with over 300 ESB end-users, and summarizes the styles and applications of ESB technology. More importantly, the secret to ESB success—captured in the synopsis he has dubbed the "seven points of mediation"—is revealed to illuminate why all ESBs are not created equal, and provides practical advice about how to be successful using an ESB in your SOA initiatives.

Download Chapter 4 of SOA: Getting It Right >
Podcast Seven Points of Mediation >

Who’s in Charge of the Architecture? You Are.

In Replacing Large Applications – Who’s in Charge?, Kathy Harris at Gartner writes:

"Most of the organizations have no real architectural vision for their system. The result is that they are essentially allowing the vendor to establish their architecture. This may be ok in the long run, but for many organizations, it is a de facto decision rather than an active choice."

While many vendors have the expertise to make the right recommendations for their portion of a solution, things become much more complicated when you start integrating their applications with others. Complexity increases exponentially when you consider the changes being made by other departments, in other locations, and by your partners.

The complete picture can be daunting. Great enterprise architects understand that you don’t need an exact schematic of how infrastructure will evolve over the lifetime of the business. Rather, you need to take proactive steps to incorporate flexibility into your architecture. And the best way to do this is to partner with vendors that adopt Open Integration principles to ensure that your architecture can grow to support the business as it evolves.

If you’re concerned that your vendor is prescribing your architecture for you, consider these three Open Integration requirements:

1. Make sure your vendor supports open standards. This means that they support de facto standards as well as those specific to your industry. More importantly, they should also be involved in defining emerging standards, to ensure future compatibility. If your vendor supports open standards, you have a better chance of adapting to change.
2. Develop an open architecture. Look for solutions that are modular in nature and allow you to mix and match functionality to meet your needs, while only paying for what you use.
3. If possible, leverage open source. By using open source, you reduce your price of admission to mission-critical infrastructure. More importantly, you have direct access to the source code and those who wrote it.

Organizations need to take an active role in defining architecture. All three Open Integration principles allow you to actively choose what’s right for your business, instead of being at the mercy of others and hoping it will all work out in the long run.

Is the Travel Industry Out of Touch with Our Changing Business Climate?

Did you knw that the travel industry loses $11.5 million per year through failed transactions?

Last week Progress Software released the results of a survey commissioned with an independent specialist technology market research company, Vanson Bourne Research. The researchers surveyed 149 global travel businesses and found that 67% of respondents have noticed their transaction failures soar by almost a third – even though their transactions have only increased by 12%.

Dan Foody, VP of Products at Progress Software, sites: “This research clearly highlights the far-reaching effects of transaction failures within the travel industry. The trend of increasing IT complexity is further compounding revenue loss through increased lost transactions, customer churn and inefficient use of IT resources with 97% of respondents concluding that transaction failures were increasing operational costs."

 “Companies should consider introducing a more streamlined approach to monitoring transaction flows across their IT environments, delivering the ability to respond to changing conditions and customer interactions as they occur. This will enable business leaders to capitalize on opportunities, drive efficiencies and reduce the risk of impacting customer experience. The more responsive approach will provide visibility and increase understanding of the impact IT failures have on their own IT services and their customers,”

Get your copy of the report! The report IT Impact on Travel & Leisure Industry Reservation Management, examines the causes and consequences of this situation. Read this report and learn about some of the underlying causes for the increase in transaction failures, what the consequences are, and the limitations of most traditional monitoring and management systems.

Share your experiences and thoughts by posting a Comment to this entry.

A New Renaissance for ODBMS? Part 3

I’ve been posting comments from an email discussion I had with Luis Ramos one of our Object Databases experts here at Progress (previous posts: Part 1, Part 2). In our last exchange, Luis commented on the reasons to switch from RDBMS to ODBMS, the market, and where to get more information.

In our final post for this special series we cover the question, Are ODBMS coming back? I put the question to our team of object database experts: Luis Ramos, Jeff Wagner and Adrian Marriott.

Me: Are we experiencing a new renaissance for ODBMS?

Adrian Marriott, Principal Consultant Progress Software: There are many technical reasons why OODBMS are attractive and that's why they never went away. They've been used continuously by a huge number of programs world-wide for over two decades. There's no "renaissance."

In terms of the next 'big thing' that will drive adoption and faster growth for object databases, I think it's the appearance of solid-state drives. Hard-discs with spinning platters are using archaic technology, several orders of magnitude slower than memory access and this encourages developers to think of fetching data 'out there' from disc. As solid state drives become faster - particularly for write - the idea of a seamless persistent object model that survives program invocations and can just be used directly and transparently from disc will become more compelling.

Jeff Wagner, Product Manager ObjectStore: Adrian's right, Progress ObjectStore revenues have continued to beat our expectations year in and year out. We continue to see adoption of object database technologies as people search for ways to increase performance, lower their TCO and shorten their time-to-market. Object databases can increase performance over RDBMS’s because most products offer an in-memory cache component. This can result in 3 or more orders of magnitude increase in performance. TCO is much lower since developers are not burdened with object-to-relational mapping code. This code can account for over 60% of the total code for an application persisting data in an RDBMS. This also means that development time is reduced, lowering overall development costs and getting products to market faster. Maintenance costs are reduced once an object database is deployed since it typically requires little/no administration.

Luis Ramos, Principal Consultant Progress Software: I concur with Adrian's and Jeff's points. These are excellent arguments. To that I would add that object-databases have a fundamentally more scalable architecture because the queries and processing are done at the client side, where the caches reside, rather than the server side. To scale an RDBMS, you would need to get a more powerful and expensive box. To scale an OODBMS, you can leverage as much off the shelf hardware as you need to host the clients and their caches.

Regarding solid-state drives, I am sure proponents of Relational databases would make a similar argument that using solid-state drives would boost the performance of their servers significantly. However, relational joins will still be expensive because its time complexity is fundamentally an "n squared" algorithm compared to following references which fundamentally has a constant time complexity. Add to that the cost of OR mapping if the data needs to be cached at the client.

Do you agree with Adrian, Jeff and Luis? Do you disagree? Feel free to comment in our comments section!

Recap: Putting the Information Framework to Work for You

Nearly every time John Wilmes and John Reilly get together for a webinar, not only do they deliver valuable insight and content, they spend substantial amounts of time fielding questions from the audience. Personally, I find the live Q&A to be one of the best parts of the webinars.

The webinar we hosted on the 21st, Putting the Information Framework (SID) to Work for You, was true to form. John Wilmes and John Reilly spent nearly 30 minutes after the webinar finished answering questions.

What I’ve noticed over the past several webinars is that the tenor of the questions are shifting.  We’re seeing a shift from educational/informational questions towards questions about actual implementation.  During last week's webinar we received several questions about designing enterprise information models, developing business cases, and even the existence of pro-forma RFI/RFPs.

The community is coming to the realization that managing and maintaining information exchange is clearly within the remit of integration—leave it unaddressed at your peril.  Your responses to the TM Forum webinar survey bears this out. For example, in response to the question “What percentage of your projects are spent on integrating data between your applications and services?” - 70% of you told us that dealing with data integration consumed more than 40% of the development effort. And dealing with data integration is not easy work. 

When we asked about the most difficult challenge, your top three responses were:

1. Keeping documentation up to date (19%)
2. Not enough resources to keep up with development (30%)
3. Managing change (43%)

These challenges are interrelated and what’s more interesting is that all three challenges can be directly addressed by the use of exchange modeling technology for data interoperability.  The inability to keep documentation up to date makes it difficult to manage change.  When changes comes in, your teams are forced to manually ascertain the impact and test everything to ensure success in production.  For the small selection of you who are starting from a green field addressing change will not be an issue... the first time. However, with every subsequent change caused by new developments, new requirements, new business initiatives... you will need the means to understand and manage the effects in your environment. If you don’t have that capability, you don’t have agility.

So given the resource constrained world, the choice is etiher: to continue to muddle along with an allocation of resources heavily skewed towards these maintenance tasks; or to raise the effectiveness of your teams in maintenance and use those savings to address more mission critical tasks. Perhaps more succinctly: do more with less by doing what you must do more efficiently.  

This is the conceptual basis on which we have created many successful business cases.  If you’d like to learn more, or if you would like assistance in justifying the need for exchange modeling technology and framework implementation, please do not hesitate to contact us.

A New Renaissance for ODBMS? Part 2

I’ve been posting comments from an email discussion I had with Luis Ramos, one of our object database experts here at Progress Software. In our last exchange, Luis commented on the ideal use cases for object databases. Today Luis answers a few questions about switching from RDBMS to ODBMS, the market, and where to get more information.

Me: Why would someone want to switch from RDBMS to ODBMS?

Luis Ramos: There are two reasons - Performance and TCO.

There is significant overhead in accessing object-oriented data (in Java, C++, or C#) from a relational database.

If "complex" or object oriented data is stored in an object-oriented database, then the data is stored in the same format as the object. It is very straight-forward. But if you try to store the same object data in a relational database, then you would need to decompose the data into rows and columns. This procedure can be quite difficult and imposes quite a bit of overhead specially if the corresponding object model is complex in terms if its inheritance hierarchy and interconnections between objects.

There will be a magnitude-times improvement in query performance using an object database. In a relational database, queries are performed on the server-side and results are brought over to the client and materialized. In an object-oriented database, such as ObjectStore, the objects and indexes are cached at the client (via its cache-forward architecture). Consequently, performing a query is very efficient since it is carried out at the client by simply de-referencing pointers or references of the corresponding index structures.

For those interested in cost, using an object-oriented database instead of a relational database can reduce the TCO significantly because it avoids the development and maintenance of object relational mapping (OR mapping) code.

Using a relational database from an application that is implemented using an object-oriented language such as Java or C++ requires the use of OR Mapping technology to transform data between objects and relational tables. The amount of code and effort to undertake (and maintain!) this transformation can be significant. With an object-oriented database, there is no OR mapping code thereby reducing the TCO.

Me: Why is the market so small?

Luis Ramos: Relational databases such as Oracle and SQL Server are very well known in the industry and are common place. People generally tend to use what they know, and so developers usually start out using relational databases by default. Relational databases can be made to work and developers would go through all sorts of contortions writing a significant amount of OR mapping code to force complex data into two-dimensional tables and live with up to 100x slower performance. But then pain is relative. If you have not heard of object-oriented databases and these benefits, how would you know better? Object database vendors need to do a better job at marketing their products.

Very few people really know about them.

Me: Where can I learn more?

Luis Ramos: You can always email me if you have questions about ODBMS or log a question here on the blog in the comments section.  My email is lramos@progress.com.

There’s also a wide range of information on the ObjectStore and PSE Pro websites. There are a couple of white papers that are worth reading: here’s one for Real-Time Data Caching, and another a white paper on Coding Java Applications on Cache Forward Architecture.

Stay tuned! Our next (and final) post in this special series on object databases is where we will talk more about the question "A New Renaissance for ODBMS?"

Why on-premise SOA cannot offer cloud pricing

David Linthicum suggests that SOA infrastructure vendors must switch to a cloud pricing model and get paid only on delivering value, further taunting the vendors to put their money behind their products. Interesting proposition... But I don't believe it will work. And the comparison to cloud or SaaS may not even be valid. For SaaS, its easy to identify the value delivered - as that of a business solution being made available as a service. For cloud, the SLAs are at the platform level and can only ensure availability - they cannot ensure business value. On-premise SOA is even worse because the operating platform is not even in the SOA vendors control, so guaranteeing even the availability at the same level as cloud is not even possible. At best the vendor can assure that the SOA platform will not fail, but they definitely cannot guarantee against failure from the hardware, OS or other layers in the solution stack that reside with or on the on-premise SOA.

The business value that David refers to is way beyond just platform availability, and that surely cannot be assured by any single component in the overall solution stack - unlike in the SaaS model where the SaaS provider controls the complete solution stack and can provide the guarantees.

A New Renaissance for ODBMS? Part 1

Recently a team of our object database experts - Adrian Marriott and Luis Ramos - attended the 2009 International Conference on Object Databases. Not only did they present on design patterns and discuss the resurgence of object oriented databases, but Adrian won an award (and a netbook!).

Adrian Marriott was the first place winner of the award for best Common Persistent Model Patterns for Performance and/or Scalability Optimization. He beat out 25 other compelling patterns with his Query Visitor pattern which allows one "to define new result set formats without changing the underlying persistent object model."

Roberto V. Zicari, Editor of ODBMS.ORG, said of Adrian’s pattern...

"It is common practice that some database designers treat an Object Database (ODB) like a Relational Database (RDB). That is they are very query intensive rather than model intensive in their design. Some designers start with a “relational” model, and then adjust it to a model that is more "ODB-oriented", or closer to their problem domain, in order to get better results. This  task is difficult.  Marriott`s pattern, Query Visitor, can speed up the database  development process by providing a tested, proven development paradigm."

Luis Ramos contributed to the lively discussion in one of the more intriguing panel discussions at the conference: A New Renaissance for ODBMSs?  As part of our post-ICOODB2009 coverage, I asked Dr. Ramos to share some thoughts about object databases and their use.

Me: What are some use cases that benefit from an Object Oriented Database?

Luis Ramos: There's three main uses cases: complex (multi-dimensional) data, transactional caching, cloud-databases—and given today’s SaaS-world, you can see why ODBMSs are becoming more and more relevant.

By complex, multi-dimensional data, I mean data that is hard to render into rows and columns.  For an easy example of non-complex information consider the current roster for the Boston Red Sox Baseball team; the roster lists each player and their name, number, position and statistics–it looks like a rectangle with two dimensions.  This formulation makes it easy for me to ask the question – what is the current batting average of the 1^st baseman for the Red Sox?

But consider a slightly different question – how are individuals in a family tree related?  Consider Barack Obama’s family tree. If you visualize the data, it is not a matrix at all. It looks more like a tree of nodes. This would be very natural to store as objects with links to other objects. Consider asking a question like, is George W. Bush related to Barack Obama? Answering this question is quite easy in an object database. You simply follow pointers from the node representing Barack Obama and see if you can reach George W. Bush (and apparently they’re 11^th cousins). Following pointers or de-referencing references is certainly a lot more efficient than doing an arbitrary number of joins.

On the transactional caching use case. Our clients have selected object databases over other data caching technologies such as Memcache and Tangosol for a these important reasons: transactional access, durability, and automatic cache replacement. With object databases, there’s transactional access to the cache thereby preserving the data integrity of the cache – you don’t find this in many caching solutions.  Also, the cache is durable. If the application is terminated intentionally or otherwise, recreating the cache is fast and efficient because it is being populated from an object database.  In this case, there is no overhead for running SQL queries to find to the objects to bring into the cache and no overhead to transform between relational data and objects. Third, ObjectStore automatically manages the cache if the amount of data being accessed in the cache exceeds the amount of memory.

On cloud-databases, object-oriented databases are a more natural fit for persisting cloud data, which is inherently tuple-based, for a number of reasons. First, storing tuple-based data, whose values are arbitrary (strings, integers, double, boolean, etc) and that are automatically indexed could be challenging to do in a relational database. This type of problem is reminiscent of the issues related to formulating a relational model to store an object model that has an inheritance hierarchy. Second, scaling a relational database is not easy. The usual practice to scale a relational database in order to support more load is to use more powerful hardware. In an object-oriented database such as ObjectStore, it is easier and simpler. Since the queries are performed at the client (the cloud node or service), scaling the database can be accomplished by simply launching more services.

Stay tuned, the next post will cover additional Dr. Ramos’ comments about RDBMS to ODBMS, the market and where to get more information.

REST maybe part of the answer

In the last few weeks/months I have got more immersed in the world of REST and what it means to build and have a REST application.  While at first glance it may seem that REST is a lighter weight version of a web-service implementation, it is by no means the complete answer and I do not believe REST is simply a replacement of Web Services because of how they operate is different.

When developing Object applications, in whatever language, you hit a reality that if your application is successful, then it will grow beyond the confines of the machine/JMV that it runs on, and as such you have to refer to objects that are beyond a single instance of the application's memory space.  This can be done using a number of different technologies such as object databases that can map the same object into different application instances, but the locks are maintained by the central server so consistency of the objects are maintained, even if the object is distributed in 100s of applications at a time.

Now technologies such as IIOP (Corba), RMI, DCOM and Web Services have been used to access objects and invoke operations on the instances to retrieve data.  This essentially solves the long-wire problem of accessing an object and then returning an instance of the object, the issue here being is that typically a representation of the object that you want is returned, i.e. using Java serialization or XML and the reference to the actual instance is lost, unless you have a known object identity that you are working with.

REST adds the concept of the pointer to the SOA infrastructure in a way that any business object can be referenced by a URL and thus referenced anywhere within the internet/cloud or a single program space. This is something that is not readily available in Web Services and now makes any RESTful resource available to anyone (given the correct permissions).  Now this sparks a whole debate on to how to identity a particular object and to detect duplicates etc, this is beyond the scope of this blog but will have the same techniques that are used in the object-oriented world, but now working at the cloud scale.

So now that REST gives us some new capabilities, how does this change the current implementations that you are working on?  Interestingly, I think many of the challenges existing in the Web Services world still need to be addressed in the REST world, I just believe that REST is probably better able to address some of the immediate ones and in fact just like every Progress customer has another application to connect to Web Services will always be required.

The other cool thing about REST is that I can typically use the services directly from an HTML page on a browser, something that was impossible for Web Services because HTML and WSDL are not compatible.  However, there is still the need to mediate REST in the same way that applied to Web-Services, for instance to be able to turn a switch to expose the same business entity as REST and Web-Service and in fact any possible protocol.  Allowing external access to your business objects will require caching and a level of indirection to provide for maintenance, performance, etc., and the virtualization of the underlying technology to ensure that an object is not a manifestation of the technology that stores it, i.e. database tables and rows are not part of the object serialization.  Security, Quality of Service (QoS) and Quality of Protection (QoP) are observed along with SLA commitments and the ability to monetize the object and charge accordingly.  This last point maybe very interesting as it would allow organizations to potentially charge for access to their data for different reasons and allow IT to start to become a profit-center rather than just a cost-center - something that I think people will look at as they invest in the cloud.

Now I think that the promise of REST is very interesting, in more ways than Web Services was, but whether the ideal is purely based around REST or a hybrid of many technologies is going to depend on the goal of what is to be achieved. In any event, it is very exciting.

SaaS, PaaS... why not SOAaaS (SOA as a Service)?

A busy day in the rapidly converging SOA and Cloud worlds. Oracle talks about plans for the cloud, retracting from the skepticism expressed some months back. Intuit announces a PaaS platform. Another SOA infrastructure vendor dabbles with SOA on the cloud - striking dichotomy here. On one hand we are still trying to figure out how exactly to make SOA projects successful. And on the other, we are talking about SOA in the Cloud and PaaS platforms for SOA. Even so, I find it a natural progression.

PaaS, and therefrom the SOA impact, follows the success of SaaS. Which itself was the best thing to happen to ISVs in recent times. A combination of emerging application models, web based software UI approaches, new cloud platforms, and a very wide acceptance of externally hosted software solutions - less of technology and more of mindset. Now it is only a logical extension of the SaaS paradigm that now one will expect to build custom solutions on the web (PaaS). Or host solutions directly on the web based infrastructure (Cloud). And the moment there are applications, SOA cannot be far behind. The nature of the cloud beast is also such that the significance of SOA and distributed management/governance becomes even more critical given the rather loosely coupled and a less-controlled computing environment.

So while enterprises may be OK with having solutions hosted externally on the web/cloud, they may still want the integrated enterprise where these external solutions are seamlessly available in the enterprise integration platform (SOA) and also in the enterprise distributed management and governance platform. We can probably extend these to bring the external apps into the prevailing GRC and BAM framework in the enterprise, so it is only natural that SOA becomes a first class consideration when SaaS/PaaS/Cloud are in the picture.

Not far will be the support for SOA as a primary attribute of cloud platforms. Right off the bat one will have the ability to build and host applications over the web, with the default web based UI models and a very integration ready platform - both for consuming/orchestrating services over the web, and also to expose new services (off this application) over the web.

Now... taking this a bit further, one could look at explicit platforms just FOR integration and SOA!  Even now there are BPM vendors like Cordys that are providing a web based orchestration platforms (PaaS). These can easily be extended to offer a complete services and integration-application platform on the cloud. Only, we need to figure out the use cases where one needs integration off the cloud. Needing SOA as a Service (SOAaaS).

Astounding Statistics Regarding Order Fallout & Revenue Leakage

Sometimes it’s really nice working for a profitable and established public company. With the launch of Actional 8, we’ve put some discipline on our focus around Business Transaction Assurance (BTA). We’ve rolled out lots of internal materials to train our field to understand the “problem” from a customer perspective, so that they can engage in conversations around the problem, rather than “pitch products.”

In any case, I’m reading through some research around order fallout (revenue leakage) Progress just sponsored that was completed by Vanson Bourne. Vanson Bourne carried out over 200 company interviews (these, to my knowledge, were not all Progress customers). The companies are in the USA, Europe, and Asia/Pacific and all had a minimum revenue of $200M. The study was not limited to the Telecommunications industry, but does call out specifics about both the Telecommunications and Hospitality industries because of their heavy reliance on consumer transactions.

Even realizing that we sponsored the research and so it is perhaps somewhat suspect, I can’t help but be amazed at the raw statistics the report showed. Some examples:

• 86% of telecommunications companies believe that order fallout causes delayed or lost revenues; 64% believe it causes customer churn also
• 98% Say that order fallout “definitely” or “probably” causes increased operational costs
• 75% say that having to deal with increased order failures increases demand on resources (warning: circular reference here... increased demand on resources causes order fallout to worsen, and so on...)
• Transaction volumes increased in 2008 by 16% on average, yet transaction failures increased 35% on average over the same period of time
• 86% of companies reporting high levels of IT complexity admitted to an increase of transaction failures; those with the most complex environments had their transaction failures increase by almost half in 2008
• On average, these companies have 11 full-time employees manually finding/fixing transactions, and it takes approximately 2 hours on average to fix them; Telcos have teams of 18 people, on average.

There are phenomenal implications to these numbers. The inefficiency is outstanding. And, there is plenty more where that came from!

I’m a student of life, and in particular of software companies. I’m always amazed when companies spend huge efforts to win new business, only to discard the relationship once the deal is won. Sure, no one does this purposely, but... through compensation and other more subtle motivations, there aren’t too many people in companies whose job it is to deal with happy customers without problems to keep them happy. I find this curious, because “relationship revenue” has a much lower cost of sale than “RFP revenue” and puts much less burden on every aspect of the company.

I digress a little, only to make the point that the companies in this survey are LOSING MONEY THEY HAVE ALREADY EARNED. Therefore, these lost transactions represent the most profitable ones they process! It’s like there’s a queue for service, with someone at the head saying “I’ll take yours, I’ll take yours, I’ll take yours, Nope - you, drop that on the floor, but don’t worry, keep our service.”

The report goes on to talk about some of the interesting “management” facts that usually are part of press releases about “new software versions” but often don’t actually make it into the software release (hasn't anyone developed a compiler that can compile press releases yet... it's a product some companies really need). It seems like there are a lot of difficulties with the software meant to solve these problems. Such as:

• The performance of the apps being monitored is negatively impacted (>60% companies reported this to be the case)
• The management system requires additional servers (~48%)
• Management systems need additional people to run them  (~47%)
• Can’t run all the management/monitoring features in production due to their resource requirements (~42%)
• They use up too many CPUs (~38%)
• 44% of companies don’t manage the end-to-end order process because of the above reasons!

All that said, I still have a personal favorite...

When asked “How often do you lose orders even though these systems say everything is OK?”, fully 35% of companies admit this happens “often” or “frequently.” The number soars to 56% of the very large companies (revenue >$2B) and 69% in companies that have the highest IT complexity scores. (78% of companies self-classify themselves into the highest complexity ratings)

It goes without saying that I believe Actional solves the problems in the first list, without introducing the ones in the second.

Who cares what I think though when you have Forrester studying two of our IN PRODUCTION customers, and finding the same results:

• Staff required to fix root-cause messaging problems was reduced by 85%
• Service quality reports are available in near real time, and created automatically; in the past they were difficult to prepare and were not readily available
• Of the two companies studies (admittedly, not a large sample!), one had a payback of <12 months, the other around 20 months

I’ve blogged before about the importance of the scalability and performance of the management system, without which features don’t really matter. This report validates my earlier post, and calls out specifically that many management features can’t be used in production because of the impact on performance. I wonder if the vendors of these management systems didn’t oversell their products... I wish they had asked that question in the survey! (And, when looking for that link, I found another post that was also mentioned in the report... the importance of actually knowing you have a problem.)

Thank you for reading this far. Email me if you like a copy of the Vanson Bourne report and the Forrester case studies. (These case studies are anonymous, but all research was completed by and validated by Forrester Research).

Welcoming Scottish Widows to the Family

For the record, I think I've had three hours of horizontal sleep in the last 48 hours or so, and that was on a hard wood floor - though I did have a pillow and blanket.

That said, the title of this post makes perfect sense to me... See, we've just announced (this past Tuesday) a new customer relationship for Progress - Scottish Widows. Scottish Widows is a part of Lloyds TSB Group, and is a well known provider in the life, pensions and investment industry in the UK.

The press release is typical - you can read it - but it essentially says:

Scottish Widows performed a 20 month selection of an integration platform and selected Progress' offerings because they perform better than other offerings. Scottish Widows hope to

1. launch new products faster
2. integrate easier (probably why they hope to launch new products faster)
3. lower their IT costs overall

In fact, it's this last point that's very interesting. I was only involved with the deal on the periphery but from the beginning the deployment team of both Scottish Widows and Progress are integrating metrics into determining the success of the deployment. Among other things, they will be measuring their cost savings by using Progress Software. From an Actional perspective, this means diagnosing issues faster and reducing support costs, as well as improving customer (and end-user) satisfaction.

We've done a similar case study with a Financial Services institution, validated by Forrester Research. Email me if you'd like a copy of the case study or watch the on-demand webinar with Forrester that explains the results (as a special perk to readers... you don't have to register). The case study provides real numbers about the Total Economic Impact (TEI) and measurable results achieved by this institution.

The press release also mentions that Sonic ESB and DataDirect Shadow (mainframe integration) were also selected. Congratulations to those teams, they're great products. We're starting to see some really interesting dynamics driven by customers selecting multiple products from the Progress SOA infrastructure portfolio, but more on that after I've gotten some more sleep.

Embedded Not-so-lightweight ESBs

David Linthicum touched upon a very prevalent problem of technology not living up to its expectations set during the hype curve! He says, "With cloud computing, SOA, or whatever we come up with next, we still need to figure out core issues within our enterprises that no set of technologies or emerging trends can solve." In the service-oriented architecture (SOA) case, we all know that the widely professed use cases of systems "automatically" looking up service registries on the web/intranet, locating the required services, getting the interfaces, picking the right one, and actually making a service request is just a pipe dream! The reality is nowhere close! It is difficult to even find enterprise wide service buses, much less the dynamic on-the-fly service-look up-and-use!

I recently came across a scenario where even a large international bank has SOA infrastructure solving regional problems - and not necessarily as a corporate backbone! I was at this tech discussion with the regional technology head of a large bank, exploring how Apama (Progress' Complex Event Processing (CEP) engine) fits into a biz requirement they have. Specifically, he wanted to understand how Apama works and how it is a good fit for their biz problem. They were trying to build a business performance tracking solution (a la GRC) that involves tracking key biz activities, ensure due process is followed, and then proactively detect potential service-level agreement (SLA) failures so they can be averted. A classic use case for Apama!

Now, when we started to see where the business events information came from and how they could be "wired" to emit the required events into Apama, we suggested that they could just tie into their enterprise service bus (ESB)—we knew they had standardized on one. As we dwell further, it turns out they are using an ESB to solve departmental integration requirements, and there are multiple instances and no central ESB that can help integrate these "departmental integrations".

Now that was a revelation. (I am not complaining because suddenly our opportunity now included Sonic ESB, along with Apama). While we have been reading (and I myself have been blogging and talking) about how SOA failed to live up to its expectations, I still expected at least a large number of banks to have standardized backbones. But not to be! I guess, like in many large enterprises, even here SOA adoption has started purely to solve departmental integration problems—more as localized islands of "integration". And its nothing like the enterprise-wide integration platforms or service bus that we have so much noise about since 2004. This reality check at this bank was a clear validation of this failure! It seemed clear that the technology promise was something, and the actual on the ground realization was something totally different!

Come to think of it, it does seem like a good model for use cases such as BAM or GRC - to have a complete platform that in addition to the CEP engine and dashboard, it also includes a "local/embedded" ESB. Now, embedded is not about being lightweight or smallas it is about being available out of the box when you get the platform. In this case, the ESB is available along with CEP, well integrated - so no surprises as you start using it. And the purpose is to "wire" exchanges from existing ESBs or applications in the enterprise to the CEP environment.

Not a bad idea at all! Embedded ESBs!

Cloud Computing... It Depends on Who You Ask

What exactly is cloud computing? Lately, this is one of the most often repeated question in any discussion about the Cloud. And the answer depends on who you ask! (Personally, I think that if there are not at least three different popular definitions of a new technology, then the technology is not in its "hype" phase of the hype cycle :-) ).

Cloud is often confused to be just one of:

• Utility Computing - The on-demand resources with dynamic provision offered by virtualization platforms such as VMware, EC2 etc.
• Grid computing - Loosely coupled discreet computing nodes, that come together to form a larger computing-grid.
• SaaS - Just the notion of a software abstracted on the web is cloud to some.
• PaaS - To some, cloud is if one were to "build" the solution and then deploy it on a virtualized platform on the web.
• User centric cloud - RIA desktops that provide seamless and integrated access to information and services on the web, while still retaining a "sane" visual front to the user.
• Even Wikipedia defines it as: The majority of cloud computing infrastructure as of 2009 consists of reliable services delivered through data centers and built on servers with different levels of virtualizationtechnologies. The services are accessible anywhere in the world, with The Cloud appearing as a single point of access for all the computing needs of consumers.
• And, I am sure there are more!

Syscon attempted to reconcile the definitions by getting no less than twenty one (!!) industry experts to define cloud. Their definitions had wide ranging defining attributes: elastic, virtualized, services over the web, multi-tenanted, pay-as-you-use, massively scalable technology enabled services, SaaS is the consumer-face of cloud computing, distributed utility grid (wow!), on-demand resources with APIs, and more!

What I see missing though is the end enterprise view. It is this view that brings all of the above (and more) together. As I see it, this is "cloud computing". A fairly abstract IT enterprise landscape. Where the biz requirement is real—with its solution made available over the web, with simple web based user access, with guaranteed SLAs on service availability and performance, with seamless integration to other information sources/services over the landscape. The exact location, platform, configuration, size and such are all abstracted from the enterprise by the solution provider. Further, the solution provider also takes care of maintaining the application (enhancements/bug fixes), managing the production environment, and managing any backend integration requirements that the client-enterprise may have - offering a complete "managed service". The enterprise needs to focus just on the biz at hand, and in defining the IT solution that is needed. The solution-service providers takes care of the rest. All of them coming together to form the enterprise "cloud" - accessed over the web, managed over the web and probably even orchestrated over the web.

The enterprise Cloud will also evolve over time. When there is just one ISV making a solution available in the above managed model, that is SaaS. If the same ISV also "builds" it on the web before making it available on the web, then that is PaaS. If the ISV just uses storage on the web, then that is cloud storage. If the same solution is deployed within the enterprise network on a virtualized platform such as VMware, then that is virtualization. If the solution is distributed across discreet computing "nodes" on the web, then that will be the computing grids. Now fast-forward a few years... If the IT landscape in an enterprise architecture is a combination of all of the above types of solution environments, then there is an opportunity for a loose "biz grid" that can bring all of these together, which in many ways is already prevalent today - the SOA infrastructure! SOA is all about discreet services that are location agnostic, with standards based description and easy access. Today's SOA environments are very very vendor dependent (that is assuming the emphasis is on the reliability and performance). Soon we may see this a reality. A vendor neutral SOA environment that will bring together all the disparate IT solutions in the enterprise, both within and on the cloud. Likewise, Web 2.0 will define the user experience that brings all of the same solutions in a seamless integrated and very-functional UI.

Below is a visual of what I'm thinking... use Comments to let me know what you think.

Goodbye SOA, we hardly knew you.

Anne Thomas Manes of Burton Group posted how SOA is Dead; Long Live Services.  This one is too juicy not to comment on!

Firstly, I totally agree with Anne.  I think "SOA" is definitely dead in 2009.  And I also agree that the need for services will grow.  But, we still wrap too many things into the same terms.

At a simple level, I'll break it into four points, I'd like to talk about where I think things are going:

Service-oriented applications.  BPM, cloud computing, SaaS, PaaS, composites, mashups. All of these are doing well and will continue to do well even in the 2009 economy.  Why?  1) They can be leveraged as needed; 2) They can be done fast; and 3) Using them typically provides measurable benefit from day one (if it doesn't, then you should do something else shouldn't you!).  It's precisely the laser focus solution aspect of these that is their greatest strength.  Get in and get out while solving a problem that someone really cares about.

Service-oriented architecture (SOA).  This is where you look from a higher level above all your service-oriented applications to create a cohesive architecture that all of these follow.  Dead in 2009.  Why?  Because approaching this from an architectural perspective will always fail.  Is it a noble goal?  Sure.  But without other changes in your organization (see below), it just can't be successful.  Said another way, before you can start on a successful SOA infrastructure initiative you must finish the items below.

Service Delivery.  By service I don't mean what you're probably thinking (a thing you create).  I mean service in the most pure sense: as in "what value do you deliver to others" (with the follow up of "how do I continue to improve the value I deliver to others").  This requires a transformation for most IT organizations... and most haven't even started yet. To be fair, there are organizations that have figured this out (take S&P for example: their whole business is service delivery, so of course they understand it) - but the vast majority don't fit into this category - and if you do not pass go, you can't collect your $200: So this is the next step for most IT organizations that want to transform.  Think: "How do I create my first true service?" Here's a hint: you can't do it by creating a project, task force or cross-functional team (more on this in a subsequent post since this is a richer topic than I can cover here).

Service-oriented IT.  Once you've mastered service delivery on a small scale, then you can tackle it on a larger scale: turning IT from being project and application oriented to being service oriented.  Warning: this is a major transformation that can't be done without the CIO leading the charge.  Sorry, but no enterprise architect can make this happen.  And it can't happen until you've figured out service delivery.  This is not a 2009 project for most organizations.  It is a 2010+ transformation.

Most organizations that went down the SOA route tried to do SOA without first figuring out what it means to deliver a service, and without realizing they needed to get this right (and begin the service-oriented IT transformation) before they could never be successful at service-oriented architecture.

Service-oriented architecture is a technical and architectural transformation.  But it can't exist without first focusing on the organizational and cultural transformation of service delivery and service-oriented IT.

This is why SOA is dead in 2009 and why services will live on. But who knows... maybe once an organization figures out service-oriented IT, SOA will be resurrected - but don't wait up.

Do we really need this complexity?

My first post on the Progress SOA Blog! This gives me jitters... All the writing till now was somehow different. There was a carefree attitude to that writing. When I wrote my book (nearly 40% of the book on SOA approach to Integration), or when I write my personal blog posts, I never had to worry about the impact. Now as informal as this may be, at the end of the day it does have a non-personal tone. Even so, I am excited about this. :-)

My views essentially are reflections on the software services and SI ecosystem and what/how they use technologies in the integration solutions space. Lately, I have been also quite caught by the SaaS/PaaS promise in the SOA context - even if I am quite skeptical about the hyped up value.

Recently, I was speaking on this topic at the IndicThreads conference on Java at Pune. I included a slide on the complexity of SOA. While in my flow I was using it to talk about some of the SOA enablers in our SOA portfolio in simplifying this complexity, the presentation and some following questions triggered a thought... How much of this does the industry actually need? Aren’t these more of a fringe scenario that needs all of this? Isn’t the REST vs. SOAP argument a good case in point? 80+% of the world needs only basic remote service mechanism. While REST more than suffices here, this basic need from the world is but just a small part of SOAP! So, who needs the rest?

The simplicity of REST is stark - at least when compared to the extremely unwieldy SOAP and its numerous accompanying specifications. And it is further compounded by all the WS* standards. Making things worse are frameworks for frameworks such as WS-Policy. WS-policy is a framework that layers on SOAP. And there are other standards that use WS-Policy as the basis for definition. In effect, we have XML over HTTP, on which is defined the SOAP standard, on SOA is the WS-Policy. And on WS-Policy are standards such as WS-S & WS-T. Phew!

Now compare this with REST - that actually has no other specification other than HTTP itself! Granted, this seriously limits what we can do with REST. But lets accept it - the world needs mostly ONLY the likes of just REST and only a very small fraction needs the multiple layers of specifications such as SOAP! Even in the SOAP land this is very well captured in how AXIS evolved - trying to keep pace with the rather haphazard offshoots of SOAP. And now there is CXF whose basic premise is to simplify SOAP and Web Services. From the word go, it has embraced the simplicity and is also trying to be more about services and less about SOAP. In the spirit of the latter, CXF also supports REST right off the bat.

I see 2009 as year where this simplicity is bound to get consolidated. The more simpler and easier the SOA framework becomes, the more it will get accepted. FUSE open sources SOA framework will be one very serious contender in this space. Functionally as complete as any other SOA solution, yet lean and mean with a good set of tools. Watch out for more in the months to come.

Meaningful Measurements

At our annual sales kickoff in Miami last week, I heard a story about one of the Regional Sales Managers that put a smile on my face, and it made me like Progress a little more. When asked by the CFO how he did last year, the regional sales manager responded "six cents".

Six cents. Huh?

Well, apparently there is a culture here at Progress that you look at regional profitability, and then each sales manager knows how their deals affect the company's bottom line. Being a numbers geek, and one who likes to measure things properly, I think it's cool.

SOA What? Well, I also heard a story about a new Actional customer who had over 98% up-time for their IT systems last year - a number they were proud to report. (Realizing, of course, that each industry has its own benchmark for up-time.)

They were proud to report it but it was meaningless to do so...

Why? Because as it turned out, while they had this brilliant up-time of their underlying infrastructure, something was preventing their largest customer from receiving any information for over three weeks. Do the math, 52 weeks in a year - down for 3, that's almost 6% downtime.

Where was the discrepancy?

The IT team was not tied into the business and was assuring system up time but NOT assuring their business transactions. So now, in order to gain credibility with the business and to make more insightful technology decisions in support of the business, the team chose Progress Actional as the management solution for their SOA and distributed applications (links to good article on what this is about by Anne Thomas Manes). Hopefully, they'll do better next year.

Have you ever had a situation, even in your personal life, dealing with a company's technology where they say all the systems are up but still, it's not working? Create a comment here and share your stories...

Data Models and Message Specifications

Enterprises of all types see increasing value in standards-based development and integration, and they often look to the TM Forum for deep and consistent models of business process, data, applications and system architecture. Moving from analysis through design to implementation, a portion of a project’s focus typically shifts from process and data models to message specifications.

Like other standards organizations, the TM Forum is well known for both data models and message specifications. These standards often overlap and interact, but they fill distinct roles, so it is interesting to trace the paths of their evolution and likely future. And integrators and enterprises reveal their integration philosophies by the way they employ them.

Data models have evolved from both "top-down" and "bottom-upv perspectives. The principal TM Forum information and data model, the Information Framework (SID), shares the top-down, domain-oriented perspectives of the other TM Forum Solution Frameworks. Like the other frameworks, it has a hierarchical structure that supports decomposition or "drill-down" into successively finer levels of detail. At the same time, it has been developed and enhanced from the bottom up by contributions from many sources at multiple levels of detail, and it embraces a "blade" concept that allows domain-specific additions to be made without disrupting the rest of a domain.

Successful message specifications have generally emphasized top-down development to ensure consistency across messages and message sets. Even large-project message specifications like the TM Forum's MTOSI and OSS/J, which rely on multiple design and implementation teams often working in parallel, use centrally managed design guidelines, message patterns, and common data models developed over time. Lessons learned from their success are reflected in the new generation of interfaces designed and implemented under the TM Forum Interface Program. This program will support a variety of interface styles, including but surpassing the scope of MTOSI and OSS/J, and all implemented on a shared framework based on a common data model.

In my next post, I'll look at the ways in which a single data model can support a variety of application and SOA integration philosophies.

Can You Unpeel a Banana?

Maybe, but the banana won't ever be the same. If we extend this really poor analogy, we'll have a more flexible and scalable banana. (Sorry, I couldn't help myself.)

Glanced at twitter and saw @windley tweet "The trend to disaggregation gives users more choice, but necessitates other tools for threading the experience together.”

I couldn’t agree more. In fact, I liked the way he said it so much, I kept re-reading his tweet. There was something in my head that wanted to come out.

As a public representative of Progress Actional, I’m quite widely available online in various communities, blogs, and such. As my mind skitters over all sorts of news and information, I leave comments here and there, I tweet on various ideas, and blog on more comprehensive thoughts.

But being the scatterbrain that I am, I lose sometimes track. Was my comment approved? If not, I may turn it into a post of my own. I just signed up to the SOA Interest Group on LinkedIn, but did they approve my request? Has that forum been quiet, or did something happen to my subscription? How do I keep track of the services I’m using, my interactions with them, and turn that to a stream of consciousness to achieve my goals? How do I analyze the progress I’ve made, the themes that resonate, and capitalize on the effort?

Damn good questions, but SOA what?

Well, reread the last paragraph there, but think services in the context of composite application development and SOA, not sites and communities.

All you services and SOA experts out there, how do you keep track? How do you track users of services? Dependencies between services? How can you keep track of which services use which protocol, regardless of whether they are part of your SOA infrastructure, or just part of your integration backbone? How do you track from the earliest part of the lifecycle when the entire system is contained on a single laptop, through to production when it’s fully deployed and distributed? Worse yet, what if you’re using shared services in development, and your development project depends upon resources outside of your team and your control? It's worse because then you have a versioning problem even before you release a single line of code to production.

Now that we’ve disaggregated, we need to re-aggregate. That is, we need to unpeel the banana. But, we can make improvements (I have the theme song to The Six Million Dollar Man in my head). Re-aggregate automatically, only loosely, and do it using a layer of meta data around the business. One user may want a view by product, another by business process, and a third by customer type, or product class. Well, I hope you get my point that the whole idea is that it doesn't matter what your meta data layer is, only that it maps scalably and flexibly to address your current and future business needs.

That's where Actional comes in. Once services are used to disaggregate an integration project, Actional is the tool to thread the experience together at a meta, or logical, layer based upon the business. And, that's how technology can help a company use service-orientation to address business needs more rapidly than ever before.

Inspired by the Uninspired

Overly complicating things is not an accomplishment, no matter how hard it appears someone's worked to complicate the simplest things.

Take grocery shopping. Simple, right? Pick food, get in line, pay at register. Doesn't get much easier.

Except at Whole Foods.

At Whole Foods you have to pick one of several lines, each color coded. Then, register numbers appear on a screen with three colored vertical lines and you have to match the color in your line (which is only displayed where you enter the line, not where you stand on line) to the color on the screen. If the register number displays on the color of your line, you get to pay for your groceries.

I swear, at my Whole Foods the colors of the lines don't match the colors on the screens. Is the product manager color blind? Were there two different unions involved? Didn't they check before they installed it? When I asked a worker in the store, he laughed and said "Yeah, I know. But, look, it's not one of the other two colors, so it must be the color of your line." For real.

So, the simple process (get in line, pay) involves six lines, seven if you count the express line (don't ask me how that works), two TV screens, three colors repeated each twice, and 24 registers (numbered, but not color coded). To heck with queuing theory, we have colors!

I want to meet the guy (or gal) that designed this system! I would pat him on the back and with admiration in my eye be like "you are one creative dude."

And I bet if I took him out for a drink, he'd explain each and every feature of the checkout line and each corner case it was designed for. I'm sure he'd apologize for the mismatched colors, but... hey things happen. In checkout line 2.0, the colors'll match.

SOA What? Well, I think this guy got too wrapped up in the art of checkout lines and got a little (OK, A LOT!) carried away. But, we all do that. Humans are a creative and proud lot... even the dumb ones.

What does this mean to your SOA infrastructure? Well, take a step back and look at your infrastructure (SOA or otherwise). Ask your users where it's too complicated - they'll be happy to tell you. And if you're lucky, like checkout line guy, they may even buy you a drink while they do.

What does it mean to me? Well, I have to be satisfied that my adventure in queuing-theory-less shopping was much more exciting than Giles' adventure in boardroom statistics.