Business Making Progress

Failing to seize the opportunity - what bankers have in common with railroad tycoons

Following last week’s CEB TowerGroup, my main observation was that bankers are facing a ‘perfect storm’ that will necessitate a change in approach across the whole industry. Unfortunately it seems that bankers are still denying the circumstances they face, believing that they will survive.

Why do I say this? In a mobile, connected world, customers are becoming used to service levels that take into account their previous buying preferences, interests, background with the service provider and even their location. They are used to accessing services wherever and whenever they need to, via any channel, and are increasingly savvy to offers and promotions. Despite media stories of ‘Groupon fatigue’, the popularity of daily deal and voucher sites speaks of a new generation of bargain-hunters that often won’t make a purchase without some kind of added incentive.

Banks are not yet interested in this consumer trend, but they should be. Instead of seeing payments as a cost centre to their business, the data it gives them about customer spending habits, when combined with today’s mobile technology, provides a massive business opportunity.

At the moment, it seems like the banks are unable to see the bigger picture. Much like the railroad tycoons in early twentieth century America, they have the cash to invest but are not seeing the opportunity. Airlines were just setting up internal flights, which eventually became the competition that turned the railroads into a second-choice service that could only compete by discounting prices. The railroad owners could have got into the airline business, but instead of seeing themselves as transport providers, they only concerned themselves with the smaller world of railways. Bankers today, similarly, see themselves as providing a customer service that goes beyond today’s boundaries of banking. Traditionally, they’ve been the consumer’s trusted advisor. Now that trust is eroding, they will lose out if they don’t capture new ground. Otherwise, banking becomes a commodity and the only way to survive will be to drive further efficiencies and reduce costs.

In the very near future, customers will realise they don’t need banks, they need banking – and will shop around for the best offer if their loyalty has not been secured.

Solving the problem will require looking to new areas. At the CEB TowerGroup event, the key note speaker made this comparison: after 100 years of failed expeditions, Mount Everest was eventually conquered when new explorers brought in nylon ropes from the world of sailing (to avoid the problem of frozen ropes snapping) and oxygen masks that had been developed for fire fighters. Innovation is ‘out there’ – you don’t need to find it in your own discipline or reinvent the wheel, just observe how other sectors are embracing change and see what might help solve business problems in your area. When it comes to innovation, bankers would be wise to remember that to get to the top, they need to look to other areas for inspiration.

So where should banks look for innovation that will help them adapt their business for the next 20 years? Much of the technology they need to become more responsive to customers’ needs and open up new revenue streams already exists. Bringing together the engines from algorithmic trading, BPM technology from the telecoms industry and location-based services from the mobile world, they could make full use of the information they already have about what consumers like to buy (and where and when) to develop new, more targeted, offerings.

We have a vision for Responsive Customer Engagement, a technology approach that means a bank can respond to a customer’s actions and turn it into an opportunity to secure more revenue – and, perhaps more importantly – customer loyalty. One business model for this could be in providing a customer buying one item on their credit card access to offers and promotions from other merchants in their locality. The merchants – also the bank’s customers – would also become more bought-in, as their payments provider becomes their access to a mass of new custom.

I will be talking more about how these technologies could help banks tap into consumer trends and bring the change they need to survive in future blog posts. For now, I’d like to leave you with a thought from the mountain: if you haven’t yet reached the pinnacle, perhaps you are using the wrong tools. Don’t get left behind!

Banks Need Transaction-Level Insight Into Mainframe Systems to Meet New Transparency Demands

Recently I participated in the first installment of a webinar series  hosted by Bank Systems and Technology. I was joined by Greg MacSweeney of Bank Systems & Technology and analyst Gareth Lodge of Celent, and we discussed the need for banks to have transaction-level insight into mainframe systems to meet customer, partner and internal demands.

We are living in an increasingly complex world, especially in the financial services space where regulations continue to tighten. Banks must increase analysis to meet this heightened demand for reporting and compliance, and to do this we need real-time access and visibility into all transactional data. As Gareth said during our discussion, “Mark Twain said we can count on two things in life, death and taxes, and now we have a third: regulation.”

In addition to meeting these new requirements, customers are increasingly expecting immediate and detailed response and communication from their bank. However, this type of service requires access to transactional-level data in real-time. This is a multi step process and is not automated, so how do we get it done in a timely matter for the customer especially when this data is hard to find and difficult to interpret?

Many banks still maintain complex system architectures comprised of old and new mainframes- many of which are very difficult to access and navigate. It’s like operating in the dark, where there is very little transparency, creating frustration for IT departments.

In the future, we hope to see banks address some of these challenges with new technology. In order to meet new regulatory demands, banks need access to transactional level data, which will improve responsiveness and ultimately customer service. However, finding transactional level data on the mainframe and being able to act upon it poses quite a challenge from a business and operational perspective.

So how do we do that? Stay tuned for my next post, highlighting second installment of our webinar series. I welcome your comments here, as well.

How Software Can Help Airlines to Keep All Their Plates Spinning

By Joshua Norrid, Industry VP, Travel and Leisure, Progress Software

I often think that managing an airline in today’s climate can be a bit like trying to keep several plates spinning at once – one mistake, and all your hard work can result in a costly, embarrassing mess. Over the last 12 months, we’ve heard a great deal from airlines which are finding themselves having to make some difficult decisions about how they approach the road ahead. Increased costs across the board have seen airlines of all shapes and sizes slashing their budgets as they look for the best way to cut costs without reducing overall efficiency.

The situation has now become so drastic that traditional industry growth has been put on hold by the majority of airlines, which are instead looking for ways to keep their heads above water by releasing funds from areas such as maintenance and distribution. So what are the causes of these increased costs? As we begin to emerge from the shadow of the global economic downturn, why are so many airlines finding it necessary to tighten their belts?

Clearly, there are a number of conflicting economic and political factors driving this need. Perhaps the most significant of these is that the cost of fuel has rocketed to as much as $100 a barrel, and as much as 31 per cent year-on-year in some territories. This price increase has had an obvious knock-on effect for the airlines industry, many of which have been faced with a straight choice between passing this increase onto their customers or trimming overheads.

Other factors, including an increase in political hotspots around the globe, are reducing the number of flights leaving each day - adding to the pressure. Either way, it’s becoming increasingly clear that airlines are facing an uphill struggle, as they fight tooth and nail to hold on to market share, while trying to ensure that customer satisfaction is not affected by their drive to reduce overall operational costs.

Perhaps this is where software developers can play a part? By allowing airlines to view business-critical information in real time, complex event processing software can help to establish which areas require greater resources, and which can be trimmed as and when circumstances dictate. This will allow them to view the performance of their entire operation in real time, and make better, more informed decisions. This software could be one way for them to avoid making potentially costly mistakes – and keep all of their plates spinning!

An Algorithmic Trading and Market Surveillance Wrap Up

Our Capital Markets and Progress Apama teams have been BUSY! Today many of them are recovering from a busy week at the SIFMA Financial Services Technology Expo in New York City.

In addition to lots of greeting and Tweeting, Dan Hubscher even had some time to post a couple blog posts:

• Tickets Please: Technology to Keep You on the Train
• Keeping the Train on Track

And while Dan helped man the floor, our VP of Corporate Communications, John Stewart, worked to get 3 press releases out onto the BusinessWire, including:

• Progress Software and Activ Financial Introduce New Market Data Adapter for Apama Capital Markets Foundation
• Progress Software Survey Reveals the Realities of Regulation, Monitoring and Algorithms on the OTC Derivatives Market
• Progress Software Launches Enhanced Market Surveillance and Monitoring Solution

Not only that, our own Dr. John Bates was tagged by Wall Street & Technology as one of our "Top 10 Innovators of the Decade for Capital Markets”.

Across the pond and beyond, Dr. Richard Bentley was quoted in the Bobsguide article “Suspect movements in share price fall to an eight-year low”. Dr. Giles Nelson traveled to India to promote Progress’ business in Capital Markets. He shared his thoughts on our complex event processing (apama.typepad.com) blog:

• The rise of algo trading in Asia - first stop Mumbai
• Still big room for growth in Japan

Phew! And, just in case you missed it, we wrapped up posting a 4 Part video series on Financial Regulation and Market Surveillance. Here are the blog posts that provide a brief overview and link to the videos:

• PLUS Stock Exchange Making Progress Around Regulation
• Using technology to prepare for regulatory change: how can market surveillance technologies help?
• Preparing for changes in the regulatory approach
• Cutting through the confusion of financial regulation

 

Dr. Ketabchi Reveals Secret Sauce of BPM

Are you a business or IT person who wants to... Reduce costs? Improve quality? Manage exceptions? Increase revenue? Most likely, it's all of the above.

Dr. M.A. Ketabchi, Chief Strategist at Progress Software, presented at the Gartner BPM Summit in March on how you can increase your business operations IQ.  He explains how you can run your operations more intelligently, and he reveals:

• The “secret sauce” of business process management
• Well-known companies that have benefitted from BPM, and how they did so
• How business event processing (often referred to as Complex Event Processing (CEP)) enhances BPM
• Typical business challenges that require a combination of BPM and BEP
• How Progress® Responsive Process Management (RPM) brings it all together

If you are interested in the replay, click here. It will be available FREE FROM REGISTRATION for the next five business days only. What a deal! ;-)

Hope you enjoy it. And, as always, make sure to share your feedback and comments here.

Building Customer Loyalty in a Churning Market

All businesses suffer from churn – the moving of customers from one service provider to another. As new and innovative services become better understood and more widespread, more suppliers enter the market and so the opportunities for customers to change suppliers increases. Churn is expensive. Recruiting a new customer can cost 5-10 times that of retaining an existing one. So how can technology help in the constant battle to retain customers? 

I’m going to illustrate what can be done by talking about mobile telecommunications – an industry where innovation is rife but where churn is a significant problem.

Mobile communications continues to grow very quickly. According to a recent Cisco survey, mobile data volumes are nearly doubling each year. By 2015 it predicts there will be 7B personal mobile devices globally. Analyst firm Ovum recently reported that in 2010 revenues from mobile data for European mobile operators exceeded that for voice calls for the first time.

Smartphones are completely changing the way that people use the Internet. It’s worth reminding oneself that now, in a pocket device, one has a phone, a camera, email, PDA, mapping with GPS, in some countries a near field payment device and of course access to thousands of applications. Morgan Stanley has predicted that in 2012 shipments of smartphones will, for the first time, exceed those for personal computers. The whole landscape of computing itself is changing.

With all this growth mobile operators should be very happy. Subscriber bases and mobile data volumes are growing. And yet, mobile operators can’t rest easy. Yes, innovation is everywhere, but most of the innovation (at least that visible to end users) isn’t going on in the mobile operators – it’s going on in the phones and the applications. End users are becoming more and more divorced from the particular network they use and, certainly in developed markets, operators primarily compete on two things only – price and coverage. The growth of the mobile Internet is pushing operators to the bottom of the value pile and risks leaving them as faceless utilities. This, in turn, leads to churn, with rates for mobile operators range from 20-40%, meaning that between 20 and 40% of subscribers will, per year, leave a network for another.

Other industries are of course liable to churn too. Insurance is one example – I recently used an online insurance aggregator to find car insurance and, within a few minutes, obtained a rate 20% below that that my current provider was offering. Online retail is another – it’s very easy to move to another retailer that might be offering a lower price on the same product. In general, churn is present wherever a product is a commodity or near commodity and where customer relationships are weak.

Some wireless operators are fighting back by identifying more ways in which they can meaningfully interact with their customers. To take a concrete example, one European telco, a Progress customer, is now continuously monitoring calls from their 30M subscribers to identify patterns of usage that indicate a different tariff would be more suitable for that subscriber. This could be as simple as noticing that the number of bundled monthly minutes used had been exceeded. A text message is then sent to the subscriber suggesting that they move to another tariff that would reduce the cost of calls in future. Time is of the essence. If the subscriber receives an offer soon after placing one of these calls they are far more likely to accept it than if the offer came through many weeks later.

The way that marketing campaigns are run can become a lot more responsive. The mobile operator may decide to run a campaign to, for example, promote a particular tariff it thinks will be of interest to a subset of its subscriber base – those people, for example, who spend more than $100 per month and roam frequently. Sending out offers via text message requires great sensitivity, as no operator wants its customers to feel it’s receiving spam. As the campaign executes results can be monitored in real-time and the target demographic of the campaign can be tightened to achieve a better response rate. Not only does this make the campaign more successful but also those subscribers that, in the end, are not targeted can become the target of a future campaign.

To do this requires a number of things. Firstly, software needs to be in place to allow the millions of subscriber calls to be analysed in real-time – an ideal use case for event processing. Secondly, there need to be tools which allow a marketing team itself, working largely autonomously without IT support, to create, test and dynamically enhance the rules which dictate which subscribers will receive the offer. And finally, positive responses to the offer need to be processed systematically through an order management system.

There are many other examples where responding quickly to subscriber activity can enhance a user’s experience of using a particular mobile operator. As Internet use becomes dominated by mobile, it’s likely that variable costs for data access, particularly where large downloads are concerned, will be introduced. The cost of a download will be calculated dynamically, dependent upon the bandwidth available within a particular cell at a particular time. At initiation of a large download (let’s say greater than 1Mb) the user could be prompted to ask whether he would like to download it at twice the normal bandwidth for another 10c. This would be a dynamic rate, calculated in real-time in response to current activity in the wireless cell and the propensity of the user to accept the rate.

So, what’s the general lesson here? By becoming more responsive to subscribers, mobile networks are increasing their value to customers, improving customer service and so reducing the likelihood of churn. Existing information about customer behaviour is being used but by being able to act on that information immediately they are able to communicate in a much more contextually suitable way so improving response rates and strengthening the customer relationship. All businesses should be looking how to use their operational information to respond to and interact with customers better. Real-time responsiveness to customer behaviour is becoming vital.

Business Process Improvement vs People

Do you think we can actually improve business processes if people are involved? I think the answer is yes… at least I hope so. Back in May 2010 I completed the introduction of a single sign-on (SSO) project that I had been working on for over a year. I admit it, it was painful. During the planning and execution phase, I experienced, firsthand, the difference between the goals of the business and the needs of IT.  It wasn’t the first time but it made me realize what I wasn’t missing… CONFLICT.

Well, two months following the implementation we saw a huge drop in leads. Why? Because people didn’t want to “join our group”. They just wanted a simple white paper or archived webinar. This is what I would call “people process intelligence”. In this case, the process intelligence that we instituted for our visitors wasn’t really wrong or broken, it was wrong because we based the new process on how we “thought” our people would interact or respond to SSO. I have no doubt that SSO still makes our customers, partners and employees happy because with a single UID and password they can access public website assets, Community assets, updates & downloads, and even technical support apps, but from a prospecting point of view, it doesn’t make sense.

For most businesses, their processes (or business events) are much more complex, but I would wager a bet that anyone owning a critical process still needs to bring together business goals and the technology requirements imposed by IT.   As I embark on Phase II, I find myself being a bit of a bully about certain things, like whether we include a Progress ID image or not (NOT). I also find myself annoyed because what I hoped would be a “short form” for guests is now 8 fields.  However, as annoyed as I am, those “extra” fields are critical to our being able to properly route and nurture a lead. It’s justified so I’ll zip my lip on that. What else am I doing? I’m getting to know my Omniture tracking powerhouse a little better. Once we go live, I want to clearly understand how my people are interacting with my form(s). I want to know when they are abandoning it (perhaps that 8^th question bummed them out), and what happens after they click Submit. Do they register for additional assets via the pre-filled form?  Do they come back at a later date and take advantage of the pre-filled form? Do they decide to “join our group”? Is anyone really sad we got rid of the big green Progress ID image? Basically, I want metrics to lead the process, not our opinions.

The fact of the matter is that people will always be involved in business process improvement. They’ll be part of planning and deployment, optimization and reporting, and ultimately “a people” could very well determine if your process is a successful one. Now, Progress Software can’t do much about "a people", but we can help improve process intelligence by giving you some of the tools you’ll need to be successful. We provide solutions for business process management (BPM), complex event processing (CEP), and application performance management (APM), just to  name a few.

Give us a call. We welcome the opportunity of telling you how we’re helping our customers bridge the gap between the business and IT, and improve operational  performance.

Real-time Visibility Will Help You Make Better Decisions

Decision management expert James Taylor believes it’s really very simple… to succeed, you need to sense and respond to what’s going on in your business, in your systems, and all around you. He’s right. In my job, I need to be alerted immediately when the website is down. Now, we aren’t saving lives here at Progress Software (although I’d like to think that applications using our software and technology are) but our website is core to our business. If our site goes down, it will result in poor customer or visitor experiences which translates to lost revenue. What I need, IT are you listening?, is a notification service that alerts me the SECOND the site fails. If I had that, I could respond quicker and make sure that IT was working to get it back up. If I had that, I could also plan and make decisions on how I would respond to my frantic boss, the irritated visitor, and the customer support rep who just lost his/her web app right in the middle of a troubleshooting session or order process.

Anyone else know my pain?

Now, we all know that my wish is doable because it’s relatively simple, but for many businesses, their business events and processes are much more complex and the consequences of process failure are much more severe. I think it’s pretty clear that in order for us to make fast decisions, we need to begin by gaining visibility into our business processes. The Progress Responsive Process Management (RPM) suite is all about giving you the operational tools and business control panels you need to be responsive to changing conditions.

So, what's the next logical step for you? Learn a bit more about the first component to building the responsive enterprise that lets you make smarter decisions - achieving real-time visibility. Listen to James Taylor’s 3-minute visibility podcast – the 1st in a series of 4 podcasts. Not only will you hear how you can achieve real-time visibility but you’ll learn how the Progress® Responsive Process Management (RPM) suite can help.

If you are interested in reading more, register to download the companion white paper Building Responsive Enterprises: One Decision at a Time.

Episode1: Visibility

Or listen via player:

Sharing some winter sun with Progress' application partners

Earlier this week I participated in the 2011 Progress Global Partner Conference which was held in Florida.

This is only the second time the partner conference has been global – previously it was held regionally – and I’m delighted to say that hundreds of representatives from business partners attended from all over the world.

Most of the partners present were what Progress terms application partners – those that have used Progress products to build applications that are then sold to end-users. As always, the sheer diversity of the applications partners create and sell is mind-boggling – from healthcare apps specialising in kidney treatment to point-of-sale retail systems deployed in 25,000 outlets worldwide to location-based content delivery platforms. Progress recognises the many varied achievements of its partners with its very own awards ceremony. The winners can be found here. And, yes, it is a tiny bit like the Golden Globes, although without the acerbic wit of Ricky Gervais.

These partners continue to be incredibly important to Progress Software. Supporting them with product innovation as well as facilitating new ways for them to deploy their applications (for example by testing out their applications in the cloud with Progress Arcade) is a key pillar of Progress’ strategy. Another strategic pillar is Responsive Process Management (RPM), launched by Progress to the market in 2010, and several sessions in the conference were dedicated to explaining how RPM fitted into the partners’ world. Adoption of RPM in the partner community is happening. An example of this is Skyward, school management software supplier, recently announcing their use of Progress’ OpenEdge BPM platform. This puts them on the first step to full RPM adoption.

John Rymer from Forrester Research, the software analysts, also addressed the conference. Amongst other topics, he talked about four big “on-ramps” of new functionality – business process management, analytics, business events, and collaboration. These, he believed, were the most effective ways for software vendors (Progress’ partners in this case) to deliver new functionality fast and will be the key technologies behind many of the next generation software platforms. Forrester’s presence at a Progress partner conference was timely. Recently one of their analysts, Mike Gaultieri, blogged about Java, despite being more popular than ever, being a “dead end” for enterprise application development. He encouraged developers to consider alternatives, including Progress OpenEdge, that offer substantially higher productivity. It was a reminder that OpenEdge remains as relevant as ever and is a great aid in application modernization. Further output on this topic from Forrester is imminent.

All in all, it was a successful, high-energy conference. Many thanks to all the partners who came, and to those that didn’t, please try and make it in 2012!

 

 

Red Flags in Morning, Firms Take Warning

A pattern is emerging within new financial services regulations where regulators and financial services firms deploy monitoring technology to "red flag" potential issues such as risk, position limits, errors and manipulation. The "red flags" raised would then alert the relevant personnel or authorities.

In the case of the Volcker Rule, prohibiting banks from proprietary trading and investing in or sponsoring hedge funds or private equity funds, the authorities would use a three tiered approach (http://tinyurl.com/2bh9ot3).  First "tripwires", such as the length of time a trader holds a position, its size or riskiness, would alert banks’ compliance departments  who would (#2) quiz the trader on the nature of the position. And (#3)regulators that keep inspectors on banks’ premises would see the tripwires and monitor both traders and compliance departments.

Over at the CFTC, regulators are looking at a similar approach to monitoring and controlling position limits on products such as oil and metals with a "points" system that would give the CFTC monthly reports that it could use to red-flag traders with large positions (http://tinyurl.com/2ugbdh6).

The tracking and red flag approach is the latest step in increased monitoring of trading operations with the ability to take response before it’s too late. At Progress, we have been advocating using monitoring and surveillance technology to help catch inside trading and avoid fat fingered trading errors for years. With new regulations, monitoring becomes not only mandated but more complicated. Red flags are likely to be flying all over the place within as little as months, both inside and outside financial services firms, presenting a fine opportunity for our Responsive Process Management software solution.

As the financial services world becomes more compliant, the ability to manage red flags becomes more critical. Every process within a financial services firm must be scrutinized, from trade entry to risk management, to analyse and understand internal and external events. This take sophisticated technology. This is where Progress Software's RPM software fits in. According to technology research firm Ovum: "Unless an organization has already made a significant investment in creating an operational responsiveness solution around best of breed products, it will be worth seriously considering the competitive advantage and improved effectiveness that could be achieved by deploying RPM."

Ovum noted in a Technology Audit note that multiple technologies are required to gain a comprehensive insight and respond more rapidly to changes to the environment. These include: business process management (BPM) to model, implement, and execute the processes; business analytics to determine how effectively the processes are working; complex event processing (CEP) to understand the implications of many streams of internal and external events; business rules management to determine the appropriate actions for a given set of conditions and variables; and visibility into end-to-end transactions to track and audit their progress.

The interrelationships between all of these components and the vast amount of information that has become available must be understood before its impact on processes can be ascertained and appropriate tuning performed. In other words, RPM is the answer.

RPM can monitor an increasing number of information feeds, both within or external to the organization, then apply business policy and governance rules, then automatically tune the  established process or alert a human decision-maker (if necessary) and present him/her with current, relevant information on which to base the most appropriate response.

According to Ovum: "All of these individual capabilities already exist (at different levels of maturity), but the cost and complexity of integrating these into an effective business solution is beyond the means of most organizations. Hence Ovum believes that the requirement identified by Progress represents a genuine market opportunity." Well said. 

 

If a Customer Tweets into the Ether, Do They Make a Sound?

As you may have read in yesterday's post, we hosted our final Progress Software Summit in San Francisco on Wednesday. We again heard from the venerable, John Rymer of Forrester Research. In his speech, “Design Solutions for Faster Change and Greater Business Impact” he touched more strongly this time on the concept of customer service as it relates to real-time business event processing.

We have previously cited his example of Maytag responding to mommy blogger Dooce’s issues with the company’s product and subsequent public ribbing of the company via social media channels. This was a great example of reacting to a customer, one empowered by their own online reputation and influence, to the best outcome for all parties involved. Yesterday, however, John gave some examples of companies who are not only reactive but also proactive in how they approach customer service issues via social media, the ultimate in real-time dialogue with consumers.

In keeping with our summit series’ central topic, how businesses can become more operationally responsive, John discussed how empowering employees to solve problems sans red tape is important. He brought up the example of Best Buy’s Twitter team that is able to answer product queries, solve problems, and rectify direct criticisms / customer aggression in real-time mitigating the risk of a decline in brand sentiment and loyalty with every happy or at least satiated customer with whom they interact.

With more and more channels through which customers can complain and a decreasing reliance on traditional customer service lines and email aliases that long go unresponded to, the masses are taking to channels like Twitter forcing organizations to respond in new ways as well as make decisions about issues faster and more efficiently than ever before.

Overall, attendees left thinking about how they see and respond to business issues in new ways from every speaker. From John though they also left with new places to take long hard looks at in planning their customer service efforts.

Talking about Twitter... take a minute to follow me, John R. Rymer, and Progress Software.

Mr. Spock, a duck, and the Maytag Repairman…

You may be asking yourself what these three names have in common and we found out at the Progress Software Summit on responsive process management in New York City yesterday.

Following Larry Kudlow’s speech the audience heard from Progress Software CEO, Rick Reidy, yours truly, and Forrester analyst John R. Rymer. What we heard from Rymer repeatedly was that business event processing is the next big architecture movement. He argued that a company’s responsiveness was based on its ability to empower its people to be responsive through the elimination of restrictive policies and regulations. The use case of this was the interaction between influential mommy blogger Dooce and Maytag – with hundreds of thousands of Twitter followers, Dooce became an “empowered customer” whom Maytag could not and would not ignore when it came to responding to her gripes about her new washer. Being able to respond quickly and nimbly, without months of red tape allowed Maytag to weather what could have been a scathing PR debacle.

Rymer was also uniquely able to discuss Hans Solo and Captain Kirk as the intuitive business users in comparison to Mr. Spock and Obi Wan Kenobi who represented analytical business users. While a big hit with this tech-heavy crowd, this analogy also served to reinforce my discussion at the need to cater to pure business users, business analysts as well as IT. All organizations have Kirks and Spocks.

This all tied nicely back to the analogy that started our day that came from our CEO, Rick Reidy. If you and your competitors are ducks in a pond, who is the leader and who simply follows the flock? In order to be a leader one must be able to just nearly see the future or, at the very least, not drive in the rearview mirror. The idea that responsive process management, business event processing and, more basically, seeing and responding in a timely way can make you that leading duck was an easy to digest message that rang true for everyone who made it out to today’s Progress Software Summit.

Afterwards... it was off to Chicago!

Fighting payment fraud with Complex Event Processing

Today Progress Software announced that SEB bank in Estonia has deployed the Apama Complex Event Processing (CEP) platform to look for payment fraud.

This is exciting news and the fact that such a production customer has gone public is a great validation of SEB's belief in the system they've built. A lot of banks are somewhat reticent to talk publically about fraud. Everyone knows it goes on of course, but banks usually prefer that this is a quiet fight, going on behind the scenes. SEB on the other hand see their use of CEP to detect fraud as a positive demonstration of their commitment to customers and card issuers to ensure that banking with SEB is as safe as possible.

A few weeks ago I visited SEB in Estonia and was hosted by Ain Rasva, the head of technology, together with the head of the fraud team. One of the key reasons they chose to invest in CEP was rising transaction rates and a realisation that conventional data management architectures were simply not suitable to look for fraudulent patterns in a timely fashion. Yes, transactions could be captured in, say, a database management system, but then queries to determine whether a pattern of fraud was emerging had to be run and re-run at arbitrary time intervals. An event processing approach is simply a better way of doing this job, both performance-wise and conceptually. With CEP, SEB can now detect a potentially fraudulent pattern of card use immediately and then start managing the case, using tools to conduct further analysis and to manage communication with the customer. This can be done now in minutes, rather than the hours it took previously. Patterns of fraud detection are constantly changing and SEB needs to be responsive to this - the fraud detection rules need to change frequently and quickly. With Apama, these fraud rules can be created, modified and tested rapidly, and this can be done largely by the fraud department itself rather than relying upon IT to effect each change.

Card payments continue to change and increase. There were 82B payments in Europe in 2009 and card payments are growing at a CAGR of 12% per year. Europe wide regulation such as the Single European Payment Area will significantly change the types of fraud that banks need to look for. SEB has placed itself in a good position to ensure that payments are conducted as safely as possible.

Gain Actionable Insight Into Your Operations at Summit 2010

The Progress Software Summit 2010 brings together the visionaries and experts who are applying today’s proven technologies that will help companies achieve operational responsiveness. Learn how applying business solutions such as Responsive Process Management (RPM) will help you gain agility, reduce risk and achieve 20/20 foresight. Transform the way you do business.

At the Progress Software Summit, you will:

• Learn how you can make better business decisions by achieving real-time visibility and actionable insight.
• Connect with your business peers and innovators, and share how you are - or are not - achieving operational responsiveness.
• See the latest innovations in business process management, complex event processing and more.

JOIN US IN A CITY NEAR YOU!

November 2, 2010 HUDSON NEW YORK 356 West 58th Street New York, NY	November 3, 2010 W HOTEL CHICAGO 172 West Adams Street Chicago, IL	November 10, 2010 BENTLY RESERVE 301 Battery Street San Francisco, CA
Our New York event begins at 11:00 am and ends at 5:30 pm, and is followed by a Cocktail Reception. Hear from Progress plus industry expert John Rymer, VP & Principal Analyst at Forrester Research, Inc. Our SPECIAL GUEST at the New York event will be Larry Kudlow: Mr. Kudlow is host of CNBC’s primetime “The Kudlow Report” and co-host of “The Call”. He is also the host of “The Larry Kudlow Show” which broadcasts each Saturday on WABC Radio. He is a nationally syndicated columnist and the author of American Abundance: The New Economic and Moral Prosperity. Learn more >	Our Chicago event begins at 11:30 am and ends at 5:45 pm, and is followed by a Blackhawks game. Hear from Progress plus industry expert John Rymer, VP & Principal Analyst at Forrester Research, Inc. Our SPECIAL GUEST at the Chicago event will be Dan Hampton: A first-round draft pick of the Chicago Bears in 1979, where he continued to play 12 years as a defensive lineman, Dan Hampton is an engaging football Hall of Fame personality who is well-known and liked for his anecdotal and motivational speaking style. Learn more >	Our San Francisco event begins at 11:30 am and ends at 4:40 pm, and is followed by a Yacht Cruise. Hear from Progress plus industry expert John Rymer, VP & Principal Analyst at Forrester Research, Inc. Our SPECIAL GUEST at the New York event will be Richard Karlgaard: Rich Karlgaard has a unique vantage point on the trends driving the business and investment climates. He is the publisher of Forbes magazine, where he writes a bi-weekly column. Mr.Karlgaard is author of the Forbes book, Life 2.0: How People Across America Are Transforming Their Lives by Finding the Where of Their Happiness. Learn more >

Delivering Operational Responsiveness

We've already learned that operational responsiveness is more than agility and business process optimization, it’s about plugging decision makers into business events and giving them the tools and information they need to respond to the unexpected, thereby allowing them to capitalize on opportunities, drive greater efficiencies, and reduce risk. We've also learned why it's so hard to achieve. Now, how do we help deliver it?

Dr. John Bates, Senior Vice President, Chief Technology Officer and Head of Corporate Development at Progress Software

Watch Part 3 of our 7 part video series. In this video, Delivering Operational Responsiveness, John explains how we've brought together three proven technologies that help companies achieve operational responsiveness – business transaction assurance, complex event processing (CEP) and business process management (BPM). All these powerful technologies are further enhanced with business rules and analytics.

The best part? You drive. A business control panel will give you the ability to gain real-time visibility into business events, immediately sense and respond to changing conditions, and achieve continuous process improvement.

Read what Wikipedia has to say about Responsive Process Management (RPM) and learn how Agent O applies RPM to tackle credit card fraud in real time. You can also follow Dr. John Bates on Twitter.

Enjoy past videos of this seven part series:

What Is Operational Responsiveness?
Why Is Operational Responsiveness So Hard To Achieve?

 

Transformational IT change at British Airways

Gordon Penfold, Chief Technology Officer, at British Airways, started off by telling us that not only is the Boeing 747 40 years old this year, but so is the IT that supports the 747. Having been early to the world of real-time processes, the technology is now facing end of life at the end of 2010, and it's time for a major change.

Gordon shared with the audience at the Progress Software Summit (#progressswsummit) that to get to a SOA-based infrastructure that will deliver the vision of a single real-time infrastructure linking retail, customer, operational and corporate data and processes has required significant technical and strategic change. BA needs to use their enterprise architecture to improve quality and minimise cost, directly affecting BA's objective of being a global airline of the highest quality and providing exceptional customer service. And the term "global" isn't some platitude in this case as BA has operations and staff distributed all over the world. As Gordon said, BA is a peripatetic company.

BA's ‘common architecture’ will ensure consistent implementation, enabling BA to run existing services and implement new ones effectively. SOA is Gordon’s preferred option. The aim is to reduce complexity and increase agility, introducing a plug-and-play capability that can also manage ‘heritage’ applications, created by the industry and service providers – all to support over 15 million transactions a day.

Progress is supporting the change at BA with its Sonic, Actional and DXSI products, and is working with Gordon’s team on a proof-of-concept for Savvion and on how Apama can support with interactions with partners and other airlines. At the half-way point in the implementation, BA is now live with its Progress powered Service Integration Platform, which links the business to IT processes in a controlled way.

Gordon made some kind remarks on working with Progress - the best-in-class products and the added value provided by the Progress people who worked at BA on site. All great to hear from a business with very real mission critical requirements.

Are you a sitting duck or one that will respond immediately to threats?

While many organisations are being ‘cautiously optimistic’ about what the future holds, the realities of today’s tough business environment could leave them as sitting ducks, according to Rick Reidy, CEO at Progress Software. They might take consolation that they’re in the same pond, but when interest rates in Japan hit near-zero, banks continue to fail and mistakes can lead to a ‘flash crash’, the pond is not a safe place to be. Businesses may have money, but fear and uncertainty is holding back decision-making – we await further regulation and want to know the consequences of recent government changes.

 
Listening to Rick’s keynote at our UK business summit (#progresswsummit, if you want to follow on twitter), in the impressive surrounding of Chelsea Football Club’s ground, London, it seems most of the audience agrees – it’s not good enough to sit around and wait to see if growth returns, and you cannot grow simply by cutting costs. You have to take control of your own ‘growth agenda’, as Rick put it. Businesses that want to survive the next five years need better visibility, through putting processes in place that enable them to react quickly to meet customer demands, adapt to market changes and take advantage of new opportunities. As Rick has advised, businesses need to act on up to the minute information so that leaders can make decisions based on foresight, not hindsight.
 
If you’re a regular reader of this blog you’ll already know that we call this ‘operational responsiveness’: the ability to sense and respond to customer and market changes so that organisations can move quickly to meet challenges and take advantage of new opportunities. 
 
Rick has talked about what this means in the airline industry: the notion of irregular operations has become a weekly reality as companies face intense market pressure, striking staff and disruption from natural phenomenon. ‘Swivel chair’ communication between operational areas is no longer good enough. To react quickly enough, they need responsive processes in place that can help them maintain services and inform customers, almost as-it-happens. If they don’t, they will face massive fines, lost custom and damaged reputation – risks no company can afford at present.
 
We’ll be hearing more from Gordon Penfold, CTO at British Airways, about their approach to becoming operationally responsive to meet the challenges of today and tomorrow. Watch this space for my take on his talk…

 

Stamford Bridge, here we come

Tomorrow sees Progress Software taking over Stamford Bridge, home ground to the world-famous Chelsea Football Club. We’re not just there to check out the players’ dressing rooms – we are being joined by James Caan, of Dragons' Den fame, as well as the great and the good of the UK business community, to discuss how businesses can start to make decisions based on foresight, not hindsight, in their operations.

Gordon Penfold, Chief Technology Officer at British Airways, will be sharing his insight on ‘operational foresight’, revealing how the organization has set itself up to better deal with the irregular operations that have become a fact of life in the last year. And Mike Gualtieri, senior analyst at Forrester Research, will be sharing his views on where the next wave of truly responsive business management is coming from, and which trends to watch for. And Progress' own Chief Executive Officer, Rick Reidy, will be giving a keynote too.

I'll be there, speaking in one session but also blogging and tweeting from the event. So watch this space for the latest updates.

For those of you attending, I look forward to seeing you there.

www.progresssoftwaresummit.com

 

Why Is Operational Responsiveness So Hard To Achieve?

We've already learned that operational responsiveness is more than agility and business process optimization, it’s about plugging decision makers into business events and giving them the tools and information they need to respond to the unexpected, thereby allowing them to capitalize on opportunities, drive greater efficiencies, and reduce risk. But why is it so hard to achieve?

Watch Part 2 of our 7 part video series featuring Dr. John Bates, Senior Vice President, Chief Technology Officer and Head of Corporate Development at Progress Software. In this 1 minute, 30 second video, Why Is Operational Responsiveness So Hard To Achieve?, John explains why many companies struggle to achieve operational responsiveness. He points out that the lack of real-time visibility is just one key reasons.

Read what Wikipedia has to say about Responsive Process Management (RPM) and learn how Agent O tackles credit card fraud in real time. Follow Dr. John Bates on Twitter.

Enjoy past videos of this seven part series:

• What Is Operational Responsiveness?

How Being Complex Makes Transaction Assurance Simpler

I have been seeing an increasing amount on interest in the marriage between Business Transaction Management (BTM) and Complex Event Processing (CEP). On July 29^th Dr Dobbs Journal published an article called Complex Event Processing: IT Liberator or Over-Engineering Hell? This article was about the synergy of BTM and CEP (although I felt it was rather biased towards one company). Also, last week Jean Pierre Garbani at Forrester published this blog in which he discussed the evolution towards BTM and CEP working together.

Business Transaction Management is a rapidly growing area of Application Performance Management (APM). BTM enables users to look into the transaction flows within their business and ensure everything is running as expected. BTM enables problems in transaction flows to be discovered – such as a bottleneck in an important business process. The really appealing aspect of BTM is it can do this without the need to change the applications in the business; BTM can “discover the transaction flows” by tapping non-intrusively into the flows going through application servers, middleware buses, business process management systems and other systems within the environment. Over time, BTM can build up a picture of the environment’s business flows, look inside the transactions and flag up immediately problems that can really hurt the business.  Thus BTM works really well in legacy environments – not just modern SOA environments. And of course it appeals to business executives and operations users – not just IT users.

Complex Event Processing is the ability to correlate events flowing through a business  - to identify patterns in real-time. These patterns might indicate opportunities and/or threats to the business that have just happened, are in the process of happening or are likely to happen right now. Events are occurrences in the business, such as stock market quotes in trading, call data records being generated in communications or packages changing location in logistics. An example of a real-time opportunity is a trading “statistical arbitrage” opportunity – to sell one instrument and buy another at a micro profit; another is the ability to upsell something to a customer who has just purchased an item on their credit card – based on their spending and buying patterns, their location and context. Threats to be detected include risk exceeding a certain key level in a bank or gaming fraud occurring in a casino. This kind of business level visibility and immediate response also appeals to business users as well as IT.

Listening to the descriptions of BTM and CEP, does it sound like there is a little overlap? Well there is some. What BTM is really good at is non-intrusively discovering process endpoints and the events they exchange – and then tracking these events. What CEP is really good at is correlating complex real-time business events in real-time, including arbitrary user-defined patterns, which can evolve over time as the business evolves. So it makes perfect sense to put these capabilities of BTM and CEP together. For BTM this strengthens the real-time correlation and pattern detection capabilities. For CEP this enables discovery of services without the need to do expensive and time-consuming instrumentation of the environment.

At Progress we have two leading products in the BTM and CEP categories: Actional and Apama. We believe that BTM and CEP capabilities are converging for certain business use cases, so as part of our Responsive Process Management (RPM) suite we now provide seamless integration between these capabilities. Of course RPM does much more than that. More on that later!

Information Overload?

Last week I had the wonderful fortune to deliver a webinar with Robin Bloor on the role of events in decision management. This event, as a part of Information Management’s Espresso Shot webinar series, gave us a chance to dig into the market conditions and the potential ways that events can affect decision management by injecting real-time information into the process.

As many of you that might be familiar with my webinars and papers will recall, I’m a strong proponent of complex event processing (CEP) as a significant contributor to the changing landscape of decision management. Let’s face it; even in the intense world of Capital Markets where Apama’s event processing thrives in high-volume, high-velocity scenarios, the result is improved decision management, but in the end it’s all about who makes the right decision first. There is absolutely no reason why that benefit cannot be enjoyed outside of the capital markets space – and in fact it is. And, the good news is that it doesn’t have to be high-volume, high-velocity scenarios. Need to find the needle in the haystack? – event processing will do nicely.

Event Processing has hit the main stream. The recognition by business that events can be harnessed and used to influence decision management is gaining momentum. But, there two things that I think Robin and I eluded to, but probably needed to push a bit more.

1. First, event processing is architecture. Yes, it’s about data. But data that’s moving, transient and very little context. I see a time in the near future where there will be event architects with the responsibility of making sense of an enterprise’s event streams.
2. Second, event processing requires technology. There’s a reason why products such as Apama exist. It’s not easy or possible without them. There’s a lifetime of R&D, experience and, in the case of Apama, patented technology to simplify your job. You’ll spend more time picking off the events that matter most and leave the infrastructure to us.

Whoever makes the right decision first – wins. And, this is something you can plug into what you have now. Go for it.

Replay the webinar >

Mission Operational Responsiveness: Progress RPM and Fraud Prevention

Imagine that you’re a Fortune 500 diversified bank. You have millions of customers worldwide using your credit cards to make purchases every day.

As a diversified bank, your customers expect great value and convenience every time they complete a transaction with you. To retain existing customers and obtain new ones, you are committed to delivering a convenient experience that meets your customers’ high expectations. Whether they have a credit card or savings account, you want their interactions to be seamless as they occur through your website, at ATMs, through merchants and in your branch locations.

Sometimes, things go wrong.

Diversified banks must balance delivering a convenient experience that meets customers’ expectations while continuously monitoring transactions in an effort to prevent bank fraud.

Meet Agent O and relive his fraud prevention adventure as he strives to monitor millions of transactions, when suddenly things go awry. Watch him combat the Syndrome crime family when they hack into ACME Trading Company’s network, stealing thousands of credit card numbers.

Will the Progress® Responsive Process Management (RPM) suite help Agent O stop the crime in time? Watch the video to find out, as the RPM suite is put to the test.

And then tell Progress how you can use RPM to become a business hero and enter to win an Apple iPad.

Information Overload? Three Ways Real-time Information is Changing Decision Management

Upcoming webinar: 26-Aug-2010 at 11am ET
More Info and Registration: http://bit.ly/dyCu0c

JOIN US! In a recent survey by Vanson Bourne, 94% of respondents say that responding to information immediately is critical to their business, yet only 8% currently report information in real time. Why? Because many are unable to filter through the enormous deluge of information coming at them from every angle. This web seminar will discuss 3 ways that real-time information and business events affect your decision-making process. Through real-time visibility and immediate sense and respond, you get the right information at the right time. The result is decision management that is “in the moment” and operational responsiveness that meets the speed of your business.

Hope you can attend: http://bit.ly/dyCu0c

Beware the weight-challenged digits

Fat fingers (or weight-challenged digits - for the more politically correct :-) ) are trading errors that can have catastrophic consequences. And they've been happening a lot recently!!

Read my full blog post here.

India: Big Potential for Algorithmic Trading

I spent last week in India, a country that, by any standards, is growing fast.  Its population has doubled in the last 40 years to 1.2B and economic growth has averaged more than 7% per year since 1997.  It’s projected to grow at more than 8% in 2010. By some measures, India has the 4th biggest economy in the world. 

Progress Software has a significant presence in India. In fact, people-wise, it’s the biggest territory for Progress outside the US with over 350 people. Hyderabad is home to a big development centre and Mumbai (Bombay) has sales, marketing and a professional services team.

The primary purpose of my visit was to support an event Progress organised in Mumbai on Thursday of last week on the subject of algorithmic trading. It was also our first real launch of Progress and Apama, our Complex Event Processing (CEP) platform, into the Indian capital markets. We had a great turnout, with over 100 people turning up. I spoke about what we did in capital markets and then participated in a panel session where I was joined by the CTO of the National Stock Exchange, the biggest in India, a senior director of SEBI, the regulator, and representatives from Nomura and Citigroup. A lively debate ensued.

The use of algorithmic trading is still fairly nascent in India, but I believe it has a big future. I’ll explain why soon, but I’d like first to give some background on the Indian electronic trading market, particularly the equities market, which is the largest.

Watch my Interview on NDTV Profit >

The market: India has several, competing markets for equities, futures and options, commodities and foreign exchange too.  In equities, the biggest turnover markets are run by the National Stock Exchange (NSE) and the Bombay Stock Exchange (BSE), with market shares (in the number of trades) of 74% and 26% respectively. Two more equity exchanges are planning to go live soon – the Delhi Stock Exchange is planning to relaunch and MCX is also currently awaiting a licence to launch. This multi-market model, only recently adopted in Europe for example, has been in place in India for many years.

It was only two years ago that direct market access (DMA) to exchanges was allowed. Although official figures don’t exist, the consensus opinion is that about 5% of volume in equities is traded algorithmically and between 15% and 25% in futures and options. Regulation in India is strong - no exchange allows naked access and the BSE described to me some of the strongest pre-trade risk controls I’ve come across - collateral checks on every order before they are matched. The NSE has throttling controls which imposes a limit on the number of orders a member organisation can submit per second. Members can be suspended from trading intra-day if this is exceeded. The NSE also forces organisations who want to use algorithms to go through an approval process. I’ll say more about this later. Controversially, the NSE will not allow multi-exchange algorithmic strategies so cross-exchange arbitrage and smart-order routing cannot take place. Lastly, a securities transaction tax (STT) is levied on all securities sales.

So, with the above restrictions, why do I think that the Indian market for algorithmic trading has massive potential?

The potential: The Indian market is very big. Surprisingly so to many people. Taking figures from the World Federation of Stock Exchanges (thus I’m not counting trading on alternative equity venues such as European multi-lateral trading facilities), the Indian market, in dollar value, may still be relatively modest – it’s the 10th largest. However, when you look at the number of trades, India’s the 3rd largest market, only beaten by the US and China. The NSE, for example, processes 10 times the number of trades as the London Stock Exchange. So why isn’t more traded in dollar terms? That’s because trade sizes on Indian exchanges are very small. The median figure worldwide is about $10K per trade. The figure in India is about $500 per trade, a 20th of the size. In summary, surely the task of taming the complexity of this number of trades and the orders that go with them is ideal for algorithmic trading to give an edge? To compare to another emerging, “BRIC”, economy, that of Brazil, where the number of firms using Apama has gone from zero to over 20 in as many months, the dollar market size is fairly similar but the number of equity trades in India is 33 times more. The potential in India is therefore enormous.

India is already there in other ways. All exchanges are offering co-location facilities for their members and debate has already moved on to that common in more developed markets on whether this gives certain firms an unfair advantage or not and whether co-location provision should be regulated.

The challenges: There are some difficulties. The STT is seen by some as an inhibitor. However, its effect is offset somewhat by the fact that securities traded on exchange are not subject to capital gains tax.

The NSE process for approving algorithms is more controversial. Firms that want to algorithmically trade must show to the NSE that certain risk safeguards are in place and “demonstrate” the algorithm to the exchange. As the biggest exchange, the NSE wields considerable power and thus its decision to vet algorithms puts a brake on market development. I believe this process to be unsustainable for the following reasons:

1. As the market develops there will simply be too many algorithms for the NSE to deal with in any reasonable timeframe. Yes, India is a low-cost economy, but you need highly trained people to be able to analyse algorithmic trading systems. You can’t simply throw more people at this. Firms will want to change the way algorithms work on a regular basis. They can’t do this, with this process in place.
2. It raises intellectual property issues. Brokers will increasingly object to revealing parts of their algorithms and their clients, who may want to run their alpha seeking algorithms on a broker-supplied co-location facility, will most definitely object.
3. It puts the NSE in an invidious position. Eventually an algo will “pass” the process and then go wrong, perhaps adversely affecting the whole market. The NSE will have to take some of the blame.
4. Competition will force the NSE’s hand. The BSE is trying to aggressively take back market share and other exchanges are launching which will not have these restrictions.

It strikes me that the NSE should spend its efforts into ensuring that it protects itself better. Perhaps a reasonable comparison is a Web site protecting itself from hacking and denial of service attacks. If they can do it, so can an exchange. And it would offer much better protection for the exchange and the market in general.

In conclusion: I’m convinced of the growth potential in India for algorithmic trading. The market is large, the user base is still relatively small and many of the regulatory and technical prerequisites are in place. There are some inhibitors, outlined above, but I don’t think they’ll hold the market back significantly. And finally, why should India not adopt algo trading when so many other, and diverse, markets have?

Progress has its first customers already in India. I look forward to many more.

Defending Against the Algo Pirates

It was an honor to sit on the CFTC Technology Advisory Committee (TAC) last week. The CFTC is the US regulator for derivatives trading – which is a big area. The topic of the first meeting of the TAC was to understand Algorithmic and High Frequency Trading – and its impact on the market. The CFTC is sensibly trying to understand where they should regulate and where they should propose best practices. Obviously the memory of the "flash crash" is still fresh in everyone's mind. One of the topics of interest was that of pirate algorithms that attack other algorithms and try to manipulate the market. I’ve just written a blog posting on it if you’d like to read more:

Full Blog Post: Defending Against the Algo Pirates

Responsiveness and the Customer Experience: Business Events

I recently did a webinar, Responding to Business Events as they Happen, where I discussed the relationship between business events and the customer experience. The catalyst came from a few of our customers where they’re definitely getting the benefit of bridging customer events with customer transactions. In most cases, they recognized that there are specific measures (or KPIs) that suddenly become more powerful if they can be transitioned into real-time instead of past-tense values. Finding out tomorrow how you did yesterday is too late for many customer experience “episodes.”

Today’s customer experience lifecycle can be pretty complicated. You’ve bred in multiple touch points to the lifecycle where a customer can enter and exit anywhere and at will. What’s lost is the common thread of control of a managed lifecycle. That makes sense for today’s business climate, but you’re left with business systems and infrastructure from an era that doesn’t fit the requirements of the new lifecycle. If you enable your customers to “serve themselves” then you need to develop your experience strategies to meet their expectations—oh, and you have to do it with what you’ve got since there’s no way you can replace your existing systems. No problem.

It’s all about events and your ability to sense and respond. The customer experience lifecycle is affected by customer events—throughout the lifecycle from prospect to on-going service and support. The customer can enter and exit anywhere, anyplace and at anytime. In this case, the customer experience cannot simply be managed from a single, siloed point such as customer service or a website. You need visibility into all potential entry/exit points where a customer may interact with your enterprise. How they interact with you is just as important as how they use your products. Much can be gained by correlating their interactions and usage.  And if you can manage all of this in real-time, you can be proactive in your response—before it’s too late. Business Event Processing (BEP) allows you to capture utilization and interaction events across the customer experience lifecycle. You can then turn those events into something that is meaningful and actionable. So not only can you get broad visibility, you can do something about it.

In a world where churn is costly, you can’t afford to let someone click somewhere else.

Next up: The Customer Experience: Use Cases and Examples

Operational Responsiveness for People

Last week met with a journalist (Dy.Editor, Murali) from a leading business newspaper in India (Hindu Business Line). We talked about many things that are at the top of my mind now- software products, future of IT industry, startups, middleware technology trends and, needless to say, Operational Responsiveness and our Responsive Process Management (RPM) strategy. Like I have been doing a lot lately, highlighted the visibility and control that RPM is enabling in a non-intrusive manner retaining all existing apps & solutions as is and still get the needed visibility across biz processes that span these applications. And get this without re-engineeirng any application. (A promise that has long been made by SOA but something that is not easy to realise without massive re-engineering to get services and automated business processes in place. So first order problems are solved by getting any re-engineering needed in place. But an organization's visibility and control needs are much beyond these first order problems).

As I was engrossed in explaining the value from RPM, this journalist posed a question from a completely different view. Asked if there is any way to track Operational Responsiveness of people? Anyway to get real-time visibility into the people part of the organizations. Not the time-sheet type tracking of who is doing what and for how long- but more of a semantic and collaboration enabling type of tracking. Of trying to capture who is "thinking" about what at any given point. And use this information to trigger collaboration. If I see a colleague of mine is presently thinking about a particular problem area, and I have inputs, I can share it knowing that he/she is going to consider/consume it right away.And do this in an RPM manner, with visibility, alerts and workflows.

Now, this was an interesting thought. I know organizations have been working on knowledge management and collaboration approaches and tools for long. In spite of very widespread penetration of Wikis, networking tools and social style collaboration in organizations, still not easy to realize the collaboration organizations could use. Often different people go thru the solving same problems each solving it again without realising someone has already solved it. Would Operational Responsiveness help here? Can the needed "visibility" into people and what they are presently working on be enabled? And then, build the "responsiveness" and "control" (read, collaboration here) capabilities. This was the gist of the journalists thought. Very interesting. Especially when he brought this up as I was talking about how RPM enables seamless and effortless visibility and control in an organization (essentially machines, applications, business functions, processes and other manifestations of machine processing- but none involving people in the organization and what they are presently thinking about!).

Shipping logistics with event processing

On Friday last week I visited Progress customer, Royal Dirkzwager, in the Netherlands. It’s nice to be able to talk about a customer publically - Dirkzwager have been a generous public reference for some time. This was my first visit to their head office in Maassluis, near Rotterdam, and it gave me a new insight into their business. Dirkzwager’s offices are right on the main entrance to the port of Rotterdam and it was great to be in their board room and watch the container ships move by as Paul Wieland, head of IT, took me through their business and what they are using Progress’ products for.

Dirkzwager is in the business of supplying information about shipping movements to the maritime industry. Its customers include the Port of Rotterdam, shipping companies and the logistics companies involved in supporting the ships and handling the goods. Dirkzwager currently supplies information on shipping movements at sea and in and out of ports from Le Havre in France to Denmark. Rotterdam is its epicentre - the biggest port in Europe and one of the biggest in the world, handling around 30,000 ships per year.  

Dirkzwager are users of the Sonic Enterprise Eervice Bus (ESB) and the Apama event processing platform. Sonic is used for delivering information to customers reliably and Apama is used for processing and analyzing the events reporting the positions of ships, up to several thousand per second. Live data is augmented with data from a reference database on tonnage, flag, crew, and lots of other information. Applications range from simple to more complex. One example is: “alert when a ship crosses this line”, to give an official port entry time (important for calculating harbour fees). A more complex example is the calculation of an actual time of arrival, using course, speed, weather conditions and historical information about the ship’s movements. This latter application can bring very significant benefits: on average, 50 companies are involved servicing a ship when it’s in port, from loading or unloading goods to servicing the radar. If these companies can receive more accurate information about a ship’s arrival time, they can optimize their operations resulting in the whole supply chain becoming more efficient – capacity is increased and costs are lowered. Paul gave some insight too into how Dirkzwager had become more efficient by using Apama to automate tasks that were previously done manually and in the process saving substantial amounts of money (the details of which, unfortunately, I can’t share here).

Dirkzwager’s industry is changing. Currently, most ships identify themselves by a VHF-radio system. This is shore based and has a range of approximately 60km. This is changing to a satellite based system which will give more continual and accurate reporting and also give simpler access to ship movements anywhere in the world (approximately 80,000 ships are in transit at any one time worldwide). This gives business expansion opportunities and also challenges when dealing with the new quantities of data that available. Paul Wieland clearly believes that complex event processing (CEP) is an ideal way for Dirkzwager to analyze this data and provide the monitoring of logistics and supply chain processes that they support for customers. Paul’s obviously a visionary – the room was buzzing with ideas. Now his team is experienced with Apama, Paul reported that people’s approach to problem solving had changed – they were thinking naturally about things in an event based way, both architecturally and in terms of data processing.

One might not necessarily think of shipping logistics as being the most innovative of industries, but Dirkzwager has always been at the forefront of technological innovation. Shipping was, and continues to be, vital to the Dutch economy. When the company was founded, in 1872, people used to stand at the opening of the port with binoculars. As a ship was sighted and recognized, someone would mount a horse and rush back to the port to report. Dirkzwager had the first commercial telephone line in the Netherlands. Paul showed me a room where some of the computing and communications equipment used previously by Dirkzwager is stored – examples include a megaphone, semaphore flags, telephones, telex machines and a mini computer. It was a mini museum for the communications industry. The use of event processing software is simply another step in this historical evolution.

What do you do with the drunken trader?

The news that Steven Perkins, (former) oil futures broker in the London office of PVM Oil Futures, has been fined 72,000 pounds ($108,400) by the FSA and banned from working in the industry is no surprise, see article here.

It could have been worse given that the broker, after a few days of heavy drinking, took on a 7.0 million barrel long position on crude oil in the middle of the night. The fine seems miniscule since it cost PVM somewhere in the vicinity of $10 million - after unwinding the $500+ million position.

The surprising thing about this incident is that it happened at all. Perkins was a broker, not a trader. He acted on behalf of traders, placing orders on the Intercontinental Exchange among other places. That he could go into the trading system and sneak through 7.0 million barrels without a customer on the other side is unbelievable.

Heavy drinking is practically a job requirement in the oil industry, my sources tell me, so this kind of thing could be a real issue going forward. As algorithmic trading takes hold in the energy markets, trading may approach the ultra high speeds seen in equities markets.  This is a recipe for super high speed disaster, unless there are proper controls in place - especially if there were a way for the broker or trader in question to enrich himself in the process.

One powerful way to prevent this kind of accident or fraud is through the use of stringent pre-trade risk controls. The benefits of being able to pro-actively monitor trades include catching "fat fingered" errors, preventing trading limits from being breached, and even warning brokers and regulators of potential fraud - all of which cost brokers, traders and regulators money. PVM is a good example of this.

Ultra-low-latency pre-trade risk management can be achieved by brokers without compromising speed of access.  One solution is a low latency "risk firewall" utilizing complex event processing as its core, which can be benchmarked in the low microseconds.  Errors can be caught in real-time, before they can reach the exchange. Heaving that drunken trader right overboard, and his trades into the bin.

Banks & Bullets: Maybe They Dodged One - But Still Needs Some Silver Ones!

By now you’ve probably seen that a deal was reached this morning by the House and Senate on regulation. Some would say it waters down provisions from the tougher Senate bill, limiting rather than prohibiting banks to trade derivatives and invest in hedge funds. This articles describes it as banks “dodging a bullet” http://www.businessweek.com/news/2010-06-25/banks-dodged-a-bullet-as-u-s-congress-dilutes-trading-rules.html

We applaud the superhuman efforts put into the new financial regulation bill by the U.S. Congress and the House of Representatives. However, as I’ve said many times, transparency and consistency are critical to successful regulation in the capital markets. One could be forgiven for fearing that the watering down of the regulations, including the Volcker Rule, may create more havoc rather than increase transparency and consistency.

One thing is for sure – there’s going to be a raising of the priority on handling the complexity and requirement for real-time risk and surveillance within institutions. Risk managers and C-level executives concerned about minimizing risk and maximizing capital will need to view trading positions and limits across the firm, including, if permitted, derivatives that are 'spun out'. Ideally the risks should be aggregated and analyzed, in real-time, giving the ability to detect and prevent “accidents”. Pre-trade risk management will be increasingly important, as firms seek to maintain capital requirements at all times.

A top-down approach to risk where managers can see, in a single view on a dashboard, the risks across all asset class silos has gone from a “nice to have” to high on the wish list – but many still wonder if it actually possible. Continual monitoring of trades in real-time can help to prevent exceeding trading limits, prevent mistakes and catch market abuse.

p.s. Many thanks to all the comments from market practitioners who comment that technology can't solve all the problems for Regulators, Banks and Trading Venues. I completely agree! But we can go a lot further than what happens right now. But of course technology is only one of the approaches. Changes in regulation is another, as is increased transparency, improved reporting (e.g. from fax to real-time data!) etc. etc. 

Rogue Trading Below the Radar

Jerome Kerviel, the trader who allegedly lost Societe Generale nearly 5.0 million euros, went on trial in Paris on Tuesday, June 8th. The bank alleges that Kerviel took "massive fraudulent directional positions" in 2007 and 2008, which were far beyond his trading limits.

It is interesting to note that Kerviel was not only experienced on the trading floor, but he also had a background in middle office risk management technology. It may have been this knowledge that enabled him to manipulate the bank's risk controls and thus escape notice for so long.

Still, it is perplexing that fraud on such a scale can go on without detection for so long, even if Kerviel did have an insider's knowledge of the firm's risk management systems. Internal risk controls are not something that a financial firm can take for granted, left to run unchecked or unchanged for months or years.

The detection of criminal fraud or market abuse is something that must happen in real-time, before any suspicious behaviour has a chance to lose a firm money or to move the market. Pre-trade risk management is paramount, with trading limits specified and checked in real-time. Internal controls should be monitored for possible manipulation, again in real-time. The good news is that technology does exist in the form of real-time market surveillance software from companies can analyse data transactions by the millisecond.

Financial institutions need to start looking inward to improve standards, regardless of current regulation. Otherwise the culture of greed and financial gain at all costs will encourage more and more Kerviels.

HOW MATCH2BLUE MAKES PROGRESS

The challenge match2blue faced… they needed to scale their existing location-aware social networking platform with an event-driven architecture that could support millions of users. They accomplished this by using Progress® Apama® (http://bit.ly/apama-bep). Today match2blue can scale to meet the needs of any organization. They’ve enabled their customers to deliver real-time information to users, and they gained the flexibility to customize solutions – all with faster time-to-market. Learn more about match2blue: http://bit.ly/prgs-match2blue

Tell us how your company is achieving operational responsiveness using an event-driven architecture!

Live Webinar with IDC's Maureen Flemming Tomorrow

When all you can do is analyze what HAS happened instead of what IS ABOUT to happen, you are missing out on important opportunities. What you really need is to get actionable insight so you can gain total control of your business. Join us tomorrow for the webinar Your Business Moves Faster When You’re At The Controls and hear IDC analyst Maureen Fleming provide more insight into how to gain comprehensive insight and continuously improve business processes by managing change more effectively.

Join the Live Event:
Thursday, 22 April 2010
11:00am EST / 4:00pm GMT / 17:00 CET
 >  Register here

Archived event:
Friday, 23 April 2010
Time: 9:00am Sydney Time
 >  Register here

Real-time. Do you take it for granted?

I can’t help but notice all the references to “real-time” lately—real-time AWD, real-time ABS, real-time computing, real-time communication, real-time arts, Real Time with Bill Maher, etc.—I even clicked on realtime.com… I got the message “Http/1.1 Service Unavailable” and the only thing I can say is that I was annoyed, in real-time! Seriously, if you search on “real-time”, Google estimates 1,850,000,000 results.

The use of real-time isn’t new to me. I’ve been listening to Progress talk about it since earlier than 2004. Progress had the bad (at least I think so) habit of organizing their development, marketing and sales efforts around how we are organized as a company. Sometime around 2004 the division that I worked in was actually called the Real-time Division (I have a hat to prove it). We went so far as to structure our website under www.progress.com/realtime/. This division developed, marketed and sold our ObjectStore and Apama product lines. Today we’re organizing ourselves more closely to solutions and products so when people tell me about a “page not found” when they visit someplace under ~/realtime, a shiver runs up my back and gives me a headache. For anyone who’s worked on the web side of the business, change is not always good.

Anyway, I digress... In some cases, I do take “real-time” for granted. When I press the brake of my car, I expect my car to stop in real-time—even if I really get thrown into a tailspin because of all the rain around here. I bought an AWD car a few years ago and I’m apparently naïve because I don’t know of a button I need to push to have real-time AWD. Real-time computing, now that’s a different story. I bet many technologists might not take “real-time” for granted. Do you? Wikipedia notes that early references to real-time computing were in reference to high performance. When the Real Time Division first started talking about “real-time”, it was related to event processing—complex event processing—the ability to monitor, analyze and act on business events as they occur. Behind most transactions is a database and as most of us know, updating a field/row of the supported database isn’t always real-time. It may only take a second but we all know what a second can feel like when we’re trying to win the Gold, selling or buying stock at the best price, or reacting to the brake lights blasting in front of us. Apama’s technology doesn’t wait for the database to update, it “automagically” processes the event WHEN IT HAPPENS—learn more about Apama.

Last week we announced the launch of the Progress® Responsive Process Management (RPM) suite. This solution suite presents “real-time visibility” as a core feature/benefit of the Progress RPM suite. When I first started reading the content for the launch, I associated Real-time Visibility to our Apama technology but it’s actually associated with multiple Progress technologies—Apama (for visibility into events), Actional (end-to-end visibility into transactions) and Savvion (visibility into processes). It made sense, at least to me, because it’s about “seeing” how transactions are operating as they occur—from design time through runtime. In this case, I do not have to take real-time for granted, and it’s not automagic, because I’m looking at a dashboard that’s showing me exactly how an event is being processed. With Real-time Visibility, I can see (visualize) how the transaction is flowing and if there is a problem I can immediately respond and work to find a resolution.

To be honest, nothing I do is really time or mission-critical. And while I take for granted that my CMS is publishing our newest press release to our home page the minute it gets published, our marcom VP is not. While he’s tapping his foot behind me, I’m wishing I could bring up my “real-time visibility” console and see what’s causing the delay—is it the CMS, the network, what? In industries like capital markets, energy, insurance, and telecommunications, real-time visibility is the difference between increased revenue vs lost revenue, and happy customers vs miserable customers. With Real-time Visibility, you don’t have to take anything for granted.

It's About Quality... End-to-End Process Quality

Hey everyone! I haven't written in a while because I'm still getting used to my new role and am freshly back from a nice two-week vacation. Why does all the juicy stuff get released while I'm away? Anyway, I'm sure it's not about me (this time). [Apparently I have a lot to say! Sorry for the length of this post in advance. The short version is: 1) It's all about improving "business quality", and 2) Make sure to read the Gartner blog referenced below.]

I wanted to weigh in on this RPM thing... Last week we announced the launch of our Responsive Process Management (RPM) suite. What's it about? I'll leave the "brochure-ware" posts to the marketing folks. I have been talking to many of our customers and field, and while there's some confusion about what it means... RPM's benefits are quite obvious.

It's all about the "quality of the business".

Now marketing may not like how I leave out all sorts of important words/phrases... like real-time business visibility, immediate sense & respond, or continuous process improvement. Those concepts are all very important but they sound very familiar.

What differentiates the Progress solution is that we're enabling, quickly and layered over your existing infrastructure, the ability to react in real-time to your business conditions and exceptions to more tightly control the quality of your business as your customers experience your service/products.

Don't get caught up in the words. Progress' offering is new (and IMO innovative). I know, the jury's out and we have to prove that. But trust me on this one... at least long enough to let me explain.

Let me start by pointing out that all other vendors that "already do that"... well they don't. Don't believe me? Believe Mark McDonald at Gartner. In a blog from February 2010 he points out:

"... in 2010 more than half of CIO's do not feel confident in their ability to achieve results when improving business processes."

If all these other vendors have had a solution for this problem in the market for five years, but 50% of CIO's still don't trust it, something's wrong with the solution.

We can argue this point back and forth for hours, as vendors, because we love our solutions and it's a totally unproductive waste of time. Of course I (usually) believe my stuff is better than others. If not, I should just go get a job with the other guys. Right?

So, let me share some personal thoughts on process quality, what it really means (from a consumer's perspective, my own experience with a bank on a recent transaction), and it's importance will be obvious. I'll not attempt to prove that the Progress® RPM suite is better than the others here... but I hope you'll leave with an understanding of our intent and some of the less obvious challenges companies face when trying to solve problems with their business process quality.

I recently used a government program to refinance a home through Wells Fargo Bank. It's a very paper heavy process. I had the original mortgage with Wells Fargo and since it was a government program, there wasn't any variability to the process. It's a coastal property, so they need to verify up-to-date flood insurance. Of course, since they held the original note on the property -- they had up-to-date flood insurance proof on file.

A few things struck me right away:

1. Even my mortgage broker had no idea what the process was like because it was a government program. He told me that he'd submit the information, and eventually something would happen.
2. The person who eventually called (couple weeks later) and asked for my insurance certificate (which, remember, they had on file -- and, yes I could have canceled the insurance, but I sent them the same proof they had on file, so my point stands.) didn't leave a phone number. Or an application tracking number. Or an email address. Just a fax number and a first name. It took me a few days to get to a fax machine, on which I left my phone number and email asking for a confirmation of receipt because I wanted to make sure we completed this step.
3. I got another call (from another person) to discuss the process and confirm the final information. The inbound caller ID didn't show my bank's name, so it went to voicemail. It took me a couple of days to return the call during which she called again. When we finally spoke, she handed me off to yet another number/person who I'd have to call if I had any questions when I received the closing package.
4. The closing package asked for the insurance certificate again. And, the loan amount was higher than my current balance. The HUD-1 I got had the same numbers, but they weren't explained anywhere. Having all the dysfunctional experiences above, I had very little confidence in the process and almost decided to scrap it. I had to call the call-center again. When I did they quickly and clearly answered my question and confirmed that they had the insurance info so didn't need it again.

Why did these things stick on my mind? In large part because they're unnecessary and affected my experience with the bank. Do I really want a company this sloppy and disorganized in charge of my money?

The process worked. But, the quality of the process left a lot to be desired. In left me feeling uncertain (as I mentioned, I almost didn't proceed) and it was inefficient for them. Multiple calls to the data center, answering questions that everyone would have and where the answers were readily available. And, before you think it's sloppy human process (the part about not leaving a return phone number with questions, etc.)... there are tells that can be used to recognize poor human process too (like tracking how many calls to the call-center occur per mortgage, and how long the average response time is to each time they ask me for information... and whether there is a correlation, etc.).

Process Visibility

Somewhere, someone in the bank knew the process. They had to. But that information wasn't shared-with or accessible-to my broker. He submitted the paperwork and told me I'd hear from someone.

If you worked with two banks... one said "we'll call you soon" and the other said "you'll hear from us by Tuesday, after that you'll get a package in the mail that you'll have to get notarized and return, the whole thing should take about 6 weeks, so by March, you'll be on the new rate" which would leave you with more confidence? Which is the better experience?

Companies need to be able to track the end-to-end business process. And, as Mark McDonald points out, they need to do that regardless of whether the process is embedded in their legacy systems, formally orchestrated in a BPM engine, or ad-hoc across distributed services (or all of the above!).

Frankly, it must be really hard to do, or more companies would do it. Was this process hard to track because it was part of a government program? Was it handled by different "systems" or teams for compliance reasons? Was there an external government processor that the bank integrated with and left the bank without a clear understanding of the SLA? Was the technology just not agile enough to meet the speed of the business? Said differently, the government was able to roll-out a refinance program faster than Wells Fargo's IT team could react, leaving Wells Fargo hanging with both the government and their frustrated customers. We all know the government is slow... is Wells Fargo's IT really slower?

Flood Insurance

This one baffles me. Aside from the fact that they already had the information they needed, it took me a few days because they wanted it via fax. I was uncertain because I thought maybe they needed something different than what they had, and was frustrated because I had no way to communicate with the person requesting the information.

I'm pretty organized. All my paperwork is scanned and searchable on my computer (and backed up!). Had I received an email request for information they would have had it in 5 minutes. Instead it took 5 days. That sounds like a compelling optimization. Someone has to care how quickly these things get processed (besides me and President Obama). Wasn't one of the early criticisms of these programs that they weren't being implemented quickly enough? After reading this story, do you wonder why?

Loan Amount

I was told it was a no-fee process, but the loan amount was way too high. I figured the bank didn't know what they were doing. A reasonable assumption based on my prior experience with the process. My call to the call center cost the bank money, and took more time. In truth, it was two calls. The first time I called they were too busy, and I waited then had to hang up. Why not just explain it clearly up front? Do they even know how many customers call with questions once receiving the paperwork? Do they know the length of the calls? Perhaps they can see the length, realize it's a quick question, and see if they can improve... again, it's not about the people or thought that goes into things. It's about sensing current business activities, and responding to patterns to improve the quality of the business.

Summary

Finally, I'm actually annoyed that I had to call the bank to see what the rates were. In fact, the bank should be scared of that as well. Imagine if I had called a competitor instead? Apparently the federal program I qualified for had been available for a while. Why wouldn't the bank have reached out to me? It would have impressed me, but also ensured I didn't call that competitor. In fact, this is a use case I worked on recently with an insurance company (different process, same idea).

Imagine being able to sense your customers' behavior and respond to their needs before they have a chance to look at a competitor?

Understanding the quality of the end-to-end business process and having the ability to take action based on events in real-time... that's Operational Responsiveness. It elevates the game and creates a barrier to entry against competitors. That's what the Progress® Responsive Process Management (RPM) suite is meant to deliver for our customers.

PRGS Announces the RPM Suite - A Convergence of BTA, BEP, and BPM

It’s been raining here in Massachusetts for 3 days straight, but today I got to add a NEW acronym to my arsenal...

Introducing RPM - Responsive Process Management

In case you missed it, a Progress Software press release went out this morning that announced the launch of our NEW Progress Responsive Process Management (RPM) suite. The Progress RPM suite brings together our best-in-class solutions for Business Process Management (BPM), Business Transaction Assurance (BTA), and Business Event Processing (BEP) (most commonly referred to as Complex Event Processing (CEP)). The Progress RPM suite will enable enterprises to achieve a higher level of business performance than previously possible. It is scheduled to launch in late April, and the market opportunity for this type of solution is expected to be greater than $10 billion [based on IDC Research*]. The release includes a quote from Maureen Fleming, program director of IDC's business process management and middleware research service:

“Over the past two years, one of the fastest-growing areas of software investment by enterprises has been to improve their situational awareness. Logically, the next step is broadening the focus to not only gain visibility into problems or opportunities but to rapidly respond. Enterprises will increasingly look for vendors that offer a knowledgeable and comprehensive approach to building this next generation of critical business applications."

It may sound like a pretty complicated implementation but core to the Progress Responsive Process Management suite is the Progress Control Tower™, a unified product dashboard, or GUI, that displays real-time alerts, interactive interfaces and tools. The Control Tower will provide users with the ability to view what is happening within their business and to improve it from a single source - thereby gaining greater ROI. It’s fully configurable, feature-rich, interactive framework delivers a wealth of relevant, KPIs and business information. What’s more, a powerful modeling environment enables new business processes to be rapidly created, modeled, monitored, controlled and improved dynamically.

To learn more, read our 7 page brochure and visit our website. We've also written the white paper Achieving Operational Responsiveness Through Responsive Process Management that you can register for.

Over the coming months, we’ll introduce more collateral, white papers, and webinars so stay tuned.

TTYL!

Rumblings in the Cloud

Cloud computing... it's on everyone's mind these days. Personally I think it's a term that has attained such aggrandized acclaim that vendors, analysts, bloggers and anyone with marketing muscle has pulled and stretched its definition to such and extent that it could mean just about anything hosted. Cloud Computing Journal polled twenty-one experts to define Cloud Computing.  Just the fact they had to ask the question of twenty-one experts is rather telling in itself.  Well I read what the experts had to say.

So armed with my newly minted (yet fully stretched, but not of my own making) Cloud definition I happened upon this commentary about CEP in the Cloud or the lack thereof.  There's a great quote in the article: "I don’t care where a message is coming from and I don’t care where it’s going”. Correctly indicated, this in a sense defines a key aspect of CEP. Event-based applications should be transparent to messages (or events to which messages transform) origin and destination (sans a logical or virtual name).  However, unlike the author Colin Clark, I do believe the current crop of vendor products, most notably Progress Apama maintain this separation of the physical from the virtual.

The rationale behind the lack of CEP-based applications in the Cloud (ok, there's that word again) are found in other factors. To explain my reasoning I'll start by dividing CEP-based applications into two categories. Of course there are many ways to categorize CEP-based applications, but for the sake of this discussion, I'll use these two:

CEP-based Application Categories

1. Those that do things
2. Those that observe other applications doing things

Not sure I could make a simpler layman-like description, but needless to say it warrants further explanation (or definition in sticking with our theme)

CEP-based applications that do things

This category is best explained by example. Typical of event processing applications that do things are those in Capital Markets like algorthmic trading, pricing and market making. These applications perform some business function, often critcal in nature in their own right. Save connectivity to data sources and destinations, they are the key ingredient or the only ingredient to a business process.  In the algo world CEP systems tap into the firehose of data, and the data rates in these markets (Equities, Futures & Options, etc.) is increasing at a dizzying pace. CEP-based trading systems are focused on achieiving the lowest latency possible. Investment banks, hedge funds, and others in the arms race demand the very best in hardware and software platforms to shave microseconds off each trade. Anything that gets in the (latency) way is quickly shed.

In other verticals, an up and coming usage of CEP is location-based services. This is one that leveraging smart mobile devices (i.e "don't care where the message is going") to provide promotions and offers.  

• Algo Trading, Pricing, Market Aggregation
• Location Based Services (providing promotional offers and alerts)

CEP-based applications that observe other applications doing things

Conversely, event-based applications that observe other applications doing things are classified as providing visibility or greater insight into some existing business function. These event-based applications overlay business processes to take measures to improve their effectiveness. As is often the case critical business applications provide little visibility or the information is silo’ed. There is a need to provide a broader operational semantic across a heterogeneous mix of business applications and processes.  Here are a few typical examples of event-based visibility applications observing other business systems.

• Telco Revenue Assurance
• Click Stream Analysis
• Fraud Detection
  
• Surveillance
  

Of  course the demarcation line between these two classifications is not clear cut. Providing greater visibility is just a starting point, monitoring for opportunities to take action is just as important such as kicking-off a fraud watch if a suspected wash-trade occurred  (so in a sense they are doing things).

Where for art thou oh CEP
When considering the Cloud, an important point to consider is dependency. Specifically, there is a dependency that the underlying applications and business processes exist in the Cloud for (observing) CEP to overlay them.  I would offer that Enterprise business has not yet migrated their key business processes to the Cloud on a widespread scale just yet. Why not? What are the barriers? Security, regulatory compliance, DR, investment costs, limited skill sets are just a few of the challenges mentioned in this ITProPortal article.  I suspect these barriers are far reaching, keeping the pace of Cloud deployment in check to the point where it's not as yet strategic to many.
 
One of key things that makes the Cloud a reality is virtualization, it has clearly revolutionized PaaS as the Cloud. Virtualization does come at a cost, there is a latency penality for the conveinence, no matter how small for some use-cases that cost is too great.

Make no mistake, I am certain the Cloud with all it's twenty-one definitions is the future of computing. It's an imperative that will knock down the barriers and change the face of the Enterprise and when it reaches critical mass CEP will be there.

Once again thanks for reading, you can follow me at twitter, here.
Louie

. 

Peas and Carrots

In the words of the auspicious Forrest Gump some things go together like peas and carrots. Truer words were never spoken. Some things just do go together well, sometimes by design, often by accident. I don't think anyone actually planned milk and cookies or popcorn at the movies but nonetheless these things are made for each other.  When it comes to technology the same harmonious relationships exist.

In the recent Aite report on High Performance Databases (HPDB),  the market for specialized databases is surveyed along with a handful of vendors in this space.  This is a cottage industry where the big database vendors don't play. It's hard to imagine in this day and age where database technology is so standardized and mature and a multitude of choice abounds from commercial products to open source that any other database technology and a gang of vendors would have a chance. Yet it is happening and it's thriving.  

I believe it has to do with a synergistic relationship to event processing. If CEP is the "peas" then HPDB's are the "carrots". These two technologies share two fundamental precepts:

•  A focus on Extreme Performance
•  Temporal Awareness

I. Extreme Performance, Speeds and Feeds
These HPDB's which are often referred to as Tick databases, are found in the same playground as event processing technologies. In the Capital Markets industry they connect to the same market sources, consume the same data feeds. Both technologies are designed to leverage modern multi-core hardware to consume the ever-increasing firehose of data. By the same token, once that data is stored on disk, database query performance is equally important.  The massive amount of data collected and is only as good as the database's ability to query it efficiently thus creating another (historical) firehose of data which an event processing engine would be the consummate consumer.  

II. Temporal Awareness, when is the data
Time is a basic principle in event processing technology, applications typically have as a premise to analyze data-in-motion within a window of time. HPDB's design center is to store and query time series data. Some of the database vendors even bring time to a higher level business function. They understand the notion of a business Calendar, knowing business hours, business week, holidays, trading hours, etc.  Imagine the simplicity of a query where you want 'Business hours Mon-Fri for the month of February' and the database itself would know the third Monday was Presidents Day, skipping over that, thus preventing analytic calculations from skewing erroneously.

Leveraging the Synergy
These two fundamental shared principles provide the basis for a unique set of business cases that are only realized by leveraging Event Processing platforms and High Performance Databases

• Back testing algorithms across massive volumes of historical data compressing time

What if you could test new trading algorithms against the last 6 months or 1 - 2 years of historical market data but run that test in a matter of minutes? What if you could be assured that the temporal conditions of the strategies (i.e. timed limit orders) behaved correctly and deterministically matching the movement of time in complete synchronicity with the historical data? These are just a few of the characteristics that define the harmony between event processing and high performance (Tick) databases.

• Blending live and historical data in real-time

Querying historical data in-flight to obtain volume curves, moving averages, the latest VWAP and other analytics calculations are possible with these high performance databases. Leading edge trading algorithms are blending a historical context with the live market and even News. The winners will be those that can build these complex algo's and maintain ultra low-latency.

• Pre-Trade Risk Management

Managing positions, order limits and exposure is necessary, doing it in real-time to manage market risk is a mandate.  In addition to market data, these high performance databases can store pre and post trade activity to complement event-based trading systems and become the basis for trade reporting systems.

In the Trading LifeCycle, Event Processing and High Performance databases are partner technologies harmoniously bound together to form a union where the whole is greater than the sum of the parts. They are the peas and carrots that together create a host of real-world use-cases that would not be possible as individual technologies.

Myself along with my colleague Dan Hubsher we are doing a 3-part Webinar series entitled "Concept to Profit". The focus is on event processing in the trade lifecycle, but we include cases that touch upon high performance databases. You can still register for part 2: Building Trading Strategies in the Apama WorkBench where I will focus on the tools for strategy development aimed at the IT developer.

Once again thanks for reading, you can follow me at twitter, here.
Louie

Why BPM should be on the CIO’s agenda in 2010

New article by Giles Nelson published in CIO (http://bit.ly/biSKFo) online.

"In 2010, the business prerogative across all sectors is to use IT to drive efficiency and enable a business to react more quickly to customer and market changes. To do this, I believe we need to take a different view of BPM technology and try to see how it can be used to make knowledge-based business more ‘operationally responsive', reacting to customer needs and market changes instantly. This is already beginning to happen, and as it gains momentum, BPM will prove its usefulness in bringing ‘order to the chaos', and will make it onto the strategic agenda of every CIO."

The article does a great job at illustrating the synergy between business process management (BPM) and complex event processing; or in this case business event processing (BEP). Giles even provides examples of industry's already deploying these technologies. Most of us know the benefits of BPM but as he points out... "The next stage is to match it with the other side of the coin, where it can help an organisation respond to events and become truly operationally responsive - something worthy of the full attention of any CIO." Read the full article.

CEP consolidation continues

It’s been another remarkable week. I told my wife there would be a lot of traveling for me in the first part of the year and I was right. Last week was New Jersey and New York. This week was Dallas Fort Worth and Silicon Valley. I’ve been visiting key customers, journalists and analysts.

This week has also seen further consolidation in the CEP market. I have been predicting that there could not be a stand-alone CEP market and that CEP will either find a home in applications, databases, stacks or business application platforms.  In this case Sybase has snapped up Aleri to extend its database business into the CEP domain, as well as solutions in the risk space. Aleri are a good company with good people and good products. They come from the “in memory database” perspective but developed a high performing CEP engine and learned lessons from real customers that a SQL approach is not adequate to address real applications, and embedded actions statements are need within a CEP language. Also they learned that the best way to sell CEP is not as a technology but as solutions.

I think Sybase have made a smart move – for probably a bargain price – judging by the release that says they acquire the assets only. I wish my friends at Aleri all the best for the future.

From Concept to Profit in No Time Flat – High Frequency Trading

Colleagues Dan Hubscher and Louie Lovas have begun a great webinar series that outlines the “lifecycle” of an algorithmic trading strategy.  Illustrated with Apama’s Event Modeler, they use a Commodity Futures trading strategy to illustrate how trading firms can accelerate the delivery of trading strategies with a development tool that is accessible to the trading desk. This can help make significant cuts in development times, allowing firms to capitalize more quickly on opportunities.  Future sessions will explore some of the other aspects of the platform that target developers, recognizing that firms have different business models and development styles.

We’ll be posting the recorded version shortly for on-demand viewing, but if you have not registered, I’d encourage you to click here and get on board for parts 2 and 3.

New Videos - Savvion Acquisition Part I & II featuring Dr. Bates and Dr. Ketabchi

> Link to Progress Software's YouTube channel

Listen in as Dr. Bates, CTO of Progress Software, and Dr. Ketabchi, founder and CEO of Savvion, discuss the recent acquisition, and Progress Software’s entry into the business process management (BPM) marketplace. Savvion's BPM suite is a perfect fit with our enterprise BEP and BTA solutions.

Adding Leading BPM (Business Process Management) Solution to Our Portfolio

Our integrated infrastructure (or SOA infrastructure) portfolio just got broader and better! On Monday Progress Software announced the acquisition of Savvion, Inc.  Savvion offers a comprehensive, standards-based BPM suite that helps more than 300 of the world’s top-performing companies – including 24 of the ‘Fortune 100’ – automate and continuously improve critical business processes. Dr. John Bates, Progress Software’s CTO and Head of Corporate Development, says, “The Savvion BPM suite is a perfect fit for Progress because it offers leading capabilities for business process modeling and execution. The suite also uniquely includes other integrated key capabilities, including business rules management, document management, an event engine and an analytics engine.”

Progress Software made the announcement during our Global Field Operations Conference in Orlando, FL, which is being held this week. Those lucky enough to attend were able to hear David Bressler deliver a great sales pitch that really communicated the benefit of having the industries best-in-class BPM technology in our briefcase. The combination of our Business Event Processing (BEP), Business Transaction Assurance (BTA) and Integration portfolio, coupled with Savvion's BPM suite, will enable enterprises to achieve the highest levels of operational responsiveness.

To learn more about this announcement, visit our Apama Event Processing blog and read two posts by Dr. John Bates:

• Why businesses must evolve their business processes to be highly responsive, dynamic and predictive – or they will cease to be competitive
  
  
• 10 Reasons Why Progress Chose Savvion

Welcome Savvion to the Progress family, and stay tuned for more details.

Why businesses must evolve their business processes to be highly responsive, dynamic and predictive – or they will cease to be competitive

Today Progress Software announced the acquisition of Savvion http://web.progress.com/inthenews/progress-software-co-01112010.html. I believe this heralds the beginning of a very exciting phase for Progress Software. Now Progress has become a leader in Business Process Management (BPM). But more than that, combined with our other solutions, Progress is now uniquely able to empower businesses to be operationally responsive – through responsive, dynamic and predictive business processes. And this is critical to keep modern businesses competitive.

 

You might wonder about the journey Progress went through to realize what the market needed. It was all about understanding the emerging needs of our customers and where they needed their businesses to go. The part of my job I enjoy the most is spending time with customers and understanding what pain points they have - with the ultimate goal of working with them to address the pain and making them highly competitive.

 

Over the last couple of years I have been hearing more and more from customers about the need to be operationally responsive. For example, many customers have expressed their desire to proactively – and often in real-time - address the needs of their customers and respond to the behavior of their competitors. The goals are to win new business, increase customer satisfaction and triumph over their competitors. These findings hold true whether the customer be in banking, insurance, communications, travel, transport, logistics, energy, gaming or many other industries. It could be British Airways ensuring their high value customers are looked after first in the event of a flight delay, or wireless carrier 3Italia pushing real-time offers to their customers based on their profile, activity and location, or maritime logistics provider Royal Dirkzwager dynamically adjusting the course and speed of a container ship to optimize fuel usage, based on weather conditions and port berth availability.

 

Operational responsiveness is thus about being highly responsive to opportunities and threats – and even anticipating such scenarios. Market research supports what I’ve been hearing, such as the recent survey by Vanson Bourne http://web.progress.com/en/inthenews/companies-stuck-in-o-10062009.html – suggesting Operational Responsiveness has moved from a nice-to-have to a must-have.

 

There are a number of business facing solutions that have shown great promise in addressing operational responsiveness. One of those is Business Transaction Assurance (BTA). This enables businesses to discover their business processes and gain visibility on the effectiveness of these business processes – even if they are built in a wide variety of heterogeneous technologies and work across legacy applications. BTA non-disruptively discovers business processes – without any modification to existing applications – and monitors to ensure processes run to completion. BTA also discovers bottlenecks and hotspots in the processes – enabling businesses to understand just how efficiently they run.

 

Another important solution is Business or Complex Event Processing (BEP or CEP). This enables business users to model the detection of and reaction to patterns indicating business opportunities and threats in real-time. Examples could be an opportunity to up-sell to a customer on the web-site now (opportunity) or risk exceeding a key level (threat).

 

And then of course there’s Business Process Management (BPM). This enables business users to model and execute a business process flow. BPM is also widely used for Business Process Improvement (BPI) – the re-engineering of (parts of) existing processes to improve their effectiveness.

 

The really cool thing we realized in talking with our customers is what happens when you use BTA, BEP/CEP and BPM together. Suddenly businesses are empowered to discover how effective they run, to detect opportunities and threats dynamically and to invoke business processes in response. The business becomes dynamic and responsive. Business users can take control and model the behavior they want their business to exhibit under certain circumstances, and through dashboards they can track the effectiveness of the business. Over time, the areas of the business processes that should be improved can also be detected.

 

Progress already has leading products in BTA and BEP/CEP with Actional and Apama. Progress chose Savvion to complete the story for a number of reasons. Savvion has a history of innovation and is a leading pure-play BPM provider. But Savvion also has a very rich platform, which includes not just BPM modeling and execution, but also an event engine, a business rules engine, a document management system and an analytics engine. The fact that Savvion enables business processes that respond to events means it immediately works well with Actional and Apama. And with high performance, scalability and availability, Savvion fits perfectly into Progress – where we pride ourselves that all of our products exhibit these characteristics.

 

In summary, Progress is now a best-of-breed BPM vendor – and not just at the departmental level – but at the enterprise level. But we’re also more than that. Our goal is to enable operational responsiveness and ensure our customers gain competitive advantage through the power of responsive, dynamic and predictive business processes.

Why businesses must evolve their business processes to be highly responsive, dynamic and predictive – or they will cease to be competitive

Today Progress Software announced the acquisition of Savvion http://web.progress.com/inthenews/progress-software-co-01112010.html. I believe this heralds the beginning of a very exciting phase for Progress Software. Now Progress has become a leader in Business Process Management (BPM). But more than that, combined with our other solutions, Progress is now uniquely able to empower businesses to be operationally responsive – through responsive, dynamic and predictive business processes. And this is critical to keep modern businesses competitive.

You might wonder about the journey Progress went through to realize what the market needed. It was all about understanding the emerging needs of our customers and where they needed their businesses to go. The part of my job I enjoy the most is spending time with customers and understanding what pain points they have - with the ultimate goal of working with them to address the pain and making them highly competitive.

Over the last couple of years I have been hearing more and more from customers about the need to be operationally responsive. For example, many customers have expressed their desire to proactively – and often in real-time - address the needs of their customers and respond to the behavior of their competitors. The goals are to win new business, increase customer satisfaction and triumph over their competitors. These findings hold true whether the customer be in banking, insurance, communications, travel, transport, logistics, energy, gaming or many other industries. It could be British Airways ensuring their high value customers are looked after first in the event of a flight delay, or wireless carrier 3Italia pushing real-time offers to their customers based on their profile, activity and location, or maritime logistics provider Royal Dirkzwager dynamically adjusting the course and speed of a container ship to optimize fuel usage, based on weather conditions and port berth availability.

Operational responsiveness is thus about being highly responsive to opportunities and threats – and even anticipating such scenarios. Market research supports what I’ve been hearing, such as the recent survey by Vanson Bourne http://web.progress.com/en/inthenews/companies-stuck-in-o-10062009.html – suggesting Operational Responsiveness has moved from a nice-to-have to a must-have.

There are a number of business facing solutions that have shown great promise in addressing operational responsiveness. One of those is Business Transaction Assurance (BTA). This enables businesses to discover their business processes and gain visibility on the effectiveness of these business processes – even if they are built in a wide variety of heterogeneous technologies and work across legacy applications. BTA non-disruptively discovers business processes – without any modification to existing applications – and monitors to ensure processes run to completion. BTA also discovers bottlenecks and hotspots in the processes – enabling businesses to understand just how efficiently they run.

Another important solution is Business or Complex Event Processing (BEP or CEP). This enables business users to model the detection of and reaction to patterns indicating business opportunities and threats in real-time. Examples could be an opportunity to up-sell to a customer on the web-site now (opportunity) or risk exceeding a key level (threat).

And then of course there’s Business Process Management (BPM). This enables business users to model and execute a business process flow. BPM is also widely used for Business Process Improvement (BPI) – the re-engineering of (parts of) existing processes to improve their effectiveness.

The really cool thing we realized in talking with our customers is what happens when you use BTA, BEP/CEP and BPM together. Suddenly businesses are empowered to discover how effective they run, to detect opportunities and threats dynamically and to invoke business processes in response. The business becomes dynamic and responsive. Business users can take control and model the behavior they want their business to exhibit under certain circumstances, and through dashboards they can track the effectiveness of the business. Over time, the areas of the business processes that should be improved can also be detected.

Progress already has leading products in BTA and BEP/CEP with Actional and Apama. Progress chose Savvion to complete the story for a number of reasons. Savvion has a history of innovation and is a leading pure-play BPM provider. But Savvion also has a very rich platform, which includes not just BPM modeling and execution, but also an event engine, a business rules engine, a document management system and an analytics engine. The fact that Savvion enables business processes that respond to events means it immediately works well with Actional and Apama. And with high performance, scalability and availability, Savvion fits perfectly into Progress – where we pride ourselves that all of our products exhibit these characteristics.

In summary, Progress is now a best-of-breed BPM vendor – and not just at the departmental level – but at the enterprise level. But we’re also more than that. Our goal is to enable operational responsiveness and ensure our customers gain competitive advantage through the power of responsive, dynamic and predictive business processes.

A BEP On Our Radar...

Last year Progress Software started talking a lot about Business Event Processing (BEP)—more commonly known as complex event processing (CEP). It really kicked into gear when we commissioned an independent technology market research company, Vanson Bourne, to conduct a survey and report on their results. Vanson Bourne interviewed 400 companies representing energy generation, telecommunications,and logistics sectors in the US and Western Europe. Why these industries? Well, because of the volume and complexity of their "business events" (service delivery) - both through systems and processes, and customer, partner, and supplier interactions. The results of the survey, detailed in the paper Overtaken by Events? - The Quest for Operational Responsiveness, demonstrates that harnessing business events, smart interpretation, and fast response are clear objectives for these industries. And the need is immediate.

Below I've included the Executive Summary of Key Findings. If you'd like to get more detail on their findings, visit our website and download the complete paper. 

---

EXECUTIVE SUMMARY OF KEY FINDINGS

The Objective Is Operational Responsiveness

Operational responsiveness is the ability of business processes and systems to respond to changing conditions and customer interactions as they occur, enabling business leaders to capitalize on opportunities, drive greater efficiencies, and reduce risk. The survey identified a number of key pointers as to why businesses would be keener than ever to improve how they respond operationally, for example:

Customers

• 91% said they are trying to act in a more personal “one-to-one” way with the customer. That means paying more attention to specific, individual feedback.
• 74% reported that areas such as digital market channels, mobile platforms, and social channels have caused a significant increase in the flow of information into and through their business. That means paying more attention to events in the context of a blizzard of communication.

Competition

• 70% of the businesses surveyed said that it would it be an advantage to be able to price their products based upon dynamic factors, in response to intra-day changes, such as changes in competitor prices/activity.

Process efficiency

• Operational incidents can be costly: 82% of companies surveyed have to continuously monitor processes to try to prevent them happening.
• 72% said their business processes take too long, and they need to shorten them.

Businesses want to respond quickly and more accurately to business events at the operational and business planning level. Real-time information delivery is seen as an important contributor, seen as having a role in three key areas:

• Monitoring KPIs—overseeing pre-ordained service or business performance benchmarks.
• Automatically alerting end users when certain conditions occur—flagging exceptional circumstances or activity for colleagues to take action.
• Automating response processes—delegating conditional processes to the operational systems

Of the companies surveyed 82% are planning investments in real-time technology by mid-2010 in the hope of achieving the vision.

But the Road Is Long...

The survey reveals that most companies still have a long journey on the path to operational responsiveness as defined above. Here are a few stand-out numbers that underline the current situation:

Service delivery and process gaps

• 67% hear about problems in service from customers before they have identified those problems themselves.
• Only 8% report currently business information in real-time: indeed only 19% report on an intra- day basis.
• 72% think that their business processes take too long and they need to shorten them.
• 89% cannot get a single view of process performance because information on business processes is held in multiple different operational systems. 80% use middleware to try to bring data together but not to the satisfaction of those in charge of operations.

Business planning gaps

• 34% say that, by the time they are able to see a change or trend in one of their business processes, they have missed some if not all of the opportunity to react to it.
• 47% of companies surveyed report that business information is typically analyzed to identify patterns and trends historically and not in real time.
• 58% admit that they have significant gaps in the information they need to support their business decision making.

Real-time Information and Business Event Processing (BEP)

In fact, 94% of businesses said that real-time information is important to them, and 78% said immediacy of response to business events provides a competitive advantage. But where business information is incomplete and/or sits across a range of disparate, non-compatible operational systems (as is admitted by most of the companies surveyed here), then speed alone is not enough. Where BEP is being tried out, users are already witnessing the power of combining and correlating across platforms, as well as the desired advantages that real-time systems would provide:

BEP benefits experienced so far

• Filter and analyze lots of events quickly—66%
• Take automatic actions in response to certain sequences of events occurring—55%
• Better monitoring of existing operational systems—50%
• Normalizing and correlating events from multiple different sources—45%
• Providing real-time visibility into information for business end-users—42%
• Spotting time-sensitive event patterns—25%

If you'd like to get more detail on their findings, visit our website and download the complete paper.