Business Making Progress

BPM Gives the Business a Technical Role in Process Improvement

Do you think that's a good thing?

I’m trying to get up to speed on Business Process Management (BPM) - in case you missed it, Progress Software acquired Savvion in January – and I'm really embracing the part of BPM that gives the business a technical role in process improvement.

One of first Savvion resources I read was the interview-style paper, The Benefits of Adopting SOA and BPM, and the first thing that made me go “yeah!” was:

“The key to understanding the significance of BPM is to understand the significance of the most critical element of businesses, the people. Regardless of the role people play in the business, they care passionately about what they do and how they do it. They also care about improving the way they do their work. Because the people are involved in how processes are executed, it is important to enable them to perform their work easily and effectively by delivering the right information to them at the right time.”

In this quote, I think "the people" are the business because they are usually the closest to the business process. Early in my career with Progress, I worked on projects with IT to deploy various web functionality, including single-sign on, lead flow into our CRM, and launches of two CMS’s. When I first started, IT owned practically every process and you had to work through them to get even the smallest change made to the website. I recall working on the single-sign system and feeling very frustrated because they’d say, “you don’t need to worry about that” when I asked questions about how the flow, or visitor experience, would work. I came back to them with a Visio flowchart of how I thought it should work and the jaws dropped – they were either stunned that someone from the business wanted to be involved, or they were thinking “who the hell does she think she is.” (Personally, I think it was the latter.) In any event, we ended up working on that diagram together for a month or two and outcome was a great functional spec that we could refer to throughout the development process, and beyond. Out of that experience came my intense respect for the role of IT, but also the importance of “the business” being involved. I know that BPM is not just about aligning IT with business owners but today... it is the key point that made me think "that's cool.”

As I continue to read and reflect on my experiences, I am interested in who actually presents the idea of applying BPM to their existing enterprise infrastructure. Is it IT, perhaps a CIO? Or, is it the business, perhaps a CFO? Whoever decides, in my opinion, bringing the business (the people who know what they want and need) and IT (the people who know what technologies can help) together is a really good thing. And if it works, I’m confident that companies will reap the rewards of operational innovation, efficiency, and a greater return on their investments.

Peas and Carrots

In the words of the auspicious Forrest Gump some things go together like peas and carrots. Truer words were never spoken. Some things just do go together well, sometimes by design, often by accident. I don't think anyone actually planned milk and cookies or popcorn at the movies but nonetheless these things are made for each other.  When it comes to technology the same harmonious relationships exist.

In the recent Aite report on High Performance Databases (HPDB),  the market for specialized databases is surveyed along with a handful of vendors in this space.  This is a cottage industry where the big database vendors don't play. It's hard to imagine in this day and age where database technology is so standardized and mature and a multitude of choice abounds from commercial products to open source that any other database technology and a gang of vendors would have a chance. Yet it is happening and it's thriving.  

I believe it has to do with a synergistic relationship to event processing. If CEP is the "peas" then HPDB's are the "carrots". These two technologies share two fundamental precepts:

•  A focus on Extreme Performance
•  Temporal Awareness

I. Extreme Performance, Speeds and Feeds
These HPDB's which are often referred to as Tick databases, are found in the same playground as event processing technologies. In the Capital Markets industry they connect to the same market sources, consume the same data feeds. Both technologies are designed to leverage modern multi-core hardware to consume the ever-increasing firehose of data. By the same token, once that data is stored on disk, database query performance is equally important.  The massive amount of data collected and is only as good as the database's ability to query it efficiently thus creating another (historical) firehose of data which an event processing engine would be the consummate consumer.  

II. Temporal Awareness, when is the data
Time is a basic principle in event processing technology, applications typically have as a premise to analyze data-in-motion within a window of time. HPDB's design center is to store and query time series data. Some of the database vendors even bring time to a higher level business function. They understand the notion of a business Calendar, knowing business hours, business week, holidays, trading hours, etc.  Imagine the simplicity of a query where you want 'Business hours Mon-Fri for the month of February' and the database itself would know the third Monday was Presidents Day, skipping over that, thus preventing analytic calculations from skewing erroneously.

Leveraging the Synergy
These two fundamental shared principles provide the basis for a unique set of business cases that are only realized by leveraging Event Processing platforms and High Performance Databases

• Back testing algorithms across massive volumes of historical data compressing time

What if you could test new trading algorithms against the last 6 months or 1 - 2 years of historical market data but run that test in a matter of minutes? What if you could be assured that the temporal conditions of the strategies (i.e. timed limit orders) behaved correctly and deterministically matching the movement of time in complete synchronicity with the historical data? These are just a few of the characteristics that define the harmony between event processing and high performance (Tick) databases.

• Blending live and historical data in real-time

Querying historical data in-flight to obtain volume curves, moving averages, the latest VWAP and other analytics calculations are possible with these high performance databases. Leading edge trading algorithms are blending a historical context with the live market and even News. The winners will be those that can build these complex algo's and maintain ultra low-latency.

• Pre-Trade Risk Management

Managing positions, order limits and exposure is necessary, doing it in real-time to manage market risk is a mandate.  In addition to market data, these high performance databases can store pre and post trade activity to complement event-based trading systems and become the basis for trade reporting systems.

In the Trading LifeCycle, Event Processing and High Performance databases are partner technologies harmoniously bound together to form a union where the whole is greater than the sum of the parts. They are the peas and carrots that together create a host of real-world use-cases that would not be possible as individual technologies.

Myself along with my colleague Dan Hubsher we are doing a 3-part Webinar series entitled "Concept to Profit". The focus is on event processing in the trade lifecycle, but we include cases that touch upon high performance databases. You can still register for part 2: Building Trading Strategies in the Apama WorkBench where I will focus on the tools for strategy development aimed at the IT developer.

Once again thanks for reading, you can follow me at twitter, here.
Louie

The Debate on Dark Pools

Dr. Giles Nelson, Chief Technology Strategist with Progress Software and a co-founder of Apama, has weighed in on the continuing debate about the possible need to regulate dark pools (Why The Outlawing Of "Dark Liquidity Pools" Debate Rumbles On) in FreshBusinessThinking.com.  Beyond the thoughtful summary of the current issues, one might want to read any article that includes a reference to "the devil's spawn."   

Provisioning Dimensions for the Cloud

At last Saturday's class for my Middleware Internals at IIIT-Hyderabad, I was introducing cloud computing and provisioning. Some basic questions came up - even computer science students from a Top-10 institution in the country have questions like "Isn't SaaS Cloud". What many miss is that Cloud Computing is more about virtualization-over-the-web and the enabling of mechanics such as integration and provisioning.

To this end (virtualization-over-the-web), Software-as-a-Service (SaaS) provides the end users [i.e. the enterprise] value based views of a 'virtualized' application wherein all the operational and infrastructural aspects are managed by the service provider. Likewise PaaS provides the virtualized view of an application platform on which the end user can build a solution. Or with IaaS, where just the infrastructure/OS is virtualized over the web on which any solution can be installed and configured. The definition of cloud also varies based who you ask. Platform-as-a-Service (PaaS) providers will tell you that cloud is when you build applications on their platform. IaaS providers will tell you that if you use their infrastructure, then that is cloud. But I feel the real cloud is what the end enterprises see--a virtualized over-the-web application landscape in a combination of IaaS, PaaS & SaaS. It's a very heterogeneous environment that enables the IT solutions for the various business needs that the enterprise may have. This integrated infrastructure gets the best of breed with no constraints on technologies, platforms, payment models, and even physical location, while still enabling some common binding elements such as Web 2.0 enabled user interface, common administration approach, common integration approach and even provisioning capabilities across the various platforms in the cloud. 

Provisioning is also emerging as an important common aspect of cloud computing. It has emerged from something intrinsic to specific platforms such as Amazon EC2, and now to a more generic expectation across all cloud services.Though the dimensions and approaches to its realization may be different in different providers, a few key dimensions are hardware resources, application platforms or cross cutting dimensions like user provisioning or business service provisioning. Examples include specific resources like hardware (say 2 CPUs), OS (linux ver x.y), app platform (tomcat servlet engine), or an instance of a specific application. And more importantly non physical resources like provisioning a user (for example: enabling access to multiple systems/apps for a new employee).

Through 2010 I think we should be seeing more enabling abstractions, models and utilities for provisioning in the heterogeneous cloud computing environments.

Why BPM should be on the CIO’s agenda in 2010

New article by Giles Nelson published in CIO (http://bit.ly/biSKFo) online.

"In 2010, the business prerogative across all sectors is to use IT to drive efficiency and enable a business to react more quickly to customer and market changes. To do this, I believe we need to take a different view of BPM technology and try to see how it can be used to make knowledge-based business more ‘operationally responsive', reacting to customer needs and market changes instantly. This is already beginning to happen, and as it gains momentum, BPM will prove its usefulness in bringing ‘order to the chaos', and will make it onto the strategic agenda of every CIO."

The article does a great job at illustrating the synergy between business process management (BPM) and complex event processing; or in this case business event processing (BEP). Giles even provides examples of industry's already deploying these technologies. Most of us know the benefits of BPM but as he points out... "The next stage is to match it with the other side of the coin, where it can help an organisation respond to events and become truly operationally responsive - something worthy of the full attention of any CIO." Read the full article.

Tomorrow - Webinar Discusses How Smart Grids Are Helping Utility Companies

Paving the Way to Smart Grid Modernization: A Standards-Based Reference Architecture
Wednesday, February 17, 2010 1:00 pm (EST)
> Join us for the event: http://bit.ly/smart-grid-webinar

During this live event, presenters - Terry Nielsen, VP at UISOL, and Conrad Chuang, Product Marketing and Mark Brooks, Principal Solution Architect, at Progress Software - unveil our unique, standards-based reference architecture that uses a product integration bus to simplify technology deployment and upgrades. By leveraging their domain, technology and integration expertise, they'll provide insights on:

• Market and regulatory conditions.
• The readiness of legacy applications to support real-time smart metering data.
• he cost and difficulties of integrating disparate utilities systems
• Steps for moving to a smart grid-compatible environment, cost-effectively and non-intrusively.

Pre-register or join us tomorrow!
Wednesday, February 17, 2010 1:00 pm (EST)
Wednesday, February 17, 2010 10:00 am (PST)

View the archive of this webinar >

Poor Customer Service and Snoring Keep Me Awake...

Last night, like many weeknights, I pre-planned my escape by putting my phone in the guest room. As I entered the room and got ready for the one-click alarm set, I noticed that I had a new voice mail so I called in to check it. Here is what it said:

"Hello, This is an automated voice message from Jet Blue Airways for Pamela Gazley with important information about a change to your scheduled flight. Your flight has been canceled. We look forward to serving you. Goodbye."

My first response was "why?"; my second response was "which flight?" My 2nd response was a little silly because of the 4 flights I have scheduled for the next week, only one involved Jet Blue; however, what if there were others? Does Jet Blue Airways know that there are products out there that can improve their processes so that they can deliver better customer experiences? Does Jet Blue have any business analysts that might think, "hey, what if when we send out the automated voice message, we include the flight number, date of travel, and the reason for the cancellation"? I guess not because I was completely annoyed.

Now I don't work for Jet Blue but I'd bet that they have some kind of integrated infrastructure that would allow them to improve business processes. If this "automated message" knows my name (though pronounced wrong), it should know my flight number, the date of travel, and the reason for the cancellation. My advice to Jet Blue: learn more about business transaction assurance,  business process management (BPM), and give us a call because we can help you achieve operational efficiency and improve your customers' experience.

Brazil Embraces High Frequency Trading - Do You?

The trading business feels like a fight, now as ever, with the threat of sweeping regulations as the most pressing concern of the moment.  This business even sounds like a war, too - with algorithms names like "Raider" and "Sniper;" and with terms like "dark pools," and "low latency arms race" drawing focus from regulators and media alike.  But the war-like aspect remains because trading is a highly competitive business. 

The imperative to increase market share will remain a top priority this year along with risk management and regulatory compliance, and the technology required to compete is available to everyone.  As Apama has expressed before here, the markets are still driven by those with the flexibility to quickly adapt to new regulations, the insight to understand new market behaviors, and the imagination to conceive a trading strategy that can capitalize on the opportunity.  It's no wonder that the relatively small number of firms using high frequency trading strategies are responsible for over 70% of US equity trading volume.  These pressures push the rest of the capital markets in the same direction and the trend is unlikely to reverse. 

On February 8th 2010, Apama announced that Banco Fator Corretora, a Brazilian bank and brokerage firm, has deployed the Progress® Apama® Algorithmic Trading Accelerator. Apama plays a critical role in Banco Fator’s new electronic trading strategy, enabling it to more effectively develop high frequency, proprietary trading tactics, achieve rapid customization, and perform low latency execution of trades on behalf of its buy-side clients.  Banco Fator is also working with its clients to design customized algorithmic trading strategies that provide them significant competitive advantage, and the bank explicitly emphasized the importance of providing its clients with a fast method to enter the high frequency trading business.

Why so much emphasis on high frequency trading (HFT)?  Reasons will differ among traders and regions, but a short primer on HFT and some ideas are here.  The Brazilian market has expressed a strong opinion on the matter, with over 15 customers deploying Apama internally to automate execution and/or alpha-seeking strategies in the past year-and-a-half, with many further rolling out the platform to downstream clients. 

So, do you have an opinion on HFT as well?  What characteristics should a platform for HFT have to enable you to be more competitive?  Let us know - leave a comment, or take our poll.

-Dan

Celebrating the shadow of Punxsutawney Phil

As the week of Punxsutawney Phil’s appearance commences, I must admit I’m one of the few that is happy to hear we have another 6 weeks of winter to look forward to! Why you ask, well that means more skiing and an extended window of time until I have to start worrying. Worrying about when to apply “Step 1” nutrients to my lawn, about how much the voles and moles have destroyed my plants, and about whether or not I warded off the dreaded Dutch Elm disease with the systemic treatment that was applied last fall. Bottom line is I have a 6 week reprieve.

But... what if I could apply technology to my yard, garden and even the infrastructure of my home. What if I could apply some fundamental concepts like automatic discovery, monitoring, management and control, across my household infrastructure? The possibilities are endless. I could set up points of visibility at strategic points in my yard such as the base of my newly planted Double Pink Weeping Cherry and at the perimeter of my bulbs. In the house I’d want visibility at the base of my water heater, sunk pump and egresses. With all this visibility I’d then establish a console where a complete infrastructure map would clearly reveal all activity that transacts in and around each point of visibility and if any issues were detected I could quickly and easily pin-point the root cause. And with points of control I could then dynamically control and avoid potential danger or damage such as voles eating the bark of my young cherry tree or avoid having my basement flood. Ohhhh wouldn’t this make life so much easier and a lot less costly.

Even though technology has not yet met my household needs, I still have hope. Maybe someday the principles and benefits of solutions like Actional for business transaction assurance will apply not only to the needs of enterprises with business critical transaction but to my needs as well. With Actional enterprises gain complete and automatic end-to-end visibility into their heterogeneous environment. Visibility that helps organizations understand the value of each transaction with the ability to dynamically control and optimize outcome. If enterprises can gain this level of business transaction management, I think it only makes sense that our next market should be the household management sector!

CEP consolidation continues

It’s been another remarkable week. I told my wife there would be a lot of traveling for me in the first part of the year and I was right. Last week was New Jersey and New York. This week was Dallas Fort Worth and Silicon Valley. I’ve been visiting key customers, journalists and analysts.

This week has also seen further consolidation in the CEP market. I have been predicting that there could not be a stand-alone CEP market and that CEP will either find a home in applications, databases, stacks or business application platforms.  In this case Sybase has snapped up Aleri to extend its database business into the CEP domain, as well as solutions in the risk space. Aleri are a good company with good people and good products. They come from the “in memory database” perspective but developed a high performing CEP engine and learned lessons from real customers that a SQL approach is not adequate to address real applications, and embedded actions statements are need within a CEP language. Also they learned that the best way to sell CEP is not as a technology but as solutions.

I think Sybase have made a smart move – for probably a bargain price – judging by the release that says they acquire the assets only. I wish my friends at Aleri all the best for the future.

From Concept to Profit in No Time Flat – High Frequency Trading

Colleagues Dan Hubscher and Louie Lovas have begun a great webinar series that outlines the “lifecycle” of an algorithmic trading strategy.  Illustrated with Apama’s Event Modeler, they use a Commodity Futures trading strategy to illustrate how trading firms can accelerate the delivery of trading strategies with a development tool that is accessible to the trading desk. This can help make significant cuts in development times, allowing firms to capitalize more quickly on opportunities.  Future sessions will explore some of the other aspects of the platform that target developers, recognizing that firms have different business models and development styles.

We’ll be posting the recorded version shortly for on-demand viewing, but if you have not registered, I’d encourage you to click here and get on board for parts 2 and 3.