Business Making Progress

Can market surveillance help to keep traders on track?

"According to TABB Group new compliance costs are indicated at between 512 and 732 million euro, with ongoing costs between 312 and 586 million euros.  But while regulators are still determining what regulation will look like, the need for market surveillance is undiminished. Traders made about 13.3 billion euros ($18.2 billion) from market manipulation and insider dealing on EU equity markets in 2010, according to an EU commission study.  With some arguing that firms can only do so much to survey markets themselves as trades cross multiple brokers and gateways, the panel discussed the need for fragmented market data to be brought together in a consolidated tape and surveillance performed at an aggregate market-wide level."

This is just an excerpt from a recent post to our Event Processing Blog by Richard Bentley, VP Capital Markets, Progress Software. Can market surveillance help to keep traders on track? Read the entire post.

My Baby Has Grown Up

I was proud to recently be appointed CTO and head Corporate Development here at Progress Software http://web.progress.com/en/inthenews/progress-software-ap-12102009.html. But I don’t want anyone to take that as an indication that I won’t still be involved with event processing – au contrair. Event processing (whether you call it CEP or BEP) is now a critical part of enterprise software systems – I couldn’t avoid it if I tried!!

But taking a broader role does give me cause to reflect upon the last few years and look back at the growth of event processing and the Progress Apama business. Here are some observations:

• It’s incredibly rare to have the pioneer in a space also be the leader when the space matures. I’m really proud that Progress Apama achieved that. Our former CEO Joe Alsop has a saying that “you don’t want to be a pioneer; they’re the ones with the arrows in their backs!” Usually he’s right on that one – but in the case of Progress Apama, the first is still the best! Independent analysts, including Forrester and IDC, all agree on it. Our customers agree on it too.
• It’s tough at the top! I had no idea that when you are the leader in a space, many other firms’ technology and marketing strategies are based completely around you. I have met ex-employees of major software companies that have told me that there are Apama screenshots posted on the walls of their ex firms’ development centers – the goal being to try to replicate them or even improve on them. Other firms’ marketing has often been based on trying to criticize Apama and say why they are better – so their company name gets picked up by search engines when people search for Apama.
• Event processing has matured and evolved. Yes it is certainly used to power the world’s trading systems. But it’s also used to intelligently track and respond to millions of moving objects, like trucks, ships, planes, packages and people. It’s used to detect fraud in casinos and insider trading. It’s used to detect revenue leakage in telecommunications and continually respond to opportunities and threats in supply chain, logistics, power generation and manufacturing. It enables firms to optimize their businesses for what’s happening now and is about to happen – instead of running solely in the rear view mirror.
• Despite all the new application areas, Capital Markets remains a very important area for event processing. Critical trading operations in London, New York and around the world are architected on event processing platforms. The world’s economy is continually becoming more real-time, needs to support rapid change and now needs to support the real-time views of risk and compliance. We recognize the importance of Capital Market. My congratulations to Richard Bentley who takes on the mantle of General Manager of Capital Markets to carry on Progress Apama’s industry-leading work in this space. With his deep knowledge and experience with both Apama and Capital Markets, Richard is uniquely placed to carry on the solutions-oriented focus that has been the foundation to Progress Apama’s success.
• Even in a terrible economy, the value of event processing has been proven – to manage costs, prevent revenue leakage and increase revenue.  Progress announced our fourth quarter results today http://web.progress.com/en/inthenews/progress-software-an-12222009.html which saw a double digit increase for Apama and triple digit for Actional. Apama and Actional are used, increasingly together, to gain visibility of business processes without modifying applications, to turn business process activity into events and to respond to opportunities and threats represented by event patterns – enabling the dynamic optimization of business performance.
• But one thing I do believe: that soon there will be no such thing as a pure-play CEP vendor. CEP is part of something bigger. We’ve achieved the first mission, which is to raise the profile of event processing as a new technique that can solve hitherto unsolvable problems. Now the follow on mission is to ensure event processing finds its way into every solution and business empowerment platform. It is one of a set of key technologies that together will change the world.

I wish everyone Happy Holidays and a successful and profitable 2010 !!!

Get Yourself Connected

Our integration with Vhayu Velocity, announced earlier this week, is the latest connectivity option we've added to the Apama platform. This follows hot on the heels of the launch of our adapter for the Brazilian BOVESPA market, and got me musing about the importance of connectivity for our Apama CEP platform - and the effort we need to invest to develop and maintain it. All in all we now support more than 40 distinct adapters (as listed here) and are adding more all the time (we have 5 new adapters in development right now).

The importance of connectivity cannot be overstated. One could have the best CEP Engine on the planet, but if there's no way of getting events in and out of it then it's of zero use - the proverbial Ferrari with no wheels. For basic infrastructure, you can go some way with strong support for standard middlewares through JMS, databases through ODBC/JDBC, etc. For Capital Markets however, where latency is often a critical success factor, it is often about direct connectivity to the market, involving development to proprietary and extensive market-specific APIs; although good support for the FIX protocol is most definitely necessary, it is nowhere near sufficient for the low latency high frequency trading world.

Of course, once you have strong connectivity to a particular market then that becomes an enabler - our support for the native market data and order execution APIs of the Chicago Mercantile Exchange (CME), for example, is allowing Apama to play in the proprietary trading world of the Futures and Commodities markets with great success, and it is no surprise that Apama is doing particularly well in Brazil! But such connectivity does not come cheap; as exchanges regularly rev their APIs, adapters need constant care and feeding, and there are always new markets and systems that need to be connected, particularly for a platform like Apama which plays across all asset classes, on a global stage. Apama currently retains a team of 10 Engineers and QA staff who *just* build and maintain our adapters, and we regularly second other personnel to this team to cope with demand spikes.

We took a decision early on to invest in developing our own connectivity; we clearly had options - there are no shortage of intermediaries out there who purport to connect to 000s of different systems. But there's always that one system that they don't support - or don't happen to support on the platform you need - so you never get away from having to build and maintain it yourself to some extent. And then of course there's the need to integrate with proprietary systems - when you start off by targeting your product at the Tier 1 investment banks, there's an awful lot of in house systems you need to hook up to!

In fact, our earliest clients were all of this kind, requiring proprietary connectivity to their home-grown systems. Overall this has been a boon for Apama, as we were forced to focus almost day 1 on building a toolkit which allowed us to rapidly develop new connectivity – resulting in our Integration Adapter Framework (IAF). All Apama connectivity is built using IAF - and as a result benefits from common configuration and management interfaces, uniform latency measurement framework, real-time status reporting and more.

Building and supporting the IAF, developing our 40+ (and counting) adapters, keeping up with exchange upgrades and so on requires a huge investment of time and effort. Whilst this might be a much less glamorous aspect of developing a CEP Platform (my colleagues in Engineering touchingly refer to it as "the filth"), it is a hugely critical one.

... and whatever happened to the Stereo MCs anyway?

We're going on Twitter

Louis Lovas and myself, Giles Nelson, have started using Twitter to comment and respond to exciting things happening in the world of CEP (and perhaps beyond occasionally!).

The intent is to complement this blog. We'll be using Twitter to, perhaps, more impulsively report our thinking. We see Twitter as another good way to communicate thoughts and ideas.

We would be delighted if you chose to follow our "twitterings" (to use the lingo), and we'll be happy to follow you too.

Click here to follow Louis and here to follow Giles (you'll need to signup for a Twitter account).

FX: A CEP Success Story

One of the questions that sometimes comes up in our meetings with prospective clients is "Why should I buy your whizz-bang CEP platform to do <insert use case here> and then build on top of it when I can go get a shrink-wrap app that does most what I want straight off?". Of course, this is a very easy question to answer: "If you can get an app that does what you want now, and you're confident it will meet your requirements as they evolve going forward, play nice with your existing systems, and the price is right, then why are you wasting time talking to us?". Marc Adler said as much recently with his comments on Risk Management systems, which were right on the money.

Of course, this answer is really an open incitement to the audience to start a discussion about what they really need - now and in the future - and often (more often than not in my experience) it is soon apparent that there is no app out there which will do all, or even most, of what they need - and extending a shrink wrap app to do that is often more work and pain, if it's possible at all, than building it themselves.

So here the salesperson will introduce the core value proposition of an open CEP platform; the ability to support existing and future requirements, to meet the underpinning performance requirements, the necessary resilience, to provide the connectivity, and - vitally - the tooling to allow the same vendor-supported and maintained technology to be applied to the client's constantly evolving needs, with rapid time-to-market. In short, the salesperson emphasizes the benefits of a more "strategic", platform-based approach to solving the client's short and longer-term requirements and - if the salesperson has done his homework and read his audience right - that one-size-fits-all shrink wrap approach is often consigned to the trash.

The nice thing about this approach is that there's no sleight of hand here; every word of the pitch is true.

Let's look at the use of Apama within Foreign Exchange (FX). We've blogged previously about how Apama is being used extensively for real-time FX rate Aggregation and algorithmic execution; just last week our latest client to go live - Standard Chartered - talked publically about their deployment of Apama for their FX spot dealing. We've built a Solution Accelerator to offer relevant but customizable connectivity, algorithms, monitoring and dealing GUIs out of the box. We've even won awards (ok, enough already!). However, although we're (clearly) not shy about trumpeting the power of our offering in this area, the more telling aspect of our deployments - and the aspect of relevance for this article - is not what clients do with our Aggregator; it's what they do *next*.

I was visiting one of our large Tier 1 banking clients in New York a couple of weeks back. They've been using Apama for FX Aggregation for nearly a year now. I walked through the door on to the trading floor and first thing I saw was 3 screens with - unfamiliar - Apama Dashboards on them. "That looks nice" said I (somewhat gormlessly). "Yes, it's our new auto-hedger - it uses the aggregated book feed from your Solution Accelerator and lays off our risk positions over your ECN connections according to a new algo we've built". All developed and running in Apama, extending our Solution Accelerator with new algos and screens - and all unbeknown to me!

I was somewhat more in the loop concerning the work of our friends at one of the world's largest Banks, who have been using Apama for FX Aggregation for well over a year, connecting to multiple liquidity pools to improve their execution. That bank has have recently launched a new FX pricing engine - built with Apama - connecting to their Apama FX Aggregator for real-time pricing, running analytics to derive per-client spreads for market making (similar to how I describe [here]). An auto-hedger is in the works - again extending the initial FX Aggregation deployment and based on the Apama CEP platform. And at least 3 other FX clients have either rolled out or are in the process of rolling out market making solutions as extensions of Apama FX Aggregation deployments (and those are the ones I actually know about!).

Moving beyond auto-hedging and market making, we have a growing number of examples of the benefits of a more strategic "platform" approach, such as Apama clients like ING in Europe taking FX feeds into their Equities Algo systems (also built on Apama) for cross-border spreads (locking in an FX rate as the third leg of the trade), prop shops engaged in currency forward / spot arbitrage, and clients like Louis Capital FX extending their Apama-based FX agency hedging their FX options trading. (As an aside, one the hottest topics for us right now in FX is the area of real-time Forward FX price streaming to large corporates - once the province of the very biggest banks - utilizing real-time volatility and depth of book analytics ...)

The power of the more strategic approach enabled by CEP platforms like Apama, armed with the out-of-the-box connectivity, performance, resilience and tooling, to meet client's current and future needs, is nowhere more ably demonstrated by our experiences in FX.