Business Making Progress

Greetings 2012 – Say Hello to OpenEdge 11

As was mentioned in mid-December, the latest update to our OpenEdge platform is now shipping, and I am pleased to say that it is enabling hundreds of our customers and ISV partners to securely develop and deploy applications across any platform, any mobile device, and any Cloud.

One of the highlights of OpenEdge 11.0 is our patent-pending Multi-tenant Tables, in which data is physically (not virtually) separated in the database - providing greater security and control for Cloud deployments. Multi-tenancy is a critical component and key differentiator for our customers and partners, along with our multi-Cloud deployment options, business process-enabled development, and support for mobile devices.

Feedback has been very positive and many customers are migrating to OpenEdge 11.0 sooner than expected to take advantage of the increased security in the Cloud, greater deployment flexibility, reduced costs, and faster time to market. Let me share some of that feedback with you now.

Security and Flexibility through Multi-Tenancy

Jeffrey Brown, Senior Development Project Manager at Infor notes, “Progress provides us with the technology to power our Infor10 Distribution Business, a distribution application specifically designed to help distributors with complex business models run an efficient, end-to-end operation. We are interested in the new multi-tenancy capabilities in the OpenEdge platform that could provide us with the flexibility to add an additional level of security and separation of data at the database level that is unique in the industry.”

Reducing Cost While Speeding Time to Deployment

Another Progress partner, a global medical software and services provider, used OpenEdge to develop an order management system for internal call centers. Multi-tenant Tables in OpenEdge 11 provide a viable solution for compliance with data security regulations customary to the healthcare industry. Moreover, it facilitates the roll-out of their order management system to all companies they acquire moving forward, which will be deployed in a fraction of the time, for a fraction of the cost, and with better security measures.

Efficiency and Moving Down Market with SaaS

Over in Germany, EDV-Software-Service AG (ESS), a provider of ERP software and services for the mid-size housing and real estate market, is leveraging OpenEdge 11 Multi-tenant tables to move to Software-as-a-Service (SaaS) to gain efficiency and expand into new markets. Their CIO Michael Förster explained, “Progress Software understands the needs of medium-sized businesses and helps us provide value to our customers and accelerate our time-time-market with new solutions. We took part in the OpenEdge Early Adopter Program and Multi-tenancy Workshop, and in only five days were given the tools and expertise needed to get our new release ready for launch in early 2012.”

I look forward to hearing more about how our customers and partners are taking advantage of OpenEdge 11.0. For more information on OpenEdge 11.0, please review the “What’s New in OpenEdge 11.0” feature highlight.

Here’s to a great start to 2012!

Thanks and as always, please feel free to drop me a line and let me know what you think.

Consulting our crystal ball: IT Predictions for 2012

With the New Year just around the corner, many are busy thinking up unattainable diet and fitness resolutions, but we here at Progress have instead spent our time collaborating on more realistic forecasts for the coming year.

The team here at Progress put our heads together to produce our top predictions on how the role of IT within the business will change in 2012. An increased emphasis on cloud development, data security and social integration are all issues we expect organizations to prioritize over the next twelve months, but the list doesn’t stop there. Here’s a quick look at where we see business IT going in the coming year:

1. Cloud on the move. Organizations will increase deployment of the public cloud, escalating demand for cloud-enabled systems and applications.
2. Cost control evolves to efficiency. While cost was the main driver of cloud adoption in the past, the focus will now expand to include system efficiencies and time to market.
3. Data security starts with secure access. Who will have access to the data? How will it be encrypted? Who is the core owner? A strong driver that runs on a stable and tested data interface like ODBC is the best line of defense as application stacks continue to grow.
4. RIP: Non web-based applications. Approximately 80% of business apps will be web-based, and they need to be business process enabled, web-based and cloud-deployed.
5. IT border control. More than half of all content and functionality will be out of your organization’s control … in the hands of outsourcers, supply chain partners and external community databases. How will it be protected is the question du jour.
6. What’s in the fire hose? While we may see companies promoting fancy strategies for managing “fire hose data,” only those focused on responsive analytics will make meaning from the massive deluge.  
7. Limitations of freeware. This year, we will see greater support for ODBC and investment in data connectivity as companies look for dependable, robust ODBC drivers to handle financial transactions securely and quickly.
8. All hail the social enterprise. Social collaboration apps will dominate as employees look for ways to more effectively share and innovate across regions and lines of business; in fact, users will begin to expect these capabilities to be offered as standard, embedded features in business applications. 

And there they are: Progress’ IT predictions for 2012… how do they stack up to what you have in mind? We welcome comments below or on Twitter at @DrJohnBates or @ProgressSW

Our 2012 Predictions: What to expect in capital markets

What will we see for capital markets in 2012?  The countdown to 2012 has begun. On the capital markets horizon is a great deal of change – no surprise to those following this year’s rollercoaster of rogue algorithms and regulation tension. So with no further ado, here are our capital markets predictions for 2012:

 1. Billion Dollar Blunder. At least one financial institution will take a billion dollar (or more) hit when a rogue algorithm goes wild. The algo will go into an infinite loop, taking on an irreversible and un-hedged position, which cannot be shut down. Losses will challenge those by human rogue traders, which banks and financial institutions will prevent from happening next year.

2. Occupy HFT. The public, government and regulators will start the "Occupy HFT" movement -- a popular uprising against the ultimate elite of those making money in this climate. Despite immense financial industry pressure, regulators in both the US and the EU will be panicked by investor and political disapproval of HFT and will rein it in with draconian rules and controls.

3. SEFs Spur Splash Crash. Swaps execution facilities (SEFs) will revolutionize OTC derivatives trading, enabling them to be traded electronically. This, in turn, will lead to increased risk of a cross-asset class swaps "splash crash" which will confound regulators, who have little understanding of these markets.

4. Global Regulation Rocks. Countries will finally realize that regulatory harmonization is a good thing and that individual self-interest is not. Banks and financial services firms will realize that they need to think like regulators, taking control of internal surveillance and compliance before regulators make them do it.

5. RICs Get Smarter. The RICs in BRICs are getting smart order routing and gearing up for an increase in algorithmic trading. This, coupled with looser regulations, will begin to attract regulatory arbitrageurs and Volcker Rule escapees.

6. The Wild East. The West's supremacy in financial markets will further decline as new trading regulations - the Volcker Rule in the US and MiFID in Europe - create a surge of regulatory arbitrage favoring more lightly regulated geographies such as Russia and China. Wall Street and the City of London will lose human and financial capital as a result.

7. Financial Terrorism. An exchange or trading destination will be hacked by financial terrorists intent on manipulating markets for political gain. This will lead exchanges and ECNs to add more stringent monitoring and market surveillance capabilities.

8. Head in the Clouds. Explosive growth in foreign exchange trading and SEFs means that participating firms will require complex hosted solutions. Even the smallest FX broker needs aggregation and pricing services which require a big technology footprint. SEFs present new challenges as swaps markets attract algorithms and require surveillance.

9. Crime & Punishment. Regulators are cracking down hard on financial fraud and market manipulation and they will bring in some big fish in 2012. Prosecutions and punishments will increase in size and in impact.

There you have it – nine predictions for capital markets in 2012. What are your thoughts on these predictions, and have we missed any? Comment below, or tell us on Twitter at @DrJohnBates or @ProgressSW.

An Algorithmic Trading and Market Surveillance Wrap Up

Our Capital Markets and Progress Apama teams have been BUSY! Today many of them are recovering from a busy week at the SIFMA Financial Services Technology Expo in New York City.

In addition to lots of greeting and Tweeting, Dan Hubscher even had some time to post a couple blog posts:

• Tickets Please: Technology to Keep You on the Train
• Keeping the Train on Track

And while Dan helped man the floor, our VP of Corporate Communications, John Stewart, worked to get 3 press releases out onto the BusinessWire, including:

• Progress Software and Activ Financial Introduce New Market Data Adapter for Apama Capital Markets Foundation
• Progress Software Survey Reveals the Realities of Regulation, Monitoring and Algorithms on the OTC Derivatives Market
• Progress Software Launches Enhanced Market Surveillance and Monitoring Solution

Not only that, our own Dr. John Bates was tagged by Wall Street & Technology as one of our "Top 10 Innovators of the Decade for Capital Markets”.

Across the pond and beyond, Dr. Richard Bentley was quoted in the Bobsguide article “Suspect movements in share price fall to an eight-year low”. Dr. Giles Nelson traveled to India to promote Progress’ business in Capital Markets. He shared his thoughts on our complex event processing (apama.typepad.com) blog:

• The rise of algo trading in Asia - first stop Mumbai
• Still big room for growth in Japan

Phew! And, just in case you missed it, we wrapped up posting a 4 Part video series on Financial Regulation and Market Surveillance. Here are the blog posts that provide a brief overview and link to the videos:

• PLUS Stock Exchange Making Progress Around Regulation
• Using technology to prepare for regulatory change: how can market surveillance technologies help?
• Preparing for changes in the regulatory approach
• Cutting through the confusion of financial regulation

 

Red Flags in Morning, Firms Take Warning

A pattern is emerging within new financial services regulations where regulators and financial services firms deploy monitoring technology to "red flag" potential issues such as risk, position limits, errors and manipulation. The "red flags" raised would then alert the relevant personnel or authorities.

In the case of the Volcker Rule, prohibiting banks from proprietary trading and investing in or sponsoring hedge funds or private equity funds, the authorities would use a three tiered approach (http://tinyurl.com/2bh9ot3).  First "tripwires", such as the length of time a trader holds a position, its size or riskiness, would alert banks’ compliance departments  who would (#2) quiz the trader on the nature of the position. And (#3)regulators that keep inspectors on banks’ premises would see the tripwires and monitor both traders and compliance departments.

Over at the CFTC, regulators are looking at a similar approach to monitoring and controlling position limits on products such as oil and metals with a "points" system that would give the CFTC monthly reports that it could use to red-flag traders with large positions (http://tinyurl.com/2ugbdh6).

The tracking and red flag approach is the latest step in increased monitoring of trading operations with the ability to take response before it’s too late. At Progress, we have been advocating using monitoring and surveillance technology to help catch inside trading and avoid fat fingered trading errors for years. With new regulations, monitoring becomes not only mandated but more complicated. Red flags are likely to be flying all over the place within as little as months, both inside and outside financial services firms, presenting a fine opportunity for our Responsive Process Management software solution.

As the financial services world becomes more compliant, the ability to manage red flags becomes more critical. Every process within a financial services firm must be scrutinized, from trade entry to risk management, to analyse and understand internal and external events. This take sophisticated technology. This is where Progress Software's RPM software fits in. According to technology research firm Ovum: "Unless an organization has already made a significant investment in creating an operational responsiveness solution around best of breed products, it will be worth seriously considering the competitive advantage and improved effectiveness that could be achieved by deploying RPM."

Ovum noted in a Technology Audit note that multiple technologies are required to gain a comprehensive insight and respond more rapidly to changes to the environment. These include: business process management (BPM) to model, implement, and execute the processes; business analytics to determine how effectively the processes are working; complex event processing (CEP) to understand the implications of many streams of internal and external events; business rules management to determine the appropriate actions for a given set of conditions and variables; and visibility into end-to-end transactions to track and audit their progress.

The interrelationships between all of these components and the vast amount of information that has become available must be understood before its impact on processes can be ascertained and appropriate tuning performed. In other words, RPM is the answer.

RPM can monitor an increasing number of information feeds, both within or external to the organization, then apply business policy and governance rules, then automatically tune the  established process or alert a human decision-maker (if necessary) and present him/her with current, relevant information on which to base the most appropriate response.

According to Ovum: "All of these individual capabilities already exist (at different levels of maturity), but the cost and complexity of integrating these into an effective business solution is beyond the means of most organizations. Hence Ovum believes that the requirement identified by Progress represents a genuine market opportunity." Well said. 

 

Know Your ABC's: Business Transaction Management with Progress OpenEdge in the Cloud

This is the title of a session being presented at Exchange Online 2010 by Gary Cink on September 15th at 9:45 am. In this presentation, Gary will demonstrate how some of the complimentary Progress' products enhance the OpenEdge experience for business transaction management (BTM). BTM is a critical component in the new IT/Business relationship. Progress® Actional®, Progress's first class BTM product, translates data relating to an underlying IT estate into information that is relevant to various business stakeholders including operations staff, application development, quality assurance, and security & compliance personnel. With this knowledge the various stakeholders can make informed decisions, often proactively, to ensure the success of every critical business transaction necessary for the day-to-day running of a business. Actional also offers the capabilities of automating operational Service Level Agreements (SLA) against this estate, thus preventing issues or alerting appropriate staff to problems before they have even happened.

Learn more about Gary and this technical session on our View From The Edge blog.

You can also read the white paper SLAS: Lost in the Cloud? Service-level agreements (SLAs) are crucial any time a business service provider (BSP) provides an application or service to a client via the cloud. In the results of this survey, 76% of respondents report that SLAs are critical to winning new business. Yet, 84% are unable always to meet their SLAs.

Smart - But Is It Smart Enough?

Today Nasdaq group purchased Smarts - a provider of market surveillance. This is an interesting development . Read the full post on our Apama blog :

Smart- But is it Smart Enough >

Beware the weight-challenged digits

Fat fingers (or weight-challenged digits - for the more politically correct :-) ) are trading errors that can have catastrophic consequences. And they've been happening a lot recently!!

Read my full blog post here.

CFTC Launches Technology Advisory Committee

Yesterday the CFTC, the regulator in charge of Futures and Options markets, announced a new Technology Advisory Committee (TAC), chaired by the very capable Commissioner Scott O’Malia. Read the complete article.

I am absolutely delighted to be included in the group of experts that the CFTC has called together to form the TAC. I am joined by an extraordinary group of some of the industry's top executives from banks, brokers, trading firms, exchanges and clearing firms as well as some very impressive academics. On Wednesday, July 14th (tomorrow as I write), we will meet to discuss the impact of high frequency and algorithmic trading on the markets, including whether algorithms may be implicated in the May 6th 'flash crash'. From this, we’ll discuss what recommendations we have for regulation of and/or best practices for algorithmic and high frequency trading.

High frequency and algorithmic trading are essential for efficient execution and alpha generation in a complex, multi-asset, fast-moving world. However, there are a number of accusations that have been made against these forms of trading, including that they may aggravate volatility and may even have caused the ‘flash crash’. I believe evidence from the TAC participants will exonerate the accused.

I am hoping that our meetings will result in solutions that not only head-off future ‘flash crashes’, but also help exchanges, banks and brokers to better monitor and police trades. The proactive use of real-time monitoring systems can alert regulators to problems before they become a crisis. Monitoring technology can 'see' major price and volume spikes in particular instruments, how often they happen and maybe even why, and whether a pattern in market behavior caused them. It can also tell how much trading is potentially market abuse, for example, insider trading might be detected by correlating unusual trading incidents with news releases and market movements. (The FSA, for example, thinks that 30% of trading around acquisitions is insider.)

It is now possible to apply high frequency techniques to not just trading – but also to market monitoring, surveillance and pre-trade risk checks – for regulators, exchanges and brokers. The technology is out there (with proven approaches built on next generation platforms such as complex event processing or CEP) and it needn't be expensive. The CFTC's TAC is a positive step in the right direction. I look forward to the meeting and will let you know how it goes! Follow me on Twitter @drjohnbates where I'll Tweet when possible.

Optimism in the world of financial services regulation

It seems that we’re finally making some progress on making the financial markets function more safely. 

After the “flash-crash” of 6 May, US equity market operators have agreed to bring in coordinated circuit-breakers to avoid a repeat of this extreme event. There is widespread agreement on this. Industry leaders from brokers and exchanges yesterday made supportive statements as part of submissions to the SEC.

Regulators are going public with their use of real-time monitoring technology. Alexander Justham, director of markets at the Financial Services Authority, the UK regulator, told the Financial Times that the use of complex event processing technology will give the FSA “a more proactive machine-on–machine approach” to market surveillance (the FSA is a Progress customer). Other regulators are at least admitting they have a lot of work to do. Mary Schapiro, the SEC chair, believes that the technology used for monitoring markets is “as much as two decades behind the technology currently used by those we regulate”. Scott O’Malia, a commissioner at the Commodity Futures Trading Commission admitted that the CTFC continues to receive account data by fax which then has to be manually entered. 

The use of real-time pre-trade risk technology is likely to become much more widespread. “Naked” access, where customers of brokers submit orders directly to the market without any pre-trade checks, is likely to be banned. This is an important change as late last year Aite Group, an analyst firm, estimated that naked access accounted for 38% of the average daily volume in US stocks. The SEC is also proposing that regulation of sponsored access is shorn up – currently it has evidence that brokers rely upon oral assurances that the customer itself has pre-trade risk technology deployed. The mandated use of pre-trade risk technology will level the playing field and will prevent a rush to the bottom. Personally I’ve heard of several instances of buy-side customers insisting to brokers that pre-trade risk controls are turned off as they perceive that such controls add latency and therefore will adversely affect the success of their trading.

The idea of real-time market surveillance, particularly in complex, fragmented markets as exist in the US and Europe is gaining credence. The SEC has proposed bringing in a “consolidated audit trail” which would enable all orders in US equity markets to be tracked in real-time. As John Bates said in his previous blog post, it’s likely that the US tax-payer will not be happy paying the $4B the publically funded SEC estimates that such a system would need to get up and running. Perhaps the US could look at the way the UK’s FSA is funded. The FSA reports to government but is paid for by the firms it regulates.

As I mentioned in my last blog our polling in April at Tradetech, a European equities trading event, suggests that market participants are ready for better market monitoring. 75% of respondents to our survey believed that creating more transparency with real-time market monitoring was preferable to the introduction of restrictive new rules.

CESR, the Committee of European Securities Regulators, is currently consulting on issues such as algorithmic trading and high frequency trading. It will be interesting to see the results of their deliberations in the coming months.

I’m so pleased the argument has moved on. This time last year saw a protracted period of vilifying “high frequency trading” and “algo trading”. Now, there is recognition of the benefits as well as the challenges that high frequency trading has brought to equity markets and regulators seem to understand that to both prevent disastrous errors and deliberate market manipulation occurring it is better for them to get on board with new technology rather than try to turn the clock back to mediaeval times. 

New approaches are sorely needed. Yesterday saw the conclusion of another investigation into market manipulation when the FSA handed out a $150,000 fine and a five-year ban to a commodity futures broker.

Monitoring and surveillance: the route to market transparency

Again this week, capital markets is under the spotlight, with the SEC and Goldman standoff. Just a few weeks ago, the FSA and UK Serious Organised Crime Agency were making multiple arrests for insider trading. Earlier this year Credit Suisse were fined by the New York Stock Exchange for one of their algorithmic trading strategies damaging the market. Still, electronic trading topics such as dark pools, high frequency trading are being widely debated. The whole capital markets industry is under scrutiny like never before.

Technology can't solve all these problems, but one thing it can do is to help give much more market transparency. We're of the view that to restore confidence in capital markets, organisations involved in trading need to have a much more accurate, real-time view on what's going on. In this way, issues can be prevented or at least identified much more quickly.  I talked about this recently to the Financial Times, here. 

Last week at the Tradetech conference in London, Progress announced its release of a second generation Market Monitoring and Surveillance Solution Accelerator. This is aimed at trading organisations who want to monitor trading behaviour, whether to ensure compliance with risk limits for example, or to spot abusive patterns of trading behaviour. Brokers, exchanges and regulators are particularly relevant, but buy-side organisations can also benefit from it. Previously this solution accelerator just used Apama. Now it's been extended to use our Responsive Business Process (RPM) suite, which includes not only Apama, but Savvion Business Process Management, which extends the accelerator to give it powerful alert and case management capabilities. We know that monitoring and surveillance in capital markets is important now, and believe it will become more so, which is exactly why we've invested in building out product. You can read the take on this from the financial services analyst Adam Honore here and more from Progress about the accelerator and RPM. A video on the surveillance accelerator is here. 

As all this is so relevant at the moment and Tradetech is the largest trading event of its kind in Europe (although very equity focused), we thought we'd conduct some research with the participants. We got exactly 100 responses on one day (which made calculating the percentages rather a breeze) to a survey which asked about attitudes to European regulation, high frequency and algorithmic trading and dark pools. Some of the responses relating to market monitoring and surveillance are worth stating here. 75% of respondents agreed to the premise that creating more transparency with real-time trading monitoring systems was preferable to the introduction of new rules and regulations. 65% of respondents believe that European regulators should be sharing equity trading information in real-time. And more than half believe that their own organisation would support regulators having open, real-time access to information about the firm's trading activity. To me, that's a pretty strong sign that the industry wants to open up, rather than be subjected to draconian new rules.

There will be substantial changes to the European equity trading landscape in the coming year. There will be post MiFID regulation change by the European Commission acting on recommendations by the Committee of European Securities Regulators who are taking industry evidence at the moment. Their mantra, as chanted last week, is "transparency, transparency, transparency". Let's hope that this transparency argument is expressed in opening up markets to more monitoring rather than taking a, perhaps politically expedient, route of outlawing certain practices and restricting others.

Predictions for increased transparency in Capital Markets

It is my view that one of the most significant causes of the global financial crisis was a lack of transparency in financial markets.  Put simply, that means no one, not regulators or market participants, knew what the size of certain derivatives markets (like credit default swaps) was, who held what positions, or what the consequences of holding positions could be.  If financial reform brings nothing else, it should at least hold banks accountable for the business they conduct, and that means full disclosure and constant monitoring by responsible regulators.  

This action would help provide the basis for preventing future crises. No matter how inventive financial products may become, if regulators have complete and detailed information about financial markets and banks’ activities there, better assessments of risk can be made. This means that if necessary, banks’ activities can be reigned in through higher capital requirements or similar measures.  Simply limiting banks’ ability to conduct certain business is a blunt instrument that does not resolve the lack of transparency and likely will hamper economic growth.

Market transparency exhibits itself in many forms. Particularly relevant is that related to electronic trading. Therefore, I predict that regulators will require banks to implement relevant stronger pre-trade risk mechanisms. Regulators, such as the FSA & SEC, will ultimately bring in new rules to mitigate against, for example, the risk of algorithms ‘going mad’. This is exemplified by Credit Suisse, which was fined $150,000 by the NYSE earlier this year for “failing to adequately supervise development, deployment and operation of proprietary algorithms.”

Furthermore, volumes traded via high frequency trading will increase, although at a much slower pace than last year, and at the same time the emotive debates about high frequency trading creating a two-tier system and an unfair market will die down.

In addition, with regards to mid market MiFID monitoring, greater responsibility for compliance will be extended from exchanges to the banks themselves. Banks and brokers will soon be mandated to implement more trade monitoring and surveillance technology. There will also be no leeway on Dark Pools; they just simply have to change and be mandated to show they have adequate surveillance processes and technology in place. They will also have to expose more pricing information to the market and regulators.

This year will see a definite shift to an increasingly transparent – and therefore improved – working environment within capital markets. The ongoing development of market surveillance technologies and changes in attitudes to compliance will drive this forward, creating a more open and fairer marketplace for all.

Peas and Carrots

In the words of the auspicious Forrest Gump some things go together like peas and carrots. Truer words were never spoken. Some things just do go together well, sometimes by design, often by accident. I don't think anyone actually planned milk and cookies or popcorn at the movies but nonetheless these things are made for each other.  When it comes to technology the same harmonious relationships exist.

In the recent Aite report on High Performance Databases (HPDB),  the market for specialized databases is surveyed along with a handful of vendors in this space.  This is a cottage industry where the big database vendors don't play. It's hard to imagine in this day and age where database technology is so standardized and mature and a multitude of choice abounds from commercial products to open source that any other database technology and a gang of vendors would have a chance. Yet it is happening and it's thriving.  

I believe it has to do with a synergistic relationship to event processing. If CEP is the "peas" then HPDB's are the "carrots". These two technologies share two fundamental precepts:

•  A focus on Extreme Performance
•  Temporal Awareness

I. Extreme Performance, Speeds and Feeds
These HPDB's which are often referred to as Tick databases, are found in the same playground as event processing technologies. In the Capital Markets industry they connect to the same market sources, consume the same data feeds. Both technologies are designed to leverage modern multi-core hardware to consume the ever-increasing firehose of data. By the same token, once that data is stored on disk, database query performance is equally important.  The massive amount of data collected and is only as good as the database's ability to query it efficiently thus creating another (historical) firehose of data which an event processing engine would be the consummate consumer.  

II. Temporal Awareness, when is the data
Time is a basic principle in event processing technology, applications typically have as a premise to analyze data-in-motion within a window of time. HPDB's design center is to store and query time series data. Some of the database vendors even bring time to a higher level business function. They understand the notion of a business Calendar, knowing business hours, business week, holidays, trading hours, etc.  Imagine the simplicity of a query where you want 'Business hours Mon-Fri for the month of February' and the database itself would know the third Monday was Presidents Day, skipping over that, thus preventing analytic calculations from skewing erroneously.

Leveraging the Synergy
These two fundamental shared principles provide the basis for a unique set of business cases that are only realized by leveraging Event Processing platforms and High Performance Databases

• Back testing algorithms across massive volumes of historical data compressing time

What if you could test new trading algorithms against the last 6 months or 1 - 2 years of historical market data but run that test in a matter of minutes? What if you could be assured that the temporal conditions of the strategies (i.e. timed limit orders) behaved correctly and deterministically matching the movement of time in complete synchronicity with the historical data? These are just a few of the characteristics that define the harmony between event processing and high performance (Tick) databases.

• Blending live and historical data in real-time

Querying historical data in-flight to obtain volume curves, moving averages, the latest VWAP and other analytics calculations are possible with these high performance databases. Leading edge trading algorithms are blending a historical context with the live market and even News. The winners will be those that can build these complex algo's and maintain ultra low-latency.

• Pre-Trade Risk Management

Managing positions, order limits and exposure is necessary, doing it in real-time to manage market risk is a mandate.  In addition to market data, these high performance databases can store pre and post trade activity to complement event-based trading systems and become the basis for trade reporting systems.

In the Trading LifeCycle, Event Processing and High Performance databases are partner technologies harmoniously bound together to form a union where the whole is greater than the sum of the parts. They are the peas and carrots that together create a host of real-world use-cases that would not be possible as individual technologies.

Myself along with my colleague Dan Hubsher we are doing a 3-part Webinar series entitled "Concept to Profit". The focus is on event processing in the trade lifecycle, but we include cases that touch upon high performance databases. You can still register for part 2: Building Trading Strategies in the Apama WorkBench where I will focus on the tools for strategy development aimed at the IT developer.

Once again thanks for reading, you can follow me at twitter, here.
Louie

My Baby Has Grown Up

I was proud to recently be appointed CTO and head Corporate Development here at Progress Software http://web.progress.com/en/inthenews/progress-software-ap-12102009.html. But I don’t want anyone to take that as an indication that I won’t still be involved with event processing – au contrair. Event processing (whether you call it CEP or BEP) is now a critical part of enterprise software systems – I couldn’t avoid it if I tried!!

But taking a broader role does give me cause to reflect upon the last few years and look back at the growth of event processing and the Progress Apama business. Here are some observations:

• It’s incredibly rare to have the pioneer in a space also be the leader when the space matures. I’m really proud that Progress Apama achieved that. Our former CEO Joe Alsop has a saying that “you don’t want to be a pioneer; they’re the ones with the arrows in their backs!” Usually he’s right on that one – but in the case of Progress Apama, the first is still the best! Independent analysts, including Forrester and IDC, all agree on it. Our customers agree on it too.
• It’s tough at the top! I had no idea that when you are the leader in a space, many other firms’ technology and marketing strategies are based completely around you. I have met ex-employees of major software companies that have told me that there are Apama screenshots posted on the walls of their ex firms’ development centers – the goal being to try to replicate them or even improve on them. Other firms’ marketing has often been based on trying to criticize Apama and say why they are better – so their company name gets picked up by search engines when people search for Apama.
• Event processing has matured and evolved. Yes it is certainly used to power the world’s trading systems. But it’s also used to intelligently track and respond to millions of moving objects, like trucks, ships, planes, packages and people. It’s used to detect fraud in casinos and insider trading. It’s used to detect revenue leakage in telecommunications and continually respond to opportunities and threats in supply chain, logistics, power generation and manufacturing. It enables firms to optimize their businesses for what’s happening now and is about to happen – instead of running solely in the rear view mirror.
• Despite all the new application areas, Capital Markets remains a very important area for event processing. Critical trading operations in London, New York and around the world are architected on event processing platforms. The world’s economy is continually becoming more real-time, needs to support rapid change and now needs to support the real-time views of risk and compliance. We recognize the importance of Capital Market. My congratulations to Richard Bentley who takes on the mantle of General Manager of Capital Markets to carry on Progress Apama’s industry-leading work in this space. With his deep knowledge and experience with both Apama and Capital Markets, Richard is uniquely placed to carry on the solutions-oriented focus that has been the foundation to Progress Apama’s success.
• Even in a terrible economy, the value of event processing has been proven – to manage costs, prevent revenue leakage and increase revenue.  Progress announced our fourth quarter results today http://web.progress.com/en/inthenews/progress-software-an-12222009.html which saw a double digit increase for Apama and triple digit for Actional. Apama and Actional are used, increasingly together, to gain visibility of business processes without modifying applications, to turn business process activity into events and to respond to opportunities and threats represented by event patterns – enabling the dynamic optimization of business performance.
• But one thing I do believe: that soon there will be no such thing as a pure-play CEP vendor. CEP is part of something bigger. We’ve achieved the first mission, which is to raise the profile of event processing as a new technique that can solve hitherto unsolvable problems. Now the follow on mission is to ensure event processing finds its way into every solution and business empowerment platform. It is one of a set of key technologies that together will change the world.

I wish everyone Happy Holidays and a successful and profitable 2010 !!!

High Frequency Trading driving the need to build quickly, run fast

In just about any race there is usually a starting point and a finish line, unless of course you are in an arms race.  For that sort of race there may have been some nebulous beginning in the distant past, but there is no finish line. The race just keeps sprinting along, each competitor angling for an edge, regularly recharging their ammunition supply with some new weaponry to get ahead however slight or temporary.

I recently read an interesting article describing High Frequency Trading as embroiled in an arms race. I certainly believe it's well entrenched in such a conflict, but frankly this combat has arguably had a beneficial net effect especially in that it's contributed to the wellspring of invention, inspiring the creative spirit in all the supporting attributes that make High Frequency Trading a reality. Behind any trader (and trading firm) is an entire armada including the vendors supplying the underlying hardware, networks, software platforms and trading applications.  They are all immersed in the war.  As new hardware, software and/or algo's are deployed it allows the trader to do battle and speed ahead even if it's just for a short while.  Competitive pressures, increasing market volatility, regulatory imperatives, risk mitigation and a host of other challenges are the land mines and roadside bombs on the long and winding road that stall and slow causing re-tooling and re-stocking the ammunition (i.e. algo strategies). There is no time to stop and catch your breath or stand on the roadside.

Sang Lee from the Aite Group reports that High Frequency Trading has had a significant impact on the overall market, providing greater liquidity, tighter spreads and overall improving the quality of the market.  At the macro level these are great advancements and mark a natural evolutionary step due to so many market changes in recent years (i.e. electronic trading venues, adoption of CEP platforms for algo trading, etc.) in Equities and beyond (i.e. FX and Futures & Options).  Down in the trenches, the battles rage on day by day as a multitude of traders and an untold number of algo strategies provide the market liquidity by moving in and out of positions in milliseconds (or even less time). The trading firms engaged in this never ending conflict drive a set of imperatives on software infrastructures for building and deploying algos in the High Frequency battlefield:

Rapid development and customization of algo's

Algo strategies in the High Frequency world have a limited life time. They soon become obsolete (i.e. whatever alpha they took advantage of has disappeared due to the competition, economic changes, or other situations).  To react and respond to this inevitability, having the right sort of tooling to recalibrate strategies is a necessity. This includes graphical modeling tools for Quants to prototype ideas quickly, backtest with historic data, test in a scalable manner to instill confidence prior to production rollout and lastly dynamic parameterization of strategies from graphical dashboards. Not forgetting the code-slinging types, an Integrated Development Environment (IDE) for support of event processing language (EPL) development for more low-level tasks.

Abstracting over increasingly complex strategy logic

Supporting Quants with a rich and robust set of functionality from the basics (connectivity to markets) to the advanced (Linear Algebra, Black Scholes, and other statistical functions).

Support for the 'ilities (availability, security, reliability, ...) to manage the mundane

Deploy with confidence. An important role of software infrastructure is to instill confidence that deployed strategies are always available, securely accessed and run without failure.

Support for scalable performance, providing high throughput and low latency

This is probably the most paramount requirement in the arms race of High Frequency Trading.  The race to the microsecond is pushing both hardware and software vendors alike. Parallelism in CEP engines like Apama's Correlator can leverage multi-core processor architectures like the Intel Nehalem. 

Along with my colleague Dan Hubscher,  I have recorded a 30 minute webinar that describes how the Apama platform along with the Apama Algorithmic Trading Accelerator meet these imperatives.

The pre-recorded webinar, is available here:  Apama Algorithmic Trading Accelerator, Build Quickly, Run Fast.

Once again thanks for reading (plus watching and listening to the webinar in this case), you can also follow me at twitter, here.
Louie

We're going on Twitter

Louis Lovas and myself, Giles Nelson, have started using Twitter to comment and respond to exciting things happening in the world of CEP (and perhaps beyond occasionally!).

The intent is to complement this blog. We'll be using Twitter to, perhaps, more impulsively report our thinking. We see Twitter as another good way to communicate thoughts and ideas.

We would be delighted if you chose to follow our "twitterings" (to use the lingo), and we'll be happy to follow you too.

Click here to follow Louis and here to follow Giles (you'll need to signup for a Twitter account).