Business Making Progress

Our 2012 Predictions: What to expect in capital markets

What will we see for capital markets in 2012?  The countdown to 2012 has begun. On the capital markets horizon is a great deal of change – no surprise to those following this year’s rollercoaster of rogue algorithms and regulation tension. So with no further ado, here are our capital markets predictions for 2012:

 1. Billion Dollar Blunder. At least one financial institution will take a billion dollar (or more) hit when a rogue algorithm goes wild. The algo will go into an infinite loop, taking on an irreversible and un-hedged position, which cannot be shut down. Losses will challenge those by human rogue traders, which banks and financial institutions will prevent from happening next year.

2. Occupy HFT. The public, government and regulators will start the "Occupy HFT" movement -- a popular uprising against the ultimate elite of those making money in this climate. Despite immense financial industry pressure, regulators in both the US and the EU will be panicked by investor and political disapproval of HFT and will rein it in with draconian rules and controls.

3. SEFs Spur Splash Crash. Swaps execution facilities (SEFs) will revolutionize OTC derivatives trading, enabling them to be traded electronically. This, in turn, will lead to increased risk of a cross-asset class swaps "splash crash" which will confound regulators, who have little understanding of these markets.

4. Global Regulation Rocks. Countries will finally realize that regulatory harmonization is a good thing and that individual self-interest is not. Banks and financial services firms will realize that they need to think like regulators, taking control of internal surveillance and compliance before regulators make them do it.

5. RICs Get Smarter. The RICs in BRICs are getting smart order routing and gearing up for an increase in algorithmic trading. This, coupled with looser regulations, will begin to attract regulatory arbitrageurs and Volcker Rule escapees.

6. The Wild East. The West's supremacy in financial markets will further decline as new trading regulations - the Volcker Rule in the US and MiFID in Europe - create a surge of regulatory arbitrage favoring more lightly regulated geographies such as Russia and China. Wall Street and the City of London will lose human and financial capital as a result.

7. Financial Terrorism. An exchange or trading destination will be hacked by financial terrorists intent on manipulating markets for political gain. This will lead exchanges and ECNs to add more stringent monitoring and market surveillance capabilities.

8. Head in the Clouds. Explosive growth in foreign exchange trading and SEFs means that participating firms will require complex hosted solutions. Even the smallest FX broker needs aggregation and pricing services which require a big technology footprint. SEFs present new challenges as swaps markets attract algorithms and require surveillance.

9. Crime & Punishment. Regulators are cracking down hard on financial fraud and market manipulation and they will bring in some big fish in 2012. Prosecutions and punishments will increase in size and in impact.

There you have it – nine predictions for capital markets in 2012. What are your thoughts on these predictions, and have we missed any? Comment below, or tell us on Twitter at @DrJohnBates or @ProgressSW.

Computerized Compliance: Savior or Intruder?

Several top executives of UBS, one of the world’s biggest banks, resigned in disgrace this fall following the announcement that a very junior rogue trader in the London office had managed to lose the astounding sum of $2 billion.

Government regulators had already been pushing banks to make sure they knew their risks and commitments at all times. Clearly, UBS didn’t have the automated systems in place that would have alerted higher-ups to the exposure. Some critics asked how an individual ETF trader could have authorization to take such a huge risk. Presumably UBS had set limits for each level of trader, but a flaw in the system let the man keep increasing his exposure.

Automating compliance in the world banking systems should be a no brainer. With the velocity and volume of transactions, only computers can make the trades, and only computers can monitor them. Some regulators buy real-time market surveillance monitoring from Progress Software, the same company that provides real-time trading software to investors (and the sponsor of this blog).

But computerized compliance can be a two-edged sword. It can be so restrictive that traders can’t do their jobs of creatively managing risk. 

And when corporate managers start pushing their CIOs to monitor compliance in other areas they can get into difficult areas of employee privacy. MIT research fellow Michael Shrage, recently wrote in Harvard Business Review that, “very few CIOs want to become the ‘Chief Interrogation Officer’ or ‘Chief Invasiveness Officer.’ But those are roughly the roles they're being asked to assume as the enterprise dependence on their technologies expands.”  He points to requirements to monitor e-mails and text messages for disclosure of secrets or terms of harassment.

Most companies make it clear to employees that they should have no expectation of privacy for anything they do on a corporate computer or corporate network. And workers are coming to understand that anything bad that they say about their companies on their private Facebook pages or Twitter feeds could get them fired.

The issues of compliance and privacy are becoming more difficult as companies increasingly allow employees to use their own technology, such as iPads and iPhones, on the corporate network. It’s very easy to accidentally write an intemperate e-mail or forward an inappropriate picture with your corporate account rather than your private HotMail.

The ability to monitor all kinds of electronic activity by employees makes it tempting for companies to do so. But they need to carefully consider exactly what they want to monitor. And they should let employees know the boundaries.

Some companies have adopted loose guidelines. Microsoft’s unofficial policy on employee blogging is “don’t be stupid.” But with Millennials entering the workforce, understanding of what is “stupid” may be lacking. Someone has to warn them that lines that were clever on their semester-abroad blog might be offensive in a work Wiki.

Computerized compliance is a necessity in many functions. But companies need to consider carefully what they monitor and what they do with the information they gather.

Can market surveillance help to keep traders on track?

"According to TABB Group new compliance costs are indicated at between 512 and 732 million euro, with ongoing costs between 312 and 586 million euros.  But while regulators are still determining what regulation will look like, the need for market surveillance is undiminished. Traders made about 13.3 billion euros ($18.2 billion) from market manipulation and insider dealing on EU equity markets in 2010, according to an EU commission study.  With some arguing that firms can only do so much to survey markets themselves as trades cross multiple brokers and gateways, the panel discussed the need for fragmented market data to be brought together in a consolidated tape and surveillance performed at an aggregate market-wide level."

This is just an excerpt from a recent post to our Event Processing Blog by Richard Bentley, VP Capital Markets, Progress Software. Can market surveillance help to keep traders on track? Read the entire post.

An Algorithmic Trading and Market Surveillance Wrap Up

Our Capital Markets and Progress Apama teams have been BUSY! Today many of them are recovering from a busy week at the SIFMA Financial Services Technology Expo in New York City.

In addition to lots of greeting and Tweeting, Dan Hubscher even had some time to post a couple blog posts:

• Tickets Please: Technology to Keep You on the Train
• Keeping the Train on Track

And while Dan helped man the floor, our VP of Corporate Communications, John Stewart, worked to get 3 press releases out onto the BusinessWire, including:

• Progress Software and Activ Financial Introduce New Market Data Adapter for Apama Capital Markets Foundation
• Progress Software Survey Reveals the Realities of Regulation, Monitoring and Algorithms on the OTC Derivatives Market
• Progress Software Launches Enhanced Market Surveillance and Monitoring Solution

Not only that, our own Dr. John Bates was tagged by Wall Street & Technology as one of our "Top 10 Innovators of the Decade for Capital Markets”.

Across the pond and beyond, Dr. Richard Bentley was quoted in the Bobsguide article “Suspect movements in share price fall to an eight-year low”. Dr. Giles Nelson traveled to India to promote Progress’ business in Capital Markets. He shared his thoughts on our complex event processing (apama.typepad.com) blog:

• The rise of algo trading in Asia - first stop Mumbai
• Still big room for growth in Japan

Phew! And, just in case you missed it, we wrapped up posting a 4 Part video series on Financial Regulation and Market Surveillance. Here are the blog posts that provide a brief overview and link to the videos:

• PLUS Stock Exchange Making Progress Around Regulation
• Using technology to prepare for regulatory change: how can market surveillance technologies help?
• Preparing for changes in the regulatory approach
• Cutting through the confusion of financial regulation

 

Responsiveness in Retail Banking

A general theme for this blog is how organisations are having to become more responsive, whether to customers, competition, regulation or to squeeze further operational efficiencies out of processes. I want to talk about some of the likely areas of innovation in retail and wholesale banking that will make these organisations more responsive.

Regulation is one obvious area where banks, over a multi-year period, are going to have to become more adept at dealing with change. The implications of the Dodd-Frank legislation in the US is still being understood and, in the UK, banks may be facing an almost existential threat if one gives credence to the rhetoric currently coming out from government and regulators.

Regulation and compliance with regulation is only going to become more and more important. This is being recognised by non-banking institutions that work in financial services. Thomson Reuters, for example, recently announced the formation of a Governance, Risk and Compliance division to deliver information solutions to those needing to comply with regulation. Useful certainly, but regulation will always manifest itself in different ways in different banks. The processes that support regulation must be able to change on a frequent basis. Applications set in concrete only allow this to occur slowly and expensively. The method of building applications needs to be responsive itself and needs to focus around dealing with complex events, business rules, workflow and user interfaces, not around programming language coding.

As an example, a bank, now a customer of Progress Software, had to deal with complying with new regulatory requirements in a matter of weeks. This was not just a box-ticking exercise – the regulator wanted visible demonstration of the systems that had been put in place. The bank’s existing application was simply too inflexible to be changed for reasonable cost and in the time required. An alternative approach, based on Business Process Management and event processing technology was selected which allowed the compliance process flow to evolve and change over time in a much more dynamic way and which could be tailored to meet the bank’s own unique combination of circumstances.

Regulation isn’t a nice to have or something aspirational. It’s a must-have – the risk of legal sanction makes it so.

Another focus for this year will be customer experience. Generally speaking, banks have been behind other industries in getting more responsive to customers’ individual needs and how they’re actually interacting with their bank. A recent US survey, by the industry analyst firm AITE, found that only 22% of small businesses were “extremely” satisfied with their bank and that 65% of larger corporates believed that banks did a bad job of understanding their needs.

A personal anecdote will illustrate a wider point. Recently, I moved money from a savings account with my usual bank to one paying a higher rate of interest with a competitor. The first bank finally contacted me about why I wasn’t using my savings account, but only after all the money had exited the account. By then it was simply too late. Banks still haven’t got sophisticated enough about monitoring how their customers are interacting with their accounts, online, by telephone etc., and sensitively interpreting what these actions might mean and responding in a timely fashion. Again, a watchword here is responsiveness.

Customer onboarding, the process of dealing with a customer’s application for a loan, credit card etc., has got some attention, both to improve the customer’s experience but also to increase the operational efficiency of the bank’s processes.  Some banks have now a real-time view of these processes and can determine immediately whether a high-value business loan should be treated as a greater priority than a car loan or whether a number of card applications have been spending too long receiving credit checks. By knowing this kind of information now (rather than often days or weeks hence) is vital to spotting exceptions quickly and re-prioritizing resources.

One of the core services that a bank performs is payments, whether these payments are executed using cards, direct bank transfer, cheque or otherwise. This year might see some significant innovations in payments processing which will require banks to respond in a way they’ve not been forced to previously. A key tipping point may be Apple introducing a near field communications (NFC) chip into the next iPhone, expected to be released about mid-year. This will allow an iPhone to be swiped across a store terminal to pay for goods. This isn’t that new. Particularly in Japan, NFC chips have been built into mobile phones for some time. However, Apple has a current habit of energising markets and it doesn’t just have the iPhone. There are 160m iTunes accounts which already handle payments. Google and Paypal are also considering offering NFC services. The question is, will this really disrupt existing payment systems which still, even in Paypal’s case, rely upon the core payment infrastructure provided by, primarily, banks? We should hope so. Banks have been hopelessly slow at innovating in payments. It took a diktat from the UK regulator to introduce “faster payments” into the UK. Still too many payments take 3 days, which, in today’s world is very slow for any kind of transaction. Cross-border payments in Europe are still slow and expensive.

If new technology does force payments innovation, banks will be forced to keep up by increasing the flexibility and speed of their own infrastructures to deal with, potentially, many more smaller payments. Guess what? They will need to make them more responsive. If they’re not able to move quickly banks may end up simply being sending round the larger aggregated payments resulting from all the transactions occurring using more modern platforms. Cost pressures will also be present. A recent Boston Consulting Group report indicates that payments providers need to reduce costs by between 10% and 35% by 2012 to keep cost-income ratios stable.

Retail and wholesale banking may often move slowly, but, through regulation, customer pressure and new-entrant innovation, expect some significant changes this year. As Chris Skinner, omnipresent observer of all things banking has recently said on his blog, banks will have to learn “how to be real-time nimble in a world of change”. 

(This first appeared as an opinion column in cio.co.uk)

Red Flags in Morning, Firms Take Warning

A pattern is emerging within new financial services regulations where regulators and financial services firms deploy monitoring technology to "red flag" potential issues such as risk, position limits, errors and manipulation. The "red flags" raised would then alert the relevant personnel or authorities.

In the case of the Volcker Rule, prohibiting banks from proprietary trading and investing in or sponsoring hedge funds or private equity funds, the authorities would use a three tiered approach (http://tinyurl.com/2bh9ot3).  First "tripwires", such as the length of time a trader holds a position, its size or riskiness, would alert banks’ compliance departments  who would (#2) quiz the trader on the nature of the position. And (#3)regulators that keep inspectors on banks’ premises would see the tripwires and monitor both traders and compliance departments.

Over at the CFTC, regulators are looking at a similar approach to monitoring and controlling position limits on products such as oil and metals with a "points" system that would give the CFTC monthly reports that it could use to red-flag traders with large positions (http://tinyurl.com/2ugbdh6).

The tracking and red flag approach is the latest step in increased monitoring of trading operations with the ability to take response before it’s too late. At Progress, we have been advocating using monitoring and surveillance technology to help catch inside trading and avoid fat fingered trading errors for years. With new regulations, monitoring becomes not only mandated but more complicated. Red flags are likely to be flying all over the place within as little as months, both inside and outside financial services firms, presenting a fine opportunity for our Responsive Process Management software solution.

As the financial services world becomes more compliant, the ability to manage red flags becomes more critical. Every process within a financial services firm must be scrutinized, from trade entry to risk management, to analyse and understand internal and external events. This take sophisticated technology. This is where Progress Software's RPM software fits in. According to technology research firm Ovum: "Unless an organization has already made a significant investment in creating an operational responsiveness solution around best of breed products, it will be worth seriously considering the competitive advantage and improved effectiveness that could be achieved by deploying RPM."

Ovum noted in a Technology Audit note that multiple technologies are required to gain a comprehensive insight and respond more rapidly to changes to the environment. These include: business process management (BPM) to model, implement, and execute the processes; business analytics to determine how effectively the processes are working; complex event processing (CEP) to understand the implications of many streams of internal and external events; business rules management to determine the appropriate actions for a given set of conditions and variables; and visibility into end-to-end transactions to track and audit their progress.

The interrelationships between all of these components and the vast amount of information that has become available must be understood before its impact on processes can be ascertained and appropriate tuning performed. In other words, RPM is the answer.

RPM can monitor an increasing number of information feeds, both within or external to the organization, then apply business policy and governance rules, then automatically tune the  established process or alert a human decision-maker (if necessary) and present him/her with current, relevant information on which to base the most appropriate response.

According to Ovum: "All of these individual capabilities already exist (at different levels of maturity), but the cost and complexity of integrating these into an effective business solution is beyond the means of most organizations. Hence Ovum believes that the requirement identified by Progress represents a genuine market opportunity." Well said. 

 

Fighting payment fraud with Complex Event Processing

Today Progress Software announced that SEB bank in Estonia has deployed the Apama Complex Event Processing (CEP) platform to look for payment fraud.

This is exciting news and the fact that such a production customer has gone public is a great validation of SEB's belief in the system they've built. A lot of banks are somewhat reticent to talk publically about fraud. Everyone knows it goes on of course, but banks usually prefer that this is a quiet fight, going on behind the scenes. SEB on the other hand see their use of CEP to detect fraud as a positive demonstration of their commitment to customers and card issuers to ensure that banking with SEB is as safe as possible.

A few weeks ago I visited SEB in Estonia and was hosted by Ain Rasva, the head of technology, together with the head of the fraud team. One of the key reasons they chose to invest in CEP was rising transaction rates and a realisation that conventional data management architectures were simply not suitable to look for fraudulent patterns in a timely fashion. Yes, transactions could be captured in, say, a database management system, but then queries to determine whether a pattern of fraud was emerging had to be run and re-run at arbitrary time intervals. An event processing approach is simply a better way of doing this job, both performance-wise and conceptually. With CEP, SEB can now detect a potentially fraudulent pattern of card use immediately and then start managing the case, using tools to conduct further analysis and to manage communication with the customer. This can be done now in minutes, rather than the hours it took previously. Patterns of fraud detection are constantly changing and SEB needs to be responsive to this - the fraud detection rules need to change frequently and quickly. With Apama, these fraud rules can be created, modified and tested rapidly, and this can be done largely by the fraud department itself rather than relying upon IT to effect each change.

Card payments continue to change and increase. There were 82B payments in Europe in 2009 and card payments are growing at a CAGR of 12% per year. Europe wide regulation such as the Single European Payment Area will significantly change the types of fraud that banks need to look for. SEB has placed itself in a good position to ensure that payments are conducted as safely as possible.

Are you a sitting duck or one that will respond immediately to threats?

While many organisations are being ‘cautiously optimistic’ about what the future holds, the realities of today’s tough business environment could leave them as sitting ducks, according to Rick Reidy, CEO at Progress Software. They might take consolation that they’re in the same pond, but when interest rates in Japan hit near-zero, banks continue to fail and mistakes can lead to a ‘flash crash’, the pond is not a safe place to be. Businesses may have money, but fear and uncertainty is holding back decision-making – we await further regulation and want to know the consequences of recent government changes.

 
Listening to Rick’s keynote at our UK business summit (#progresswsummit, if you want to follow on twitter), in the impressive surrounding of Chelsea Football Club’s ground, London, it seems most of the audience agrees – it’s not good enough to sit around and wait to see if growth returns, and you cannot grow simply by cutting costs. You have to take control of your own ‘growth agenda’, as Rick put it. Businesses that want to survive the next five years need better visibility, through putting processes in place that enable them to react quickly to meet customer demands, adapt to market changes and take advantage of new opportunities. As Rick has advised, businesses need to act on up to the minute information so that leaders can make decisions based on foresight, not hindsight.
 
If you’re a regular reader of this blog you’ll already know that we call this ‘operational responsiveness’: the ability to sense and respond to customer and market changes so that organisations can move quickly to meet challenges and take advantage of new opportunities. 
 
Rick has talked about what this means in the airline industry: the notion of irregular operations has become a weekly reality as companies face intense market pressure, striking staff and disruption from natural phenomenon. ‘Swivel chair’ communication between operational areas is no longer good enough. To react quickly enough, they need responsive processes in place that can help them maintain services and inform customers, almost as-it-happens. If they don’t, they will face massive fines, lost custom and damaged reputation – risks no company can afford at present.
 
We’ll be hearing more from Gordon Penfold, CTO at British Airways, about their approach to becoming operationally responsive to meet the challenges of today and tomorrow. Watch this space for my take on his talk…

 

Stamford Bridge, here we come

Tomorrow sees Progress Software taking over Stamford Bridge, home ground to the world-famous Chelsea Football Club. We’re not just there to check out the players’ dressing rooms – we are being joined by James Caan, of Dragons' Den fame, as well as the great and the good of the UK business community, to discuss how businesses can start to make decisions based on foresight, not hindsight, in their operations.

Gordon Penfold, Chief Technology Officer at British Airways, will be sharing his insight on ‘operational foresight’, revealing how the organization has set itself up to better deal with the irregular operations that have become a fact of life in the last year. And Mike Gualtieri, senior analyst at Forrester Research, will be sharing his views on where the next wave of truly responsive business management is coming from, and which trends to watch for. And Progress' own Chief Executive Officer, Rick Reidy, will be giving a keynote too.

I'll be there, speaking in one session but also blogging and tweeting from the event. So watch this space for the latest updates.

For those of you attending, I look forward to seeing you there.

www.progresssoftwaresummit.com

 

Algorithmic Terrorism

I just posted a blog on the potential of algorithmic trading terrorism -- can a "denial of service" style attack cripple world markets? See the full posting here:

Blog Post: Algorithmic Terrorism >

Mission Operational Responsiveness: Progress RPM and Fraud Prevention

Imagine that you’re a Fortune 500 diversified bank. You have millions of customers worldwide using your credit cards to make purchases every day.

As a diversified bank, your customers expect great value and convenience every time they complete a transaction with you. To retain existing customers and obtain new ones, you are committed to delivering a convenient experience that meets your customers’ high expectations. Whether they have a credit card or savings account, you want their interactions to be seamless as they occur through your website, at ATMs, through merchants and in your branch locations.

Sometimes, things go wrong.

Diversified banks must balance delivering a convenient experience that meets customers’ expectations while continuously monitoring transactions in an effort to prevent bank fraud.

Meet Agent O and relive his fraud prevention adventure as he strives to monitor millions of transactions, when suddenly things go awry. Watch him combat the Syndrome crime family when they hack into ACME Trading Company’s network, stealing thousands of credit card numbers.

Will the Progress® Responsive Process Management (RPM) suite help Agent O stop the crime in time? Watch the video to find out, as the RPM suite is put to the test.

And then tell Progress how you can use RPM to become a business hero and enter to win an Apple iPad.

Beware the weight-challenged digits

Fat fingers (or weight-challenged digits - for the more politically correct :-) ) are trading errors that can have catastrophic consequences. And they've been happening a lot recently!!

Read my full blog post here.

India: Big Potential for Algorithmic Trading

I spent last week in India, a country that, by any standards, is growing fast.  Its population has doubled in the last 40 years to 1.2B and economic growth has averaged more than 7% per year since 1997.  It’s projected to grow at more than 8% in 2010. By some measures, India has the 4th biggest economy in the world. 

Progress Software has a significant presence in India. In fact, people-wise, it’s the biggest territory for Progress outside the US with over 350 people. Hyderabad is home to a big development centre and Mumbai (Bombay) has sales, marketing and a professional services team.

The primary purpose of my visit was to support an event Progress organised in Mumbai on Thursday of last week on the subject of algorithmic trading. It was also our first real launch of Progress and Apama, our Complex Event Processing (CEP) platform, into the Indian capital markets. We had a great turnout, with over 100 people turning up. I spoke about what we did in capital markets and then participated in a panel session where I was joined by the CTO of the National Stock Exchange, the biggest in India, a senior director of SEBI, the regulator, and representatives from Nomura and Citigroup. A lively debate ensued.

The use of algorithmic trading is still fairly nascent in India, but I believe it has a big future. I’ll explain why soon, but I’d like first to give some background on the Indian electronic trading market, particularly the equities market, which is the largest.

Watch my Interview on NDTV Profit >

The market: India has several, competing markets for equities, futures and options, commodities and foreign exchange too.  In equities, the biggest turnover markets are run by the National Stock Exchange (NSE) and the Bombay Stock Exchange (BSE), with market shares (in the number of trades) of 74% and 26% respectively. Two more equity exchanges are planning to go live soon – the Delhi Stock Exchange is planning to relaunch and MCX is also currently awaiting a licence to launch. This multi-market model, only recently adopted in Europe for example, has been in place in India for many years.

It was only two years ago that direct market access (DMA) to exchanges was allowed. Although official figures don’t exist, the consensus opinion is that about 5% of volume in equities is traded algorithmically and between 15% and 25% in futures and options. Regulation in India is strong - no exchange allows naked access and the BSE described to me some of the strongest pre-trade risk controls I’ve come across - collateral checks on every order before they are matched. The NSE has throttling controls which imposes a limit on the number of orders a member organisation can submit per second. Members can be suspended from trading intra-day if this is exceeded. The NSE also forces organisations who want to use algorithms to go through an approval process. I’ll say more about this later. Controversially, the NSE will not allow multi-exchange algorithmic strategies so cross-exchange arbitrage and smart-order routing cannot take place. Lastly, a securities transaction tax (STT) is levied on all securities sales.

So, with the above restrictions, why do I think that the Indian market for algorithmic trading has massive potential?

The potential: The Indian market is very big. Surprisingly so to many people. Taking figures from the World Federation of Stock Exchanges (thus I’m not counting trading on alternative equity venues such as European multi-lateral trading facilities), the Indian market, in dollar value, may still be relatively modest – it’s the 10th largest. However, when you look at the number of trades, India’s the 3rd largest market, only beaten by the US and China. The NSE, for example, processes 10 times the number of trades as the London Stock Exchange. So why isn’t more traded in dollar terms? That’s because trade sizes on Indian exchanges are very small. The median figure worldwide is about $10K per trade. The figure in India is about $500 per trade, a 20th of the size. In summary, surely the task of taming the complexity of this number of trades and the orders that go with them is ideal for algorithmic trading to give an edge? To compare to another emerging, “BRIC”, economy, that of Brazil, where the number of firms using Apama has gone from zero to over 20 in as many months, the dollar market size is fairly similar but the number of equity trades in India is 33 times more. The potential in India is therefore enormous.

India is already there in other ways. All exchanges are offering co-location facilities for their members and debate has already moved on to that common in more developed markets on whether this gives certain firms an unfair advantage or not and whether co-location provision should be regulated.

The challenges: There are some difficulties. The STT is seen by some as an inhibitor. However, its effect is offset somewhat by the fact that securities traded on exchange are not subject to capital gains tax.

The NSE process for approving algorithms is more controversial. Firms that want to algorithmically trade must show to the NSE that certain risk safeguards are in place and “demonstrate” the algorithm to the exchange. As the biggest exchange, the NSE wields considerable power and thus its decision to vet algorithms puts a brake on market development. I believe this process to be unsustainable for the following reasons:

1. As the market develops there will simply be too many algorithms for the NSE to deal with in any reasonable timeframe. Yes, India is a low-cost economy, but you need highly trained people to be able to analyse algorithmic trading systems. You can’t simply throw more people at this. Firms will want to change the way algorithms work on a regular basis. They can’t do this, with this process in place.
2. It raises intellectual property issues. Brokers will increasingly object to revealing parts of their algorithms and their clients, who may want to run their alpha seeking algorithms on a broker-supplied co-location facility, will most definitely object.
3. It puts the NSE in an invidious position. Eventually an algo will “pass” the process and then go wrong, perhaps adversely affecting the whole market. The NSE will have to take some of the blame.
4. Competition will force the NSE’s hand. The BSE is trying to aggressively take back market share and other exchanges are launching which will not have these restrictions.

It strikes me that the NSE should spend its efforts into ensuring that it protects itself better. Perhaps a reasonable comparison is a Web site protecting itself from hacking and denial of service attacks. If they can do it, so can an exchange. And it would offer much better protection for the exchange and the market in general.

In conclusion: I’m convinced of the growth potential in India for algorithmic trading. The market is large, the user base is still relatively small and many of the regulatory and technical prerequisites are in place. There are some inhibitors, outlined above, but I don’t think they’ll hold the market back significantly. And finally, why should India not adopt algo trading when so many other, and diverse, markets have?

Progress has its first customers already in India. I look forward to many more.

CFTC Launches Technology Advisory Committee

Yesterday the CFTC, the regulator in charge of Futures and Options markets, announced a new Technology Advisory Committee (TAC), chaired by the very capable Commissioner Scott O’Malia. Read the complete article.

I am absolutely delighted to be included in the group of experts that the CFTC has called together to form the TAC. I am joined by an extraordinary group of some of the industry's top executives from banks, brokers, trading firms, exchanges and clearing firms as well as some very impressive academics. On Wednesday, July 14th (tomorrow as I write), we will meet to discuss the impact of high frequency and algorithmic trading on the markets, including whether algorithms may be implicated in the May 6th 'flash crash'. From this, we’ll discuss what recommendations we have for regulation of and/or best practices for algorithmic and high frequency trading.

High frequency and algorithmic trading are essential for efficient execution and alpha generation in a complex, multi-asset, fast-moving world. However, there are a number of accusations that have been made against these forms of trading, including that they may aggravate volatility and may even have caused the ‘flash crash’. I believe evidence from the TAC participants will exonerate the accused.

I am hoping that our meetings will result in solutions that not only head-off future ‘flash crashes’, but also help exchanges, banks and brokers to better monitor and police trades. The proactive use of real-time monitoring systems can alert regulators to problems before they become a crisis. Monitoring technology can 'see' major price and volume spikes in particular instruments, how often they happen and maybe even why, and whether a pattern in market behavior caused them. It can also tell how much trading is potentially market abuse, for example, insider trading might be detected by correlating unusual trading incidents with news releases and market movements. (The FSA, for example, thinks that 30% of trading around acquisitions is insider.)

It is now possible to apply high frequency techniques to not just trading – but also to market monitoring, surveillance and pre-trade risk checks – for regulators, exchanges and brokers. The technology is out there (with proven approaches built on next generation platforms such as complex event processing or CEP) and it needn't be expensive. The CFTC's TAC is a positive step in the right direction. I look forward to the meeting and will let you know how it goes! Follow me on Twitter @drjohnbates where I'll Tweet when possible.

Optimism in the world of financial services regulation

It seems that we’re finally making some progress on making the financial markets function more safely. 

After the “flash-crash” of 6 May, US equity market operators have agreed to bring in coordinated circuit-breakers to avoid a repeat of this extreme event. There is widespread agreement on this. Industry leaders from brokers and exchanges yesterday made supportive statements as part of submissions to the SEC.

Regulators are going public with their use of real-time monitoring technology. Alexander Justham, director of markets at the Financial Services Authority, the UK regulator, told the Financial Times that the use of complex event processing technology will give the FSA “a more proactive machine-on–machine approach” to market surveillance (the FSA is a Progress customer). Other regulators are at least admitting they have a lot of work to do. Mary Schapiro, the SEC chair, believes that the technology used for monitoring markets is “as much as two decades behind the technology currently used by those we regulate”. Scott O’Malia, a commissioner at the Commodity Futures Trading Commission admitted that the CTFC continues to receive account data by fax which then has to be manually entered. 

The use of real-time pre-trade risk technology is likely to become much more widespread. “Naked” access, where customers of brokers submit orders directly to the market without any pre-trade checks, is likely to be banned. This is an important change as late last year Aite Group, an analyst firm, estimated that naked access accounted for 38% of the average daily volume in US stocks. The SEC is also proposing that regulation of sponsored access is shorn up – currently it has evidence that brokers rely upon oral assurances that the customer itself has pre-trade risk technology deployed. The mandated use of pre-trade risk technology will level the playing field and will prevent a rush to the bottom. Personally I’ve heard of several instances of buy-side customers insisting to brokers that pre-trade risk controls are turned off as they perceive that such controls add latency and therefore will adversely affect the success of their trading.

The idea of real-time market surveillance, particularly in complex, fragmented markets as exist in the US and Europe is gaining credence. The SEC has proposed bringing in a “consolidated audit trail” which would enable all orders in US equity markets to be tracked in real-time. As John Bates said in his previous blog post, it’s likely that the US tax-payer will not be happy paying the $4B the publically funded SEC estimates that such a system would need to get up and running. Perhaps the US could look at the way the UK’s FSA is funded. The FSA reports to government but is paid for by the firms it regulates.

As I mentioned in my last blog our polling in April at Tradetech, a European equities trading event, suggests that market participants are ready for better market monitoring. 75% of respondents to our survey believed that creating more transparency with real-time market monitoring was preferable to the introduction of restrictive new rules.

CESR, the Committee of European Securities Regulators, is currently consulting on issues such as algorithmic trading and high frequency trading. It will be interesting to see the results of their deliberations in the coming months.

I’m so pleased the argument has moved on. This time last year saw a protracted period of vilifying “high frequency trading” and “algo trading”. Now, there is recognition of the benefits as well as the challenges that high frequency trading has brought to equity markets and regulators seem to understand that to both prevent disastrous errors and deliberate market manipulation occurring it is better for them to get on board with new technology rather than try to turn the clock back to mediaeval times. 

New approaches are sorely needed. Yesterday saw the conclusion of another investigation into market manipulation when the FSA handed out a $150,000 fine and a five-year ban to a commodity futures broker.

Monitoring and surveillance: the route to market transparency

Again this week, capital markets is under the spotlight, with the SEC and Goldman standoff. Just a few weeks ago, the FSA and UK Serious Organised Crime Agency were making multiple arrests for insider trading. Earlier this year Credit Suisse were fined by the New York Stock Exchange for one of their algorithmic trading strategies damaging the market. Still, electronic trading topics such as dark pools, high frequency trading are being widely debated. The whole capital markets industry is under scrutiny like never before.

Technology can't solve all these problems, but one thing it can do is to help give much more market transparency. We're of the view that to restore confidence in capital markets, organisations involved in trading need to have a much more accurate, real-time view on what's going on. In this way, issues can be prevented or at least identified much more quickly.  I talked about this recently to the Financial Times, here. 

Last week at the Tradetech conference in London, Progress announced its release of a second generation Market Monitoring and Surveillance Solution Accelerator. This is aimed at trading organisations who want to monitor trading behaviour, whether to ensure compliance with risk limits for example, or to spot abusive patterns of trading behaviour. Brokers, exchanges and regulators are particularly relevant, but buy-side organisations can also benefit from it. Previously this solution accelerator just used Apama. Now it's been extended to use our Responsive Business Process (RPM) suite, which includes not only Apama, but Savvion Business Process Management, which extends the accelerator to give it powerful alert and case management capabilities. We know that monitoring and surveillance in capital markets is important now, and believe it will become more so, which is exactly why we've invested in building out product. You can read the take on this from the financial services analyst Adam Honore here and more from Progress about the accelerator and RPM. A video on the surveillance accelerator is here. 

As all this is so relevant at the moment and Tradetech is the largest trading event of its kind in Europe (although very equity focused), we thought we'd conduct some research with the participants. We got exactly 100 responses on one day (which made calculating the percentages rather a breeze) to a survey which asked about attitudes to European regulation, high frequency and algorithmic trading and dark pools. Some of the responses relating to market monitoring and surveillance are worth stating here. 75% of respondents agreed to the premise that creating more transparency with real-time trading monitoring systems was preferable to the introduction of new rules and regulations. 65% of respondents believe that European regulators should be sharing equity trading information in real-time. And more than half believe that their own organisation would support regulators having open, real-time access to information about the firm's trading activity. To me, that's a pretty strong sign that the industry wants to open up, rather than be subjected to draconian new rules.

There will be substantial changes to the European equity trading landscape in the coming year. There will be post MiFID regulation change by the European Commission acting on recommendations by the Committee of European Securities Regulators who are taking industry evidence at the moment. Their mantra, as chanted last week, is "transparency, transparency, transparency". Let's hope that this transparency argument is expressed in opening up markets to more monitoring rather than taking a, perhaps politically expedient, route of outlawing certain practices and restricting others.

Observations from Tradetech 2010

Day one of Tradetech Europe 2010 has nearly finished. I won't be here tomorrow, so here are some thoughts and take-aways from today's event.

It's fair to say that Tradetech is the premier European equities trading and technology event, and thus very relevant for Progress' business in capital markets, particularly customers using Apama. Progress has a substantial presence as always. It's a good event to meet brokers, hedge funds, exchanges and pretty much every one within the industry. Lots of old friends are here every year. Regarding the event itself, it's pretty well attended considering the recent issues with volcanic ash. It usually takes place in Paris, but I'm sure the organisers were pleased that they chose London this year as the London contingent was able to attend without disruption.

This years big theme really seems to be market structure and regulation. In the third year after MiFID, an event which brought competition into European equity markets, and after the credit crunch, issues about how the market is working, the influence of alternative venues such as dark pool,  and how high-frequency trading is affecting the market are issues front of mind.

What's interesting is how some things stay the same. Richard Balarkas, old Tradetech hand and CEO of Instinet Europe, talked about trading liberalisation in the late 19th and early 20th century. Then, vested interests were complaining about the rise of "bucket shops", giving access to trading on the Chicago Board of Trade via telegraph to people that wouldn't previously have traded. In the view of some at the time, this lead to speculation and "gambling". Regulators were wrestling at the time with the fact that only 1% of CBOT trades resulted in actual delivery of goods - the rest were purely financial transactions and therefore arguably speculative. This reminds me of some of the current debate around the "social usefullness" of high frequency trading which is going on now.

European equities trading has changed a lot. Vodafone, a UK listed stock, has now only about 30% of its average European daily volume traded on the London Stock Exchange (LSE). The rest is traded on alternative trading venues across Europe. However, Xavier Rolet, CEO of the LSE, believes that there's a long way to go. He stated  that "the European equities market remains anaemic when compared to the US". Volumes, adjusted for relative market capitalisation, are about 15% of that in the US.

Regulation of European markets is a thorny issue. Regulation is fragmented, together with the market itself. CESR - the Committee of European Securities Regulators, the nearest Europe has to a single regulator - is taking evidence on a whole range of issues and will recommend a set of reforms to the European Commission in July this year. These recommendations will relate to post-trade transparency and information quality and enhanced information about systematic internalisers and broker crossing systems. CESR is also looking at other issues such as algorithmic trading and co-location. Legislation will follow towards the end of 2010.

Equity markets are in a sensitive place. There's still more deregulation to do, more competition to be encouraged and yet, with sentiment as it is, regulators may decide to introduce more rules and regulations to prevent this taking place. The CESR proposals will be about "transparency, transparency, transparency" - as part of this we believe that more real-time market monitoring and surveillance by all participants is key to bringing back confidence in the markets and ensuring that draconian rules don't have to be introduced.

Emerging markets were talked about in one session, and Cathryn Lyall from BM&FBovespa in the UK, talked about Brazil in particular. We've seen Brazil become a pretty significant market recently. Not only have demand grown for all Progress products substantially but Apama is now being used by 18 clients for algorithmic trading of both equities and derivatives. Brazil is the gorilla in the Latin American region. It accounts for 90% of cash equities and 95% of derivatives business in Latin America. 90% of Brazilian trading is on exchange. Brazil emerged largely unscathed from the credit crunch and it's taken only 2-3 years to achieve the level of trading infrastructure that took perhaps 10-15 years to evolve in the US and Europe. More still needs to happen. Although the regulatory regime has an enviable reputation, it is moving slowly. Concerns regarding naked and sponsored access are holding up liberalisation that would lead to DMA and co-located access to the equities market, something which is place already for derivatives.

So, that's what I saw as highlights from the day. Tradetech seems, still, to be the place the whole industry gathers.

Predictions for increased transparency in Capital Markets

It is my view that one of the most significant causes of the global financial crisis was a lack of transparency in financial markets.  Put simply, that means no one, not regulators or market participants, knew what the size of certain derivatives markets (like credit default swaps) was, who held what positions, or what the consequences of holding positions could be.  If financial reform brings nothing else, it should at least hold banks accountable for the business they conduct, and that means full disclosure and constant monitoring by responsible regulators.  

This action would help provide the basis for preventing future crises. No matter how inventive financial products may become, if regulators have complete and detailed information about financial markets and banks’ activities there, better assessments of risk can be made. This means that if necessary, banks’ activities can be reigned in through higher capital requirements or similar measures.  Simply limiting banks’ ability to conduct certain business is a blunt instrument that does not resolve the lack of transparency and likely will hamper economic growth.

Market transparency exhibits itself in many forms. Particularly relevant is that related to electronic trading. Therefore, I predict that regulators will require banks to implement relevant stronger pre-trade risk mechanisms. Regulators, such as the FSA & SEC, will ultimately bring in new rules to mitigate against, for example, the risk of algorithms ‘going mad’. This is exemplified by Credit Suisse, which was fined $150,000 by the NYSE earlier this year for “failing to adequately supervise development, deployment and operation of proprietary algorithms.”

Furthermore, volumes traded via high frequency trading will increase, although at a much slower pace than last year, and at the same time the emotive debates about high frequency trading creating a two-tier system and an unfair market will die down.

In addition, with regards to mid market MiFID monitoring, greater responsibility for compliance will be extended from exchanges to the banks themselves. Banks and brokers will soon be mandated to implement more trade monitoring and surveillance technology. There will also be no leeway on Dark Pools; they just simply have to change and be mandated to show they have adequate surveillance processes and technology in place. They will also have to expose more pricing information to the market and regulators.

This year will see a definite shift to an increasingly transparent – and therefore improved – working environment within capital markets. The ongoing development of market surveillance technologies and changes in attitudes to compliance will drive this forward, creating a more open and fairer marketplace for all.

The Debate on Dark Pools

Dr. Giles Nelson, Chief Technology Strategist with Progress Software and a co-founder of Apama, has weighed in on the continuing debate about the possible need to regulate dark pools (Why The Outlawing Of "Dark Liquidity Pools" Debate Rumbles On) in FreshBusinessThinking.com.  Beyond the thoughtful summary of the current issues, one might want to read any article that includes a reference to "the devil's spawn."   

My Baby Has Grown Up

I was proud to recently be appointed CTO and head Corporate Development here at Progress Software http://web.progress.com/en/inthenews/progress-software-ap-12102009.html. But I don’t want anyone to take that as an indication that I won’t still be involved with event processing – au contrair. Event processing (whether you call it CEP or BEP) is now a critical part of enterprise software systems – I couldn’t avoid it if I tried!!

But taking a broader role does give me cause to reflect upon the last few years and look back at the growth of event processing and the Progress Apama business. Here are some observations:

• It’s incredibly rare to have the pioneer in a space also be the leader when the space matures. I’m really proud that Progress Apama achieved that. Our former CEO Joe Alsop has a saying that “you don’t want to be a pioneer; they’re the ones with the arrows in their backs!” Usually he’s right on that one – but in the case of Progress Apama, the first is still the best! Independent analysts, including Forrester and IDC, all agree on it. Our customers agree on it too.
• It’s tough at the top! I had no idea that when you are the leader in a space, many other firms’ technology and marketing strategies are based completely around you. I have met ex-employees of major software companies that have told me that there are Apama screenshots posted on the walls of their ex firms’ development centers – the goal being to try to replicate them or even improve on them. Other firms’ marketing has often been based on trying to criticize Apama and say why they are better – so their company name gets picked up by search engines when people search for Apama.
• Event processing has matured and evolved. Yes it is certainly used to power the world’s trading systems. But it’s also used to intelligently track and respond to millions of moving objects, like trucks, ships, planes, packages and people. It’s used to detect fraud in casinos and insider trading. It’s used to detect revenue leakage in telecommunications and continually respond to opportunities and threats in supply chain, logistics, power generation and manufacturing. It enables firms to optimize their businesses for what’s happening now and is about to happen – instead of running solely in the rear view mirror.
• Despite all the new application areas, Capital Markets remains a very important area for event processing. Critical trading operations in London, New York and around the world are architected on event processing platforms. The world’s economy is continually becoming more real-time, needs to support rapid change and now needs to support the real-time views of risk and compliance. We recognize the importance of Capital Market. My congratulations to Richard Bentley who takes on the mantle of General Manager of Capital Markets to carry on Progress Apama’s industry-leading work in this space. With his deep knowledge and experience with both Apama and Capital Markets, Richard is uniquely placed to carry on the solutions-oriented focus that has been the foundation to Progress Apama’s success.
• Even in a terrible economy, the value of event processing has been proven – to manage costs, prevent revenue leakage and increase revenue.  Progress announced our fourth quarter results today http://web.progress.com/en/inthenews/progress-software-an-12222009.html which saw a double digit increase for Apama and triple digit for Actional. Apama and Actional are used, increasingly together, to gain visibility of business processes without modifying applications, to turn business process activity into events and to respond to opportunities and threats represented by event patterns – enabling the dynamic optimization of business performance.
• But one thing I do believe: that soon there will be no such thing as a pure-play CEP vendor. CEP is part of something bigger. We’ve achieved the first mission, which is to raise the profile of event processing as a new technique that can solve hitherto unsolvable problems. Now the follow on mission is to ensure event processing finds its way into every solution and business empowerment platform. It is one of a set of key technologies that together will change the world.

I wish everyone Happy Holidays and a successful and profitable 2010 !!!

In defence of high frequency trading

The high frequency trading (HFT) debate seems to have entered a new and worrying phase in the UK. On Tuesday this week in an interview with the BBC, Lord Myners, the UK’s financial services minister, warned that high frequency trading had “gone too far” and that share ownership had “now lost its supporting function for the provision of capital to business”. (You can find the original interview here and reports of it in the Financial Times and The Independent yesterday).

 

 Mary Schapiro, head of the SEC, signalled at the end of October that a number of electronic trading areas were going to be looked into – naked access (where a broker sponsors a firm to have direct electronic access to an exchange), dark pools and high frequency trading.

 

It does seem now that on both sides of the Atlantic, governments and regulators are steeling themselves to act and softening the markets up to be able to accept the fact that electronic trading might have some limits.

 

The concern is that governments and regulators are going to come down too hard on electronic trading and the benefits that it gives investors will be damaged.

 

It all started with the flash order issue in the US a few months ago. Commentators were linking together various different, although related issues, in an inappropriate way. Flash orders seemed to be viewed sometimes as being synonymous with HFT, both of which were sometimes reported as forms of market abuse. All three topics are quite different. In my opinion, there are legitimate questions over the use of flash orders and a proposal to ban them is now being considered.

 

Dark pools, where large blocks of stock are traded off exchange to minimise market impact, have been the next targets. There are, again, legitimate issues. Dark pools, by their very nature, do not have good price transparency. Regulators have become concerned with their use because more and more trading is going through dark pools. Some estimates put this at between 10% and 30% in Europe and the US. This lack of knowledge about what exactly is the proportion is part of the problem itself. No one really knows what proportion of trading dark pools is taking. If a significant proportion of the market has no price transparency then this undermines the notion of a fair market for all. Regulators are looking at this and its likely that they will force dark pool operators to disclose far more information about what is being traded than they do currently. The SEC is considering limiting the proportion of a stock that can be traded through dark pools to a small percentage.

 

These legitimate issues however risk skewing the whole HFT debate to one where people will conclude that “HFT is bad”.

 

What people are now describing as HFT – the very fast and frequent, computer assisted trading of, usually, equities – is an evolution of something that has been happening in the market place for at least the last 10 years. In this time electronic trading has proliferated, not just in equities but also in all asset classes such as derivatives, bonds and foreign exchange. Far more venues for trading have been created. There are now many places where a company’s stock can be traded both in the US and Europe. This has brought competition and choice. Prices have been lowered, improving access to retail investors. Spreads have narrowed. Arbitrage opportunities are harder to find, which mean that market information is disseminating faster which, in turn, means that price transparency has improved. Because there is more trading going on, there is more liquidity available, which also means keener prices.

 

A key part of the HFT trend has been the use of algorithmic trading (the most prevalent use of complex event processing technology). Algo trading models fall broadly into one of two camps: alpha seeking, where market prices are examined to find a trading opportunity that will make money, and execution where orders are, usually, split up into smaller parts and then traded automatically in the market in an intelligent way to find good prices and to ensure those prices are not overly influenced by the trades being made themselves. For each type of model it can be very useful to react very quickly to market information, either to take advantage of a price discrepancy or to quickly pickup liquidity at a good price. Algorithmic trading is enormously beneficial for those who use it and its use is not limited to specialist hedge funds. Most algorithmic trading uses execution models that find liquidity, good prices, help minimise market impact and, lastly, increase significantly a trader’s productivity. Instead of wasting time executing several simple orders in the market over the course of many minutes or hours, the trader can simply ask a machine to do it. The trader can then spend time either covering more of the market (useful in straitened economic times) or spend more time actually delivering real value to a client.

 

Algorithmic trading and HFT have brought very significant benefits. It is these benefits that must not be threatened.

 

Trading has always involved cunning and guile, whether human or computer based. Competition has always existed in who’s got the best traders and trading systems. Organisations investing in ultra low-latency infrastructure to ensure orders arrive at an exchange in microseconds (not nanoseconds as sometimes claimed by the way – light travels 30cm in 1 nanosecond which isn’t far enough to be very useful) are part of this competitive world. Competition leads to innovation and it is this innovation that has brought so many of the benefits described above. Computer-based models can somtimes be used abusively. There are many forms of market abuse that regulators and exchange operators look for. Some exchanges and regulators have been investing in real-time surveillance technology (Progress counts Turquoise and the UK Financial Services Authority as customers using Apama) to ensure that they can spot abusive patterns of behaviour quickly.

 

We can’t start slowing trading down. We can’t go backwards and put the electronic trading genie back in the bottle. We don’t want to lose all the benefits that have come. Rather, regulators and exchanges should concentrate on ensuring maximum transparency in how markets operate and ensure that those attempting to maliciously abuse the markets are dissuaded or caught.

 

Apama 4.2 release - Cruising in the fast lane

The Apama engineering team has done it once again. True to our record of releasing significant new features in the Apama product every 6 months, the v4.2 release is hot off the presses with major new functionality. The Apama roadmap is driven by a keen sense of our customer requirements, the competitive landscape and an opportunistic zeal. The engineering team is a dedicated R&D team driven to excellence and quality. We are dedicated to delivering value to our customers. A consistent comment we've heard from analysts and customers alike is the maturity of the Apama product.  

The current v4.2 release, the third in the v4.x family adds significant enhancements in three concurrent themes - Performance, Productivity and Integration. This consistent thematic model is one we've held for a number of years. Below I've touched upon the highlights of the current release along these themes:

• Performance

High Performance Parallelism for Developers.  The Apama Event Processing Language (EPL) provides a set of features uniquely suited to build scalable event-driven applications.  The language natively offers capabilities for event handling, correlating event streams, pattern matching and defining temporal logic, etc. Equally important, the language provides a flexible means to process events in parallel.  For this we provide a context model and a new high performance scheduler. Contexts can be thought of as silos of execution, where CEP applications run in parallel. The scheduler's role is to manage the runtime execution in an intelligent high-performance way, and to leverage the underlying operating system threading model. It’s via the context architecture that the Apama Correlator squeezes the most out of operating system threads to achieve maximum use of multi-core processors for massive vertical scalability. For IT developers, this is a effective and efficient means to build high performance, low latency CEP applications without the pitfalls of thread-based programming, such as deadlocks and race conditions.

High Performance Parallelism for Business Analysts.  Not to be left out of the race, we've also ensured the scalable parallelism provided in the Apama CEP engine is available through our graphical modeling tool, the Event Modeler. We've had this graphical modeling capability since the very first release of Apama. This tool designed for analysts, quantitative researchers and of course developers, allows you to design and build complete CEP applications is a graphical model.  Parallelism is as easy as an automatic transmission, simply select P for parallel.

• Productivity

Real men do use Debuggers (and Profilers too). The Apama Studio now sports major new functionality for development, a source level debugger and a production profiler. Building applications for an event-driven world presents new programming challenges. Having state-of-the-art development tools for this paradigm is a mandate. The Apama EPL is the right language for building event-driven applications - now we have a source-level debugger designed for this event paradigm. Available in the Eclipse-based Apama Studio it provides breakpoints to suspend applications at specific points, examine contents of program variables and single stepping. It works in concert with our parallelism as well. Profiling is a means to examine deployed Apama applications to identify possible bottlenecks in CPU usage.

Jamming with Java. We've enhanced our support for Java for building CEP applications. The Apama Studio includes a complete set of wizards for creating monitors, listeners, and events to improve the development process when building java-based CEP applications in Apama.

• Integration

The (relational) world plays the event game. While we have provided connectivity to relational databases for many years we've made a significant re-design in the architecture of how we do it with the new Apama Database Connector (ADBC). The ADBC provides a universal interface to any database and includes standard connectors to ODBC and JDBC.  Through the ADBC, Apama applications can store and retrieve data in standard database formats using general database queries, effectively turning these relational engines into timeseries databases. The data can be used for application enrichment and playback purposes. To manage playback the Apama Studio includes a new Data Player that enables back-testing and event playback from a range of data sources via the ADBC. One can replay at varying speeds event data and time itself. The tested CEP applications behaves temporally consistent even as data is replayed at lightening speed.

Cruising at memory speed with MemoryStore. The MemoryStore is a massively scalable in-memory caching facility with in-built navigation,  persistence and visualization functionality.  This allows CEP applications, which typically scan, correlate and discard data very quickly to retain selected portions in memory for later access at extreme speed. This could be for managing a financial Order Book, Payments or other data elements that the application needs to be able to access at user’s requests quickly. Furthermore, if required the in-memory image can be persisted to a relational database for recovery or other retrieval purposes, and lastly the MemoryStore allows selected portions of the in-memory cache to be automatically mapped to dashboards.

Well that's the highlights. There were also about a dozen other features within each of these three themes, just too numerous to mention.

We are committed to improving the Apama product by listening to our many customers, paying close attention to the ever-changing competitive landscape and researching new opportunities.

Again thanks for reading, you can also follow me at twitter, here.
Louie

Progress Apama Capital Markets in Brazil

In recent years, the Brazilian market has grown stronger, and become very aggressive with algorithmic trading. Just back from a conference in Brazil, listen to this podcast where Dan Hubscher shares insight into the current state of Brazil’s market, and what the people down there are buzzing about.

Apama SIFMA 09 Announcements

SIFMA is happening this week in NY and this typically sparks a flurry of announcements in Capital Markets, as this event can be seen as a benchmark for what is happening in the industry, similar to TradeTech Paris for Europe earlier in the Spring.  Apama has made a series of significant announcements that capture some of the scope of the Apama platform and what we believe is required to be successful in this market.  In different niches of the blogosphere you’ll find some proponents of fairly narrow definitions of what “CEP” is or how one measures CEP “leadership” or “maturity”.  I think the Apama announcements illustrate a different perspective, reflective of the breadth of the Apama platform and the possibilities available to such a platform in terms of building new event-driven solutions in Capital Markets.

Market Surveillance and Monitoring Accelerator

Apama has previously announced market surveillance customers, FSA and Turquoise, and with our new enhanced Solution Accelerator we are capturing our capabilities in a way that allows regulators, exchanges/MTFs, and trading firms to further jumpstart deployment of monitoring applications.  The idea that it would be efficacious to have real-time monitoring of trading is becoming better understood in the indusry.  If you read the release, you’ll note that the targets are not just regulators or exchanges, but firms themselves, who would benefit from detecting potentially damaging activities prior to that activity hurting firm reputations or profits.

Apama Solution Accelerators provide a set of core functions that focus on specific domain areas, but they still retain the flexibility to evolve and adjust as circumstances require.  That is key to keeping pace with very dynamic market conditions.  These Accelerators have proven a key driver to our recent success as they marry the power of the underlying platform with the real “end game” of providing customers with solutions that deliver value.

UniCredit Customer Win

As an example of the power of Solution Accelerators, Apama also announced that UniCredit is using the Apama FX Market Aggregator (another of our roster of Accelerators) to give their FX traders access to prices from a number of FX liquidity venues.  UniCredit is also using the Accelerator to publish FX prices to its eFX downstream channels.   Again, a key point here is that the use of an Accelerator in no ways constrains a client from using the Apama tools that are part of the underlying platform to build out other capabilities that complement the Accelerator.  For some the Accelerator is close to what is needed, but for others it is an attractive launching point.  For further discussion of this, you might want to check out a recent Webinar on “FX Aggregation and Beyond” that talks to this issue.

Lime Brokerage

We added to our broad range of connectivity adapters with  an important connection to Lime Trading System’s Citrius market data feed and FIX order placement services.  This give our customers access to Lime’s trade execution capabilities for equities, derivatives, ETFs, futures and options.  Apama customers can also look to Apama applications via Lime Trading System’s collocation facilities.  Lime is a really cool firm and we see this as a great opportunity for our common customers – both those now and to come.

Connectivity is the lifeblood of CEP and other event-driven applications.  Apama has a broad range of adapters, and we have an engineering team whose specific focus is on this aspect of the product platform.  Integration adapters aren’t “sexy” and they don’t get a lot of attention in marketing literature, but they are vital and deserve a bit of spotlight.

BondDesk

BondDesk provides 2,000 broker-dealers with access to 35,000 live and executable offerings from 120 premier fixed income dealers.  In an announcement this week, we announced that BondDesk ATS (alternative trading system) customers will be able to register their interest in the availability of fixed income securities that meet certain criteria and Apama will monitor the inbound data and provide real-time notification when a matching offering becomes available on the ATS.

So four announcements this week, and more upcoming.

So stay tuned. 

.

We're going on Twitter

Louis Lovas and myself, Giles Nelson, have started using Twitter to comment and respond to exciting things happening in the world of CEP (and perhaps beyond occasionally!).

The intent is to complement this blog. We'll be using Twitter to, perhaps, more impulsively report our thinking. We see Twitter as another good way to communicate thoughts and ideas.

We would be delighted if you chose to follow our "twitterings" (to use the lingo), and we'll be happy to follow you too.

Click here to follow Louis and here to follow Giles (you'll need to signup for a Twitter account).

Adaptive CEP - A Prerequisite for CEP Success

Further affirmation that low latency execution is not the only motivator for decisions to deploy the Apama CEP platform is found in a recent quote from Yann L'Huillier, the CTO of Turquoise.  Turquoise is the London-based alternative trading venue that is using Apama for real-time market surveillance in a project that was managed by Progress Apama partner, Detica. 

In a quite interesting interview in InformationAge that speaks to the role of technology in jumpstarting the Turquoise effort to challenge the incumbents, L'Huillier explains "With regard to Apama and Detica, our thinking is that using a streaming engine with complex-event processing gives us a competitive edge, because we can change the way we read the market and adapt to the always-changing market conditions."

That has proven to be a common - and key - reason why organizations chose the Apama CEP platform.  CEP systems generally respond to conditions that they don't control, as the events are generated by external sources and the CEP system is monitoring what is happening via the events.  A successful CEP system must be adaptive, as the conditions that spawn the events are assuredly going to change.

The word "agility" has become hackneyed in the technology lexicon, but that really does describe the requirement.  As the Turquoise article suggests, it can be fundamental to competitive differentiation.  As a colleague recently noted to me, accuracy is as important as speed because without accuracy, technology merely accelerates the speed with which you get the wrong answer.  With the world changing as fast as it is, accuracy will be quite fleeting unless you can adapt quickly.