Business Making Progress

Progress Apama and Revenue Management Podcast

Listen to this podcast to learn how organizations are using Progress Apama Business Event Processing for Revenue Management.

Two Progress Customers Honored in Insurance & Technology's Elite 8 for 2009

Progress Software is pleased to report two of our customers were named to Insurance and Technology’s Elite 8 for 2009. To paraphrase the editor-in-chief of Insurance and Technology, this annual award identifies outstanding insurance technology leaders, who are managing IT in the new highly scrutinized world of financial services, with enthusiasm and determination.

In this article and video Keith Sievers from Unitrin describes how they improved their operational responsiveness by implementing an ACORD-Standards based architecture “for reduced costs, better quality, faster time to market and greater reliability and performance." Unitrin is using Progress Sonic ESB with Progress DataXtend SI to “federate and share technology across the three Unitrin companies through shared services.”  One interesting fact is that Keith Sievers and his team recognized early on that including ACORD standards in their “enterprise philosophy” would contribute to their success by reducing risk in their implementation.

Unitrin’s successful application of ACORD standards is interesting in light of fellow honoree John Kellington from ACORD. John is another Progress customer, and another of Insurance and Technology’s Elite 8 (article). John was honored for his leadership in advancing the ACORD Framework and Enterprise Architecture. It’s ACORD’s belief (and ours too) that an enterprise architecture which will enable insurance IT organizations to simplify their system portfolios, reducing maintenance burdens and improving development capabilities. John points out that reducing IT ecosystem complexity (often represented by those infamous “Spaghetti charts” or “New York City sewer maps”) eases that “resource strain that seems never-ending.”

Progress is proud to be playing a role in ACORD’s efforts to develop these critical assets for every insurance IT organization. Progress was selected to support the ACORD Standards Framework with unambiguous maps between the new ACORD Information Model and its existing Property/Casualty/Surety, Life/Health/Annuities, and Reinsurance/Large Commercial XML standards. And we have worked closely with ACORD to release the DataXtend Browsers for ACORD with the aim of enabling the ACORD-membership to become even more familiar the XML standards.

You can always learn more by emailing us at sales@progress.com. But, if you have the time, you should also consider attending the ACORD Implementation Forum in Fort Lauderdale, FL Nov 3-6, 2009. it’s a great opportunity to learn and network. This year AIF promises to be an exciting event this year with many interesting programs including:

• Frank Neugabauer (ACORD) and Boris Bulanov (Progress) introducing the ACORD Framework and semantic mapping from 4:00pm-5:00pm on 3 Nov
  
  
• Demonstrations of the DataXtend Browser for ACORD at the “You Drive” Sessions from 12:30-2:30pm on 5 Nov
  
  
• Conversations and networking with experts, like Boris Bulanov, at the Expert Café and Lunch from 12:30 – 1:30pm on 4 Nov
  
  
• And finally, the opportunity to receive your own copy of the ACORD Information Model: A Primer just by visiting Booth #3 at AIF.

Business Event Processing Podcast Overview

Listen to this podcast to get an overview of how Progress Apama Business Event Processing helps organizations to monitor, analyze and act on information in real-time.

The great cloud crash of 2009

I've been following the recent story of how Microsoft's Danger division and the T-Mobile Sidekick.  If you don't know much about Danger, it's basically a cloud service tied to a hardware device (the Sidekick).  All the data is stored in the cloud (email, contacts, etc.) and cached on the device so that when the device resets, it starts up empty and reloads its data from the cloud.  Unfortunately, many customers found out recently that when they reset their device, all their data was gone.  Poof.  No more contacts, no more calendar entries, no more emails.  It turned out they didn't have any real backups and their data redundancy was foiled by human error.  Whoops.  While they have recovered most of people's data now, there are some important lessons to be learned from it.

One of the articles I read on this was titled Don't Blame Cloud Computing for the T-Mobile Mess.  While the author's heart is in the right place (he really likes cloud computing), I can't agree: Cloud computing is absolutely to blame for this fiasco.

Let me explain by starting with an analogy.  Recently we had a little event that some people call the "great market crash of 2008".  The root cause of this is pretty well known now:  A lack of transparency into different financial instruments (e.g. CDOs) made it impossible to accurately assess risk.  And, in the absence of an accurate risk assessment, people assume things will be ok and focus on their short term gain.

It seems to me that we have the exact same situation with cloud computing.  There's essentially no transparency into any cloud provider's integration infrastructure, processes, or planning.   As a result no user can accurately assess the risk of using one cloud provider over another.  Do you think that if sidekick users had know "Danger doesn't do backups" they would have trusted the service with their data?  Of course not, most users assumed everything was OK.  They assume their cloud provider is doing the right thing.  It was only a matter of time before a crash would happen (and this won't be the last one).  In the immortal words of Otter from Animal House, "You f*cked up - you trusted us".

Cloud providers, unfortunately, think that it's not in their best interest to be transparent because, frankly, customers are conditioned to just assume everything is OK so why rock the boat.  When was the last time you walked into a grocery store to buy apples and said, "Can you cut this one open so I can see whether it's OK on the inside?"  No, you probably look at the shiny skin of the apple and assume everything is OK with the inside.

Before cloud computing can become mainstream, users of cloud services must have the ability to accurately assess risk for themselves.  In order to do this, cloud providers must provide transparency. if not users must demand it by speaking with their wallets.  Don't let the cool, shiny UI fool you into assuming everything is OK under the skin.

Progress Apama Announcing Latest Release 4.2

As a follow up to the Louie Lovas blog posting on October 16^th , this  podcast captures a discussion between David Olson and Giles Nelson on Apama 4.2 features.

Apama 4.2 release - Cruising in the fast lane

The Apama engineering team has done it once again. True to our record of releasing significant new features in the Apama product every 6 months, the v4.2 release is hot off the presses with major new functionality. The Apama roadmap is driven by a keen sense of our customer requirements, the competitive landscape and an opportunistic zeal. The engineering team is a dedicated R&D team driven to excellence and quality. We are dedicated to delivering value to our customers. A consistent comment we've heard from analysts and customers alike is the maturity of the Apama product.  

The current v4.2 release, the third in the v4.x family adds significant enhancements in three concurrent themes - Performance, Productivity and Integration. This consistent thematic model is one we've held for a number of years. Below I've touched upon the highlights of the current release along these themes:

• Performance

High Performance Parallelism for Developers.  The Apama Event Processing Language (EPL) provides a set of features uniquely suited to build scalable event-driven applications.  The language natively offers capabilities for event handling, correlating event streams, pattern matching and defining temporal logic, etc. Equally important, the language provides a flexible means to process events in parallel.  For this we provide a context model and a new high performance scheduler. Contexts can be thought of as silos of execution, where CEP applications run in parallel. The scheduler's role is to manage the runtime execution in an intelligent high-performance way, and to leverage the underlying operating system threading model. It’s via the context architecture that the Apama Correlator squeezes the most out of operating system threads to achieve maximum use of multi-core processors for massive vertical scalability. For IT developers, this is a effective and efficient means to build high performance, low latency CEP applications without the pitfalls of thread-based programming, such as deadlocks and race conditions.

High Performance Parallelism for Business Analysts.  Not to be left out of the race, we've also ensured the scalable parallelism provided in the Apama CEP engine is available through our graphical modeling tool, the Event Modeler. We've had this graphical modeling capability since the very first release of Apama. This tool designed for analysts, quantitative researchers and of course developers, allows you to design and build complete CEP applications is a graphical model.  Parallelism is as easy as an automatic transmission, simply select P for parallel.

• Productivity

Real men do use Debuggers (and Profilers too). The Apama Studio now sports major new functionality for development, a source level debugger and a production profiler. Building applications for an event-driven world presents new programming challenges. Having state-of-the-art development tools for this paradigm is a mandate. The Apama EPL is the right language for building event-driven applications - now we have a source-level debugger designed for this event paradigm. Available in the Eclipse-based Apama Studio it provides breakpoints to suspend applications at specific points, examine contents of program variables and single stepping. It works in concert with our parallelism as well. Profiling is a means to examine deployed Apama applications to identify possible bottlenecks in CPU usage.

Jamming with Java. We've enhanced our support for Java for building CEP applications. The Apama Studio includes a complete set of wizards for creating monitors, listeners, and events to improve the development process when building java-based CEP applications in Apama.

• Integration

The (relational) world plays the event game. While we have provided connectivity to relational databases for many years we've made a significant re-design in the architecture of how we do it with the new Apama Database Connector (ADBC). The ADBC provides a universal interface to any database and includes standard connectors to ODBC and JDBC.  Through the ADBC, Apama applications can store and retrieve data in standard database formats using general database queries, effectively turning these relational engines into timeseries databases. The data can be used for application enrichment and playback purposes. To manage playback the Apama Studio includes a new Data Player that enables back-testing and event playback from a range of data sources via the ADBC. One can replay at varying speeds event data and time itself. The tested CEP applications behaves temporally consistent even as data is replayed at lightening speed.

Cruising at memory speed with MemoryStore. The MemoryStore is a massively scalable in-memory caching facility with in-built navigation,  persistence and visualization functionality.  This allows CEP applications, which typically scan, correlate and discard data very quickly to retain selected portions in memory for later access at extreme speed. This could be for managing a financial Order Book, Payments or other data elements that the application needs to be able to access at user’s requests quickly. Furthermore, if required the in-memory image can be persisted to a relational database for recovery or other retrieval purposes, and lastly the MemoryStore allows selected portions of the in-memory cache to be automatically mapped to dashboards.

Well that's the highlights. There were also about a dozen other features within each of these three themes, just too numerous to mention.

We are committed to improving the Apama product by listening to our many customers, paying close attention to the ever-changing competitive landscape and researching new opportunities.

Again thanks for reading, you can also follow me at twitter, here.
Louie

Smart Integration Infrastructure for Insurance Industry

Those of us who have been entrenched in middleware and SOA infrastructure technologies over the years know the importance of having a smart integration infrastructure. Well, today we released a press release announcing that West Bend Mutual Insurance, a property and casualty insurance carrier, chose Progress Sonic ESB and Progress Actional as core applications for their service-oriented architecture. West Bend Mutual Insurance will use Sonic ESB and Actional to supplement its existing policy administration system, so its insurance agents can conduct business more easily and effectively via a single integrated portal. The insurance portal will also be a critical tool to help them improve their customer retention and acquisition. The best part is that West Bend Mutual will finally be able to enjoy operational responsiveness by being able to respond to changing conditions and react more quickly to business opportunities.

For industries like insurance that need to constantly offer new products and services to remain competitive, creating an infrastructure that is agile and scalable, and one that delivers end-to-end visibility of back-end systems, is essential. SOA is a great fit. Read the complete release.

Recap: Gartner EA Summit in Orlando

This guest post comes courtesy of Larry Fulton. Larry is an independent consultant who spent 14 years as a solutions and enterprise architect at UPS and 3 years consulting on, among other things, strategic integration infrastructure issues and enterprise service bus (ESB) technology as a senior analyst at Forrester  research.

The enterprise architecture community is always surprising to me for its enthusiastic optimism - remember that this is a field where the majority of us spend a lot of our time explaining what we do and why it is valuable, often to our own management. Attendees were very clearly engaged in the session topics, there were plenty of insightful questions, and the attendees I spoke with personally saw a lot of value in the material presented. The focus was on the practice of EA rather than specific technical aspects of modern enterprise architecture, which of course begs the question what EAs are doing to stay on top of the technology landscape.

Gartner sees the influence of EA growing over time, especially in those organizations where EA is successfully involving itself in the business and its processes. Aside from the expected pro-EA and how-to-improve-credibility messages, I heard a number of new and refreshing perspectives on EA and its future:

Gartner's Anne Lapkin confronted the "Is EA an art or a science?" dilemma head-on, and clearly stated that many aspects of EA are in fact an art. She was referring specifically to the real work of fitting and re-fitting EA's mission to the current and evolving needs of real businesses. There may be a lot of well-defined process around the tools of the trade - modeling various aspects of current and future architecture, establishing effective governance processes, and so forth - but it takes real insight based on experience to assess what EA can and should be doing to help the business succeed, and to know when that needs to change as the business itself evolves. This is an area where many experts, have been reluctant to come right out and say, "Look, you need to have the right leaders, and you can't necessarily just pick someone who is skilled in another area and expect to train them to this level of EA perspective". Another way to say this is that skilled solutions architects need to be part of EA's activities, but solutions architects don't necessarily have the perspective to define the EA agenda.

Betsy Burton's sessions illuminated the reality that enterprise architects often must fill the role of counselors - working with disparate teams with different perspectives to find common ground and move forward. On the broader business front, this same theme emerged in her recommendation that enterprise business architectures need to include a model of how people actually work together in an organization.

She also mentioned that EAs need to spend at least five per cent of their time playing so they can remain aware of current technologies. My own opinion is that this is not enough - unless you can commit at least half a day each week to some kind of research, which is to say at least ten per cent of your time, it is very difficult to stay on top of important developments.

I particularly enjoyed Bruce Robertson's session on "architecting for emergence". He talked about "EA light", or ways that EA groups should focus on what matters the most and promote application team innovation elsewhere. The idea of establishing policy and technology "guard rails" that essentially say you can do what you like as long as you conform to these particular things and as long as you don't do these other things is not new. But, looking at an organization in a methodical way to identify exactly those rules that really matter and promoting choice elsewhere is where many EA groups need to be headed. Certainly his advice to EA groups to understand local influences and priorities and their relationship to enterprise influences and priorities is a good idea for any EA group, and especially those operating in large organizations.

One of the keynote speakers, Mark Rashino, represented in my opinion the central message to EAs - IT needs to ask itself, "What new strategic capabilities can I offer?" When CIO's are asking themselves that question but don't have an answer, where will they turn? EA needs to be ready to answer that question, whenever it is asked.

Business Transaction Management with SAP

If you follow Actional, you may have seen that Progress Software announced Actional 8.1 today.  The most interesting part of the release is our new support for SAP.  Yes, we already supported SAP NetWeaver (like most of the other people in our space) so unless you're an SAP aficionado, you probably won't recognize the importance of natively supporting SAP ABAP - which is what we've announced.

SAP supports two main application server environments: one based on Java and one based on ABAP, SAP's own programming language (you might also hear the term "basis" which is another name for the ABAP application server stack).  OK, with me so far?  While SAP support both, almost all of the SAP packaged applications are written using the ABAP stack - Java is primarily used for infrastructure services (things like their portal).

With this new version, we've added the ability to trace business transactions into and through ABAP - so we can detect problems even within the ABAP portion of a business transaction.  This is critical for many SAP customers because - without the ability to do this - SAP packaged application logic is seen as a black box silo.  And, with 100's of millions of lines of code in the SAP packaged applications, that a pretty big area to have a blind spot.

While there are a lot of business transaction management vendors out there that support SAP, they are usually referring to the Java side of SAP. As a result, once a transaction hits ABAP (and almost all of them do) it enters the black box and can't be seen again until it leaves ABAP.

As you might have guessed, we're really excited to extend Actional's patented transaction tracing to SAP's core platform.  If you're an SAP user, and need to ensure the success of business transactions that span SAP and other applications, platforms, and middleware then hopefully you'll get a chance to see whether this unique Actional capability can help you.

Developing Event Processing Applications

Listen to this podcast to hear Chris Martins and Giles Nelson discuss development of event processing applications.

If You Build It They Will Come, An Apama Algo Webinar

My colleague Dan Hubscher and I just finished the first of a two part Webinar entitled "Build Quickly, Run Fast". In this Webinar we explained and demonstrated Apama as an Algo platform for high frequency and order execution algorithms.

As I've blogged in the recent past it is an arms race in High Frequency trading.  The need to build quickly is a demanding requirement to keep ahead in the race. Being armed with the right tools is paramount. Rapid development and customization of strategies using graphical modeling tools provides the leverage necessary to keep pace with fast moving markets.

To that point, in this webinar I demonstrated a couple of algo examples. The first was a complete strategy that incorporates an alpha element with multiple order execution options. In  designing and building strategies the trading signal detection is just the first part of the problem. This typically involves an analytic calculation over the incoming market data within some segment or window of time. For example a moving average calculation smooths out the peaks and valleys or the volatility of an instrument's price. Once the signal is detected it's time to trade and manage the order's executions. This is a key distinction between other CEP products and the Apama platform for building trading strategies. While it's possible to define an Event Flow in most or all CEP products for data enrichment and data analysis (i.e. the signal detection), for most other CEP products you have to switch out to some other environment & language to build the rules to manage the executions. The Apama platform is about building complete event-driven applications. So trade signal detection and order executions, whether it's a simple iceberg execution or something much more complex it can easily be designed, built and backtested in the same Apama graphical modeling environment (Of course for those more inclined to traditional development tools and methodologies, Apama offers a full suite of developer tools, an EPL, debugger, profiler and java support).




The second example in the Webinar demonstration was to build a small, but working strategy from scratch. I did this live in full view of the attendees. For this I did a basic price momentum strategy. This tracked the velocity of price movements. The trading signal was a parameterized threshold which indicated when that price moved up (or down) a specific amount for a specific duration.

This webinar is focused on highlighting the ever-present challenges investment firms face in high frequency trading:

• Fears of the Black Box
• The simple fact that markets are continually evolving
• First Mover Advantage
• Customization is king

Along with my colleague Dan Hubscher,  the Build Quickly webinar describes how the Apama platform delivers solutions to the Capital Markets industry to meeting these needs and challenges. 

Stay tuned for a link to the recording and don't forget to dial in to part II where we focus on performance requirements and characteristics. Again thanks for reading (plus watching the webinar), you can also follow me at twitter, here.

A follow up note, here's the link to the recordings for both part I and part II on Build Quickly Run Fast.

Louie

Business Events and Operational Responsiveness - our research

Yesterday, we published a press release on some research that we commissioned from a independent research firm. I wanted to give a bit more background to the research and how we intend to use it.

Our intent in doing this research was twofold:

(a) To discover something new about the markets that Progress operate in and validate some of our own beliefs about the market (or dispell them).

(b) To gather some interesting and relevant information to act as talking points around the things we think are important for our customers and prospective customers, as well, of course, as being commercially relevant to us.

We commissioned the research company Vanson Bourne to do this research and whilst we worked with them on the scoping of it, it was left entirely to them to execute on that scope.

We wanted to hear from end-users so a range of questions were posed to 400 organisations in Europe and the US in three industries - telecommunications, energy generation and logistics. No vendors, analysts or systems integrators were approached.

The questions were all around the theme of "operational responsiveness" - how good are firms at monitoring their operations, identifying issues with process execution, interacting with their customers, extracting and integrating information etc. In particular how good are firms at dealing with the business events which are flowing around, both internally and externally, and how good are they at acting on them in a timely fashion?

Why did we pick these three verticals? Firstly, we couldn't cover everybody and we wanted to go to more companies in a few verticals rather than go very broad. Secondly, we believe that these three verticals are the most interesting when it comes to the demands being placed upon them to cope with business events (Financial services is another obvious one but we know quite a lot about the demands in that industry already). Telecommunications firms are very dependent upon IT to differentiate their services; logistics companies are using more and more technology to track goods, trucks, ships etc. to streamline and automate their operations; energy producers are having to rapidly plan for the introduction of smart metering.

We're still digesting the results. But a few are worth highlighting here. Social networking is creating a significant challenge for all organisations in dealing with customer feedback - consumers expect instant feedback to their interactions. Organisations aspire to more dynamic and real-time pricing of goods and services to increase their competitiveness and maintain margins. And companies struggle with achieving a holistic view of how their processes are operating, both to operationally identify issues before they become expensive to fix or affect customer services, and to identify ways in which processes can be shortened.

We'll be talking more about the research results soon, both qualitatively and quantitatively.

What happens when poor app integration decisions are made? Webinar presents best practices on how to be successful: http://ping.fm/layXZ

Progress Apama Capital Markets in Brazil

In recent years, the Brazilian market has grown stronger, and become very aggressive with algorithmic trading. Just back from a conference in Brazil, listen to this podcast where Dan Hubscher shares insight into the current state of Brazil’s market, and what the people down there are buzzing about.

High Frequency Trading on Jon Stewart's Daily Show

Well, you know that High Frequency Trading has gone mainstream when it makes Jon Stewart's The Daily Show.  Check out this segment with Samantha Bee.  We don't see many folks in cow costumes at our trade shows, but maybe that is a sign that HFT is going mainstream? 

RE: The wrong marketing for open source

Matt Asay makes a good point in his article on open source marketing, that people generally choose to use open source software because it is cost effective, not because it is “flexible”.

Put another way, “free as in beer” trumps “free as in speech” as a motivator for people making the choice to use open source. I think that is obvious. On the other hand, I think that it is so obvious that the premise of the article that open source should be marketed on the basis of low cost over freedom of choice is wrong. I think the market widely assumes that open source is “free as in beer” so you get the benefit of that perception just by being open source in the first place.

That said, he does refer to the importance of “low cost utility.” It isn’t good enough that Linux or MySQL or ServiceMix ESB is open source, (i.e. perceived to be “free as in beer”). It also has to be good enough to be useful in helping someone solve a real problem. In other words, the beer has to be drinkable or doesn’t matter that it’s free.

We, Progress Software, are also seeing companies begin to recognize that adopting open source isn’t entirely free (as in beer). They will be more successful getting the utility from the software if they invest in developing the skills to use the software properly and have the backing of a company to support them in that effort. To take the beer analogy a bit further, (perhaps too far?), renting a mug is better than drinking the beer out of your hands.

From an open source marketing perspective, what we need to do is let people know that the beer tastes good and explain how much better the experience will be drinking it from a mug. We don’t need to spend a lot of time explaining why it’s a good thing that the beer is free.