View from the Edge

Cool Stuff: OpenEdge 11 Multi-tenancy

One of the most important and useful new capabilities in the upcoming OpenEdge 11 release (planned for December 2011) is direct support for multi-tenancy in the database. What is multi-tenancy and why would I want one? Read on and I'll tell you.

The notion of multi-tenancy arises in the field of Software-as-a-Service (aka SaaS). When a vendor offers an application to be used as a "service", its customers do not have to buy a computer system to run the application on, nor do they have to have staff trained in the care and feeding of the system or backing up the data. Instead, customers subscribe to the service and the vendor does all of that. The customer simply uses the application over the Internet and has no idea where it is or what computer it is in. When you use a search engine, you don't know where it is and it doesn't matter. It's the same with SaaS applications. A SaaS application you are probably already familiar with is email. Google, Microsoft, Yahoo, and most ISP's run email servers for their subscribers. You don't have to know anything except where to log in. Someone else is responsible for everything to do with operating and maintaining the service. You just receive, read, compose and send your mail.

All this is very easy, even trivial, for the Software-as-a-Service customer. But what about the poor vendor? The vendor has to do all that messy IT stuff. Is he going to have a dedicated computer for each of the customers, as they would if they were running the application on their own computer? No, of course not. The vendor is going do everything possible to hold their operating costs as low as they can. That is where multi-tenancy comes in. Each of the SaaS vendor's customers is called a "tenant", a word taken from the rental housing market. In an apartment block we can have many tenants in the same building, all living in separate spaces. Similarly, we can put many application tenants into the same computer.

With a number of tenants sharing the same computer, the SaaS vendor has fewer machines to buy, fewer machines to take care of, and less work to do. No tenant can see the other tenants or their data and in fact do not know anything about them or their existence. Quite a number of the SaaS vendors do this. Since it is very simple to with OpenEdge, many of them have created a separate database for each tenant. But that means you have to do backups, schema changes, and other maintenance functions individually for each tenant's database. It would be much better if tenants could share the database too. We call this database multi-tenancy.

In OpenEdge 10 and before, with quite a bit of work, you can achieve database multi-tenancy. Some of our partners have rolled up their sleeves and done it. What you need to do is this:

• First, add a "tenant identifier" column to every table. This tenant id column is a column that contains a unique identification number, perhaps an integer, assigned to each tenant. The value indicates which tenant owns the data in each row of the table.
• Next, add the tenant id column to every index as the leading key component.
• Create a table to store the tenant names and their tenant id's and assign an id to each tenant.
• Then, go through all the code in your application and everywhere that a new table row is created, assign the correct value to the row's tenant id column.
• You also have to invent a way to keep track of which tenant id is currently in effect.
• Finally, go through all the code in the application again and find all the queries. Modify each WHERE clause to add a term that says "(tenantId = currentTenant) and ". Don't forget CAN-FIND. And make sure to add the tenant id term for each table in a multi-table query.

Once you do all those things, you can have database multi-tenancy. But in addition from the obvious fact that taking this approach is labor-intensive and invasive, there are a number of other disadvantages. I will list just a few here:

0) It is error prone. If you make a mistake when you change the code to do multi-tenancy, the wrong tenant's data will be returned. Or if you forget when you or another developer is fixing a bug, the wrong tenant's data will be returned.

1) Even if you use Type ii data areas, rows from multiple tenants will be commingled in the same data blocks and the same table's allocation clusters. This negates many of the advantages of using Type ii data areas. You get lower I/O efficiency because one tenant will have to read a data block that contains other tenants' data. Your customers will probably have the perception (whether true or not) that commingling their data reduces its security.

2) You can't do per-tenant maintenance easily. How do you reindex just one tenant's data?

3) How do you restore one tenant's data when they do something foolish like run end of month processing in the middle of the month?

4) You can't do per-tenant disk space allocation or disk space usage tracking very easily, if at all.

5) There is lock interference among tenants. Table-locks can lock out all the other tenants.

In spite of the disadvantages, I think the advantages far outweigh them and it is worth considering the use of this approach. But what if you could eliminate all the disadvantages? What if you could have your cake and eat it too? That's where OpenEdge 11 comes in. All that work I said you have to do? Gone. All those disadvantages I listed? All gone. OpenEdge 11 does all the hard work.

With the OpenEdge 11 RDBMS, database multi-tenancy is an inbuilt feature. The database knows what tenants are, who they are, and where their data are. It knows where to put new data and where to get existing data for each and every tenant. You do not have to modify all of the data access parts of your application. In fact, you shouldn't have to change much of anything! Most of your code should just work.

Well, all right, maybe you do have to make a few changes. Those changes have to do with how a user logs in to the application and the database and how the user's identify is verified. As I said, the database knows about tenants. But you will have to tell it which tenant a user belongs to. In the 4GL we use something called the CLIENT-PRINCIPAL to help in detraining that.

The CLIENT-PRINCIPAL (aka the "cp") is an inbuilt and extensible security token that we added to OpenEdge a few years ago, in the 10.1 release. The cp encapsulates a user's identity once it has been validated. In OpenEdge 11 we use the cp (with some enhancements) to encapsulate both user identity and tenant identity. Depending on which cp token is currently in effect in the 4GL runtime, the database uses the tenant id to decide what data to return for a query. For code running in AppServers and accessing the database on behalf of different users at different times, the AppServer can easily switch the cp that is in effect to that of the user that made the AppServer call.

To get ready for OpenEdge 11, you should learn about the CLIENT-PRINCIPAL. The name may sound a bit intimidating but it is really very easy to use. It takes only 3 lines of code to make one and to validate the user's identity. Go and watch the video of Sarah Marshall's Exchange Online 2010 talk over on PSDN.

In the OpenEdge 11 RDBMS, each tenant gets a separate data partition for each multi-tenant table (and not every table has to be made multi-tenant), and each data partition has its own associated index partitions. The tenant id in the cp is used to control which data partition to fetch table rows from and a tenant only gets to see their own data (and data in regular shared tables). We also have a special tenant called the "super tenant", conceptually similar to the UNIX root user, that is allowed to see /all/ the data.

This scheme works really well, is very efficient, and requires very few application changes. There are of course a lot of other things in OpenEdge 11. But I don't have space to talk about them just now and we will have to do that another time.

I hope you will like the new release. It is really cool.

Mary Szekely: A Progress Original

It’s not very often that an original employee sticks around for 30 years, however, Mary Szekely (pronounce C-K per the Hungarian origins of the name) is that very person at Progress Software. A software engineer and fellow at the company, Mary is one of the initial four employees at Progress and one of the first fearless females to enter the male-dominated field of software development. We recently had an opportunity to talk one-on-one with Mary. If you are interested in reading the complete interview, click here to visit the Business Making Progress blog.

Cloud: Hype or a Business Reality?

As we come to the close of 2010 I think it's interesting to look and see what state this thing called Cloud is in.

Firstly I think it's fair to say that the visibility of Cloud (which seems a bit of a strange thing to say, given it's nebulous nature, but you know what I mean) is higher today than it's ever been.  Who'd have thought that we'd have adverts on mainstream TV at peak viewing times saying 'To the Cloud..', although how plausible those adverts are is another matter entirely, but if nothing else Microsoft has raised the profile of 'the Cloud'.

Similarly the rise of things such as on-demand streaming from Netflix, the exponential growth of Facebook and other cloud based services has driven the concept of utilizing services where I have no idea of the underlying infrastructure, and quite frankly as a consumer why should I care, to new high's.

That's all well and good for consumer type stuff you say, but what about real Business applications, both with a lower and upper case A!  Earlier in the year I attended the CloudExpo conference in Santa Clara.  This was my second year attending the conference and it was amazing to see the growth in scale from one year to the next.  The number of attendee's (6,000), the number of sessions, the number of exhibitors were all way higher than the previous year, and listening to the organizers they're planning for over 10,000 attendee's at their next event in New York in the spring of 2011.  So certainly the interest from the business community is there.

But interest is one thing, what about actual usage?  The latest research from Saugatuck, http://goo.gl/pmXQD, states "by the end of 2014, an average of 50 percent of new software deployments worldwide will be made in the Cloud, marking a tipping point in business solution acquisition, implementation and deployment that will change the fundamental nature and roles of IT, as well as ISVs."  50%, that's a huge number, and so is the comment about fundamental change!

Ok, but what about the concerns of moving to the cloud, especially the #1 concern, security?  At the CloudExpo conference I had a chance to talk to Steve Riley, an evangelist at Amazon who specializes in security.  He has an interesting perspective about Cloud security, which to cut a long story short, boils down to the fact that the Cloud is in many ways more secure that your tradtional server room.  Just think how many of your employee's or collegues can simply walk up to your servers today and potentially switch them off?  Amazon won't even tell you where their data centers are located, let alone what machine your instance happens to be running on!  It was also interesting to see that for a brief period, before government pressure, WikiLeaks moved onto Amazon to avoid DDoS attacks as Amazon is architected to handle any such attack.

The public cloud vendors such as Amazon have made huge strides in the past few months in the area of security.  Recent press releases from Amazon tell of their ISO 27001 accreditation and SaaS70-Type II audits.  But one of the big concerns, especially from many of our partners has been PCI.  Any application that stores credit card info always raised a red card when it came to Cloud.  Well, just in this past week Amazon annouced that they've achieve PCI DSS Level 1 accreditation. http://goo.gl/4JwDH  That large thud you hear, is just annother barrier to Cloud adoption falling.

I think it's safe to say that in 2011 this phenomenon that is Cloud will continue to grow, in all aspects, the sheer number of vendors, it's adoption and relevance to the CxO level and below in businesses ranging from large corporations to start-ups (what other technology allows a start up to compete with the big guys when it comes to infrastructure).  And OpenEdge will be there every step of the way, as we move forwards with Progress Arcade, OpenEdge 11 & OpenEdge BPM, you can bet we're working flat out to make sure that you are best positioned to take advantage of what can only be seen as a game changer.  

Welcome to the new business reality!

 

OpenEdge, the open Force?

Use the "Force" Luke...Ok, now that's out of the way, because admit it, that's the first thing that entered your head when you saw the title. That Force is not the Force I'm looking for....OK, enough!

No, The Force I'm referring to is force.com, the platform provided by the folks over at Salesforce as a means of building Force applications. Let's be honest here, if you ask most people to name a major player in the SaaS application space today, most people would say Salesforce.com, and as such their platform upon which it's built, force.com, is held in high regard. But here's where is gets interesting.  I was reading a whitepaper the other day where I came across some figures :

• Overall Project Cost is 30% less
• Requirement definition is 25% less
• Testing Effort is 10% less

These figures come courtesy of Galorath Inc. a research company that conducted a study into developing with force.com versus traditional development.  This got me thinking.

Back in February, Nancy posted a blog entry about the new Forrester report into the Total Economic Impact of the OpenEdge Platform.  When you look at the outcome of this report there are some striking conclusions :

• Developing with OpenEdge is 40% more  productive than alternative platforms
• Application Delivery with OpenEdge is 30% faster to market than when using an alternative platform
• Support staff productivity gain is 80% with OpenEdge as compared to alternative platforms

Interesting, no?  So when you compare OpenEdge to what is in most people's minds the 800lb gorilla in the SaaS world and a thoroughly modern platform, not only do you get a more than favorable comparison, you are actually better positioned.  Why?  Well what's the one big thing Force.com doesn't allow you to do? Deploy outside of Force! Whereas with OpenEdge you have the freedom to deploy pretty much anywhere, be that on premise, a traditional hosting model or indeed in the Cloud.

So good readers, I put it to you that OpenEdge is not only the open equivalent of something such as force.com, but indeed is the leading platform for developing modern SaaS applications where you get to decide how to best deploy it based upon your business model.   

Check out the latest Forrester Consulting study

We wanted to find out more about the economic impact that Progress Partners may realize by building applications with OpenEdge versus using a non-Progress platform. To help us, we enlisted Forrester Consulting to do some research. You can see what they discovered in this new report:  The Total Economic Impact™ of Progress Software OpenEdge Platform. Among the revealing findings:

• Developing with OpenEdge was shown to be 40% more  productive than alternative platforms
• ISVs choosing OpenEdge are able to deliver their  application 30% faster to market than when using an alternative platform
• Once deployed the support staff productivity gain is 80%  as compared to alternative platforms

Get the full report to learn more.

Canada Tech Peaks and PUG Meeting

My journey to Canada began with a flight from Manchester New Hampshire to Toronto on a sunny and mild Tuesday afternoon. I was thinking how convenient it was to fly from my home airport rather than having to take the drive and fight the traffic around Boston. The plot thickened when I was led out onto the tarmac to board a twin-propeller plane that holds somewhere around 20 passengers. I especially enjoyed the baggage handler on the ground, who, once everyone was on the plane, boarded, took off their reflective vest, closed the cabin door, and took their seat as the co-pilot for the flight! With good weather on our side, the flight was smooth and uneventful.   

The turnout for the Tech Peaks meeting on Wednesday morning was excellent- around 25 or so attended. By a show of hands, many turned out to be long-time Progress customers, some for over 20 years. My presentation topic was the OpenEdge Strategy and Roadmap, and among other things, helped the audience understand why Software as a Service is something that will likely have an impact on their business over the next few years, if it hasn't already. The audience was most engaged in dialog on the upcoming Transparent Data Encryption for 10.2B, and the future OE11 multi-tenant table support. I was followed by Frank da Costa who gave an excellent presentation and demonstration of Actional monitoring an OpenEdge environment, a new capability for 10.2B. The audience gained an appreciation for how easy and fast it is, without having to do any coding changes, for Actional to automatically build a map of the transaction flows through the system, the level of detail that is available from the console, and the ease that rules can be established to, in real time, monitor and alert on things like Service Level Agreements.

Almost forgot to mention that the hotel conference room looked out on 2 sides to an indoor shopping mall. I especially enjoyed the people who stopped to stare in the window to see what was going on- maybe new potential cutomers! While this wasn't an actual photo from the event, there was a small boy around 3-4 years old that did spend a minute with us looking in.

My second stop was in Montreal for a Thursday afternoon Progress User Group (PUG) meeting. The attendance here was also good- in the 20's, and the presentation topics were the same. I was taken slightly by surprise when the PUG President opened the meeting speaking in French, with the follow on topic on the Progress corporate overview also delivered in French. It was interesting to try and follow along, but the jokes went right over my head.

I was pleased to see that many in attendance had already made the move to OpenEdge 10.2A, taking advantage of the latest features that we have to offer. With 10.2B coming out soon, and planning underway for a major new OE11 release, anyone who currently finds themselves working with an older version of Progress/OpenEdge should take a look to see what new features they could be using to improve their business, and how they can get up-to-date. Progress Communities is always a great resource for information.

Thanks to all who coordinated and supported these events- it's easy to overlook the work done behind the scenes to make these events both possible and successful.

Are you ready for your next Disaster?

When a disaster strikes are you going to be ready?  If you can’t answer this question with 100% surety then you have a challenge in front of you.  There are many different plans to prepare yourself for a disaster.  One that I have seen used both in the public sector in the United States and the private sector is the National Incident Management System (NIMS) and more specifically the Incident Command System (ICS).  These have been developed over the last 20 years to help agencies in the United States respond to a disaster.  There are similar systems available in almost every country and region around the world.

While both of these programs have been developed to respond to a public disaster – typically a natural disaster – they provide a very good foundation for the complete disaster management cycle.

ICS breaks the process down into 4 discreet components:  Mitigation, Preparedness, Response and Recovery.  On PSDN, Marv Stone and I will do a webinar / podcast on these components and how OpenEdge 10 meets these needs for your application.

Mitigation deals with identifying requirements (business and safety for example) and minimizing or avoiding those risks altogether.  An important part of this process is identifying where you are vulnerable.  Once you have identified this you can start to figure out what you “really” need to worry about.  This moves right into the planning component of your Disaster Recovery plan.

Planning addresses what options are available, choosing the appropriate option to meet your needs, implementing that option and testing the plan.  This is one of the most critical components in the process.  It also takes the most time.  It is an iterative process and will never be done…

The next component is Response.  This is execution of the plan.  The first part of this is declaring the disaster.  This would seem like a simple thing to do, but it isn’t.  I will talk about this in my next blog.  This is where your testing with your solution will pay off.  Whether it’s OpenEdge Replication, Failover Clusters, or After-Imaging, if you haven’t tested thoroughly then this could turn into a nightmare.  A critical part of this process is documentation.  Documenting what is being done, who is doing it, and when it was done is critical to the post-mortem (or Recovery) component of your environment.  If you don’t document and learn from your mistakes, you will inevitably make them again.

The last component is Recovery.  This can be interpreted in several different ways.  In the public sector it revolves around returning to life as usual (or as usual as it can possibly be).  In the private sector it involves failing back to your production environment and getting the complete business back up in business as usual mode.  This is also affected by your planning process.  Failing back should be planned and tested in the Planning component of your plan.  Documentation is also critical in this component in order to clean up and prevent additional outage time.

Marv and I will chat about these and where OpenEdge 10 plays a part in all of these components of your Disaster Recovery plan.  Be sure to listen in!

Until next time, a disaster is only a disaster if you are not ready for it.

Brian B

About That Scalability ...

Does scalability only apply to very large deployments ?

Try this, fold a piece of paper in half by bringing the top edge down to meet the bottom edge. Now imagine an OpenEdge application with a 20GB database that uses either wifi or cabled network to run a small business. Imagine the business system occupies that half-sheet of paper, uses 2 watts of power, weighs two pounds, and costs $400 US.

That's reality today. I bought one of those little Eee laptops recently, and was delighted at how easy the linux system is to use, and how easy it integrates with my Windows-dominated infrastructure. Nice job. Linux even runs my spreadsheets and presentations unchanged (you can even change the laptop to XP if you want). What a nice parallel this "mini-system flexibility" is to OpenEdge! The ability to scale down is partly one of architecture and partly discipline in the implementation.

But why scale down? One reason could be new products or new markets. Another reason is to focus on providing a consistent service at a high performance level. Have you seen applications that have just seemed to grow and grow. Application bloat leads to a loss of control over the key function of the software, and it seems that poor usability and uneven performance come right along for the ride. So maybe scaling down is another way at looking for clear boundaries between layers of an application as well as separation of functions in an application environment. That highly connectable 2 pound workstation was no accident! It was drawn from the OLPC (One Laptop Per Child) design effort which is spawning a lot of creativity in the Ultra-Mobile PC market. A solid software architecture has some similarities in terms of defined interfaces, guiding principles, and scalable design.

Does your application scale up? Or down? Or out? Sometimes a "readiness review" of your architecture from scalability or an service-oriented view can lead to a plan that leads into the business roadmap by opening up new options. We hope that the new capabilities in OpenEdge ABL can help you implement that roadmap in a smooth way that phases in the stages you plan for your evolution. Are you trying this approach? How well is this  working for you? The AutoEdge application gives some nice examples of how the ABL supports the scalable OpenEdge Reference Architecture. I was just reviewing a presentation about AutoEdge that will be done for Exchange in Paris. If you are attending, this session could be very useful. Another interesting session shows a racetrack example which illustrates how a SOA environment comes together with Sonic, OpenEdge, and other fine products. Check them out and let me know what you think.

What about Deployment?

From the deployment side we still support efficient storage of data, so that 20GB database on the tiny laptop might hold a surprising amount of business data. we still support compatibility of the database from the little laptop up through large systems too. With OpenEdge 10.1B, every database can grow to have very large tables, wide indexes, and a full range of datatypes. You even get built-in management features like automatic defragging now. On a small database, it is easy to make it run "hands off". For large databases, you get on-line management of table space, schema additions, and the ability to increase some key performance parameters without taking downtime. OpenEdge needs less tuning than those big databases, and we are still the #1 pure-play database for applications that need a capable database "built right in". Our vision is to let your production environment evolve smoothly with a self-tuning, self-healing, and self-provisioning database that handles the transactional business load. What else? Well, improvements in appservers keep coming, as do improvements in the reporting capabilities of OpenEdge SQL. Similarly, support for Sonic keeps getting easier with 10.1C, and we'll talk about that in a separate note. We do take performance and manageability seriously, and the roadmap for this area really features a lot of customer-requested capabilities. We really appreciate your advice and insights into your needs. Please keep them coming! What are your top 5 needs in the database and appserver area?

Whatever happens in the scalability area, one thing remains. Scalability is not about standing still. It's about a dynamic range and that means opportunity for you and for us. Together.