Nati Shalom's Blog

Moving Enterprise Workloads to the Cloud on a Massive Scale

Cloud has quickly become a disruptive technology, with the main disruption being a revolution in the way in which organizations run their IT.

The poster child for these cloud discussions was often the new Web 2.0 companies such as Google and Amazon who have been true pioneers in this new and emerging world. Quite often enterprises were used as the reference for how bad IT can get, and quickly became the representation of the antithesis for IT in these cloud discussions and conferences.

Two weeks ago I was at the OpenStack conference and other meetups that took place around the same time. The thing that was pretty apparent is that the large majority of the audiences in each room were from the enterprise world. What’s more, a quick look at the OpenStack foundation shows that those leading one some of the most trendy technologies in the cloud space are run by Enterprise HW and Software companies such as HP, IBM, Dell, Red Hat, among others.

Could this mark the beginning of a new era for enterprise and the cloud?

- A series of recent surveys on the subject seems to indicate that this is indeed the case:

Research conducted by HP found that the majority of businesses in the EMEA region are planning to move their mission-critical apps to the cloud. Of the 940 respondents, 80 percent revealed plans to move mission-critical apps at some point over the next two to five years.

A more recent survey, by research firm MeriTalk and sponsored by VMware and EMC (NYSE:EMC), showed that one-third of respondents say they plan to move some mission-critical applications to the cloud in the next year. Within two years, the IT managers said they will move 26 percent of their mission-critical apps to the cloud, and in five years, they expect 44 percent of their mission-critical apps to run in the cloud.

This shouldn't come as a surprise as  most enterprises are using the cloud today but don't necessarily know they are doing so, and getting huge benefits from it (SalesForce.com, ServiceNow)

The Cloud needs Enterprises just as Enterprises need the Cloud?

As I was sharing this observation with a few of the folks during the conference, I heard an interesting comment by one HP Executive Thomas Ryan who made an interesting statement: "Linux wouldn't be where it is today if enterprises wouldn't have adopted it". Quite often we look at the cloud (OpenStack in particular) as the new Linux. If that's true, then it looks like we’re reaching the point where for cloud to remain a viable technology it needs enterprise adoption just as enterprises needs the benefits the cloud brings.

The Renaissance of Enterprises in the Cloud

So are we seeing the renaissance of enterprises in the cloud? It's a bit too early to say, but there are clear indicators that this is where the market is heading. There are a couple of reasons for this:

1. IT is becoming a new manufacturing pipeline for enterprises in the new information era

It's clear today, that in this data driven world, the ability of any business to survive is determined by how agile they can be, how quickly they can respond to change, and react accordingly. In this context, it becomes more apparent for many enterprises that their ability to reach this level of agility is dependent on how well they are running their IT. At the same it becomes clear that cloud is just a better way of running their IT, and in this context it’s pretty much a no brainer.

2. Cloud technology is maturing

Many of the technology gaps that prevented enterprises from adopting cloud technology such as security, complexity, and more, have been addressed or are in the process of being addressed. There are many options today including private cloud, hybrid cloud, with different degrees of SLAs and cost that make it easier to run enterprise workloads in the cloud.

3. Strong business incentive

Enterprise applications tend to be one of the more costly workloads today in terms of IT infrastructure and resources, due to both the size and the complexity involved with running these systems. Any savings with running these systems would yield a significant impact to an organization’s bottom line, as indicated in MeriTalk research:

Enterprises can save $16.6B, or 21% if they moved just three mission-critical apps to the cloud.

The Challenge - How to bring Hundreds of Enterprise Apps to the Cloud

All that, of course, is great, but the reality is that cloud economics only start making sense when there are true workloads that utilize the cloud infrastructure.

If the large majority of your apps fall outside of this category, then you’re not going to benefit much from the cloud. In fact, you’re probably going to lose money, rather than save money. 

The Current Approach

• Focus on building IaaS - Current cloud strategies of many enterprises has been centered on making the infrastructure cloud ready. This basically means ensuring that they are able to spawn machines more easily than they were before. A quick look at many initiatives of this nature shows that there is still only a small portion of enterprises whose applications run on such new systems.
• Build a new PaaS - PaaS has been taught as the answer to run apps on the cloud. The reality however, is that most of the existing PaaS solutions only cater to new apps and quite often the small, and “non” mission-critical share of our enterprise applications, which still leaves the majority of our enterprise workload outside of our cloud infrastructure.
• App Migration as a One Off Project - The other approach for migrating applications to the cloud has been to select a small group of applications, and then migrate these one by one to the cloud. Quite often the thought behind this approach has been that application migration is a one-off project. The reality is that applications are more of a living organism – things fail, are moved, or need to be added and removed over time. Therefore it’s not enough to move apps to the cloud using some sort of virtualization technique, it’s critical that the way they’re run and maintained will also fit the dynamic nature of the cloud.

Why is This not Going to Work?

Simple math shows that if you apply this model to the rest of your apps, it’s probably going to take years of effort to migrate all your apps to the cloud. The cost of doing so is going to be extremely high, not to mention the time to market issue which can be even an even greater risk in the end, as it will reflect on cost of operation, profit margins and even the ability to survive in this an extremely competitive market, if it is too long.

What's missing?

What we’re missing is a simple and systematic way to brings all these hundreds and thousands of apps to the cloud.

Moving Enterprise Workloads to the Cloud at a Massive Scale

Instead of thinking of cloud migration as a one-off thing, we need to think of cloud migration on a massive scale.

Thinking in such terms drives a fairly different approach.

I've listed the principles, that IMO, are the crux to such a strategy, below:

• No Code Change - Any change, even the slightest one, to our existing apps will slow down our migration efforts substantially. Furthermore, many of the apps are third-party apps, where we often times don't even have control over the application code, so changing the app is likely not an option. There are many things that we can benefit from by moving our apps to the cloud, such as automation, ease of change, and configuration. We can obviously benefit more if we change our apps to take better advantage of the cloud infrastructure, but the right order would be to do this after we’ve already migrated the app to the cloud in the first place. There, of course, will still be many cases where just automating the way we run our apps and make them available on-demand would be sufficient enough.
• Plug-in to the Current Way of Running Apps - The common way for running apps today is either through custom shell scripts in the simple case, or through configuration management tools such as Chef or Puppet in the more advanced case. If we want to migrate apps at a massive scale, we need to be able to plug-in to the most common environments, and make it possible to import them almost as is. In this way, we could leverage the knowledge and know-how that has already been invested into these systems and make the migration process vastly shorter.
• Use Shared Infrastructure Across the Stack Regardless of the Language - Using different frameworks for each element of our application stack based on the language and environment in not going to scale for obvious reasons, such as complexity, a large number of moving parts, inconsistency, among other reasons.
• Add Cloud Properties from the Outside - There are many cloud properties that we can apply to our existing apps without forcing any changes. For example, we can automate the scaling from small to bigger machines, we can also automate the configuration, and setup. We can significantly simplify the way we run the app by making it available on-demand. All these cloud properties can be plugged into almost any existing app without forcing a real change, as noted in the MySQL example.
• Use the Baby Steps Approach - One of the main inhibitors to migrating apps was the all or nothing approach. There are many risks involved in migrating apps that go well beyond the technology challenge. The easiest way to mitigate such risks is by taking them one at a time. For example, we can first make our apps cloud ready, in which case we will abstract the automation part from the target infrastructure. By doing so, the decision where to run our app becomes tactical and less strategic. We can choose to run our production in our private data center, and run only testing and demos on the cloud. In this way, we can also make the migration gradual and smoother.
• Incorporate Bare-Metal into our Cloud - One of the main challenges that I’ve seen for moving true enterprise workloads to the cloud wasn't necessarily related to the cloud, but to the virtualization overhead that is often used as the underlying infrastructure of many clouds. Today it is possible, to gain access to bare metal clouds  which make it possible to run almost any workload including I/O intensive ones without any performance overhead, as I noted in my post about the emergence of bare-metal clouds.

Final Notes

I often like to compare the revolution that we’re experiencing right now in the IT industry to a similar revolution that happened in the automotive industry few decades ago.

You probably know what I’m referring to. The automotive industry experienced a similar revolution when Henry Ford came up with his Model-T car. The change wasn't so much in the engine or other parts of the car, but rather in the scale and cost in which cars could now be produced. As I noted earlier, in many aspects cloud is the IT industry manufacturing assembly line.

As with cars, it doesn't make sense to run only 10% of our cars on the new manufacturing assembly line, and build the rest in the traditional way. So is the case with our IT. Now the question becomes how to make the move.

In this post I outlined what I believe should be the principles to migrate apps at a massive scale.The second post describes more specifically the Cloudify way of doing this.

PaaS as Infrastructure

The blurring line between PaaS and IaaS 

The core of IaaS services is centered around  Compute, Storage Network however over the past years the infrastructure moved up the stack and includes more application services such as Database as a Service, Hadoop Map/Reduce, and application lifecycle services. 

On the other hand, PaaS services offer similar services through the PaaS platform itself with the main difference being that quite often those services were designed to fit only the application that runs on that particular PaaS.

So rather than the traditional view, where we draw a clear line between the IaaS and PaaS layer we need to view them differently, where the PaaS and IaaS layers overlap each other on the software services layer.

 

IaaS and PaaS share a similar technology stack

A close look into the OpenStack infrastructure shows that even on the infrastructure level running the compute, storage and network services relies on messaging services such as RabbitMQ, databases such as Postgress, a load balancer such as Nginx and other software services. Running these software services for our infrastructure is not much different than running our apps.  With both the infrastructure and application we would need to address the scalability and high availabilityof these services and manage them through our infrastructure.

If that is the case wouldn't it make sense to share the same underlying technology to run our infrastructure and our apps?

Sharing the software services between your IaaS and PaaS

A typical public PaaS platform is built out of two main parts - a self-service portal, and a backend system that runs our apps.

With many of the public PaaS solutions such as GAE, Heroku the backend that runs our PaaS is treated as a blackbox.

Amazon Elastic Beanstalk does things differently in that respect - as it took a bottom-up approach. It uses the same AWS infrastructure to run Elastic Beanstalk and the infrastructure - this provides a lot of flexibility to its users, and equally as important, consistency between the way they run and manage their app and infrastructure. VMware took even a bigger step in this regard with CloudFoundry and BOSH and made their entire PaaS backend open source which gives users even greater flexibility. With Cloudify we took this idea a step further by introducing the concept of recipes into the core of the Cloudify infrastructure, and recently introduced our integration with Chef in a way that will enable users to run their Chef cookbooks as part of Cloudify. Chef has already gained popularity as a configuration and automation framework for setting up many of the OpenStack cloud deployments. Having Chef tightly integrated into the PaaS layer, simply put, makes perfect sense.

Example - Building your own RDS with Cloudify

To illustrate the idea of how we can use share the same deployment framework between our IaaS and PaaS layer I'll pick one of these software services to demonstrate the idea with. In this particular case, I've chosen to use a database as a service. Database as a service is often a good reference for a service that fits in both our infrastructure and application. Here is how we can easily set a database as a service with Cloudify:

1. Directly through the use of Cloudify recipes

If you allready developed your scripts for setting your database you can use cloudify recipe to wrap those script into a service. 

 Here a snippet of how that would look like:

service {
	name "mysql"
	icon "mysql.png"
	type "DATABASE"
	elastic true
	numInstances 1
	minAllowedInstances 1
	maxAllowedInstances 3	
	compute {
		template "SMALL_LINUX"
	}	
	lifecycle{
 
		install "mysql_install.groovy"
		start "mysql_start.groovy"
		startDetectionTimeoutSecs 900
		startDetection "mysql_startDetection.groovy"
		stopDetection {	
			!ServiceUtils.isPortOccupied(jdbcPort)
		}
        }	

You can refer to the full MySQL recipe here.

2. Through the use of Chef cookbooks

If you are already using Chef you can plug-in to the Chef MySQL cookbook by extending the Cloudify recipe to use Chef, as you can see in the snippet below:

service {
    extend "../../../services/chef"
    name "mysql-vanilla"
    type "DATABASE"
    numInstances 1
…
}

In addition to that you could attach Chef attributes and run parameters into the Cloudify recipe that will tell Chef which cookbook and recipe to run:

runParams = [
  "mysql": [ "bind_address": "0.0.0.0",
             "tunable": ["query_cache_limit": "2M"]  
  ],
  "run_list": ["recipe[mysql::server]"]]

You can read the full details on how to run Chef with Cloudify in our documentation.

Running through bare metal environment

Even in the case where we share the same software services through our infrastructure and application, we tend to run them differently.

Quite often infrastructure services will run on bare-metal environments, as we don't want to create dependencies between our virtualiation layer and the underlying infrastructure that runs it.  

To enable this level of flexibility, we need to abstract the underlying compute layer from our service orchestration layer. In this way, we can use the same recipe and deploy it on either a bare-metal environment or a virtaulization layer. (Read: Configuring a Traditional Data Center [BYON])

Final notes

Up until recently, PaaS and IaaS were treated as two separate and closed entities and the integration between the two was done pretty much in a blackbox approach. The fact that both the IaaS and PaaS underlying infrastructure were closed was the primary reason behind this.

Now with the growing adoption of open source IaaS such as OpenStack and CloudStack, and similarly with PaaS such CloudFoundry and Cloudify we have the opportunity to change this paradigm. There isn't much of a reason why we shouldn't be able to share the same orchestration, and configuration layers between our IaaS and PaaS layers. Amazon Elastic Beanstalk is a good reference why this makes a lot of sense.

Special thanks

Special thanks to Boris Renski who was the main inspiration behind this post.

References:

• Presentation on the Chef Meetup Group
• Online presentation