Three years ago when we started working on the first generation of our PaaS offering, cloud portability seemed to be pretty much “mission impossible.” At the time, we made a conscious decision to focus only on Amazon for our first generation PaaS as practically it was the only cloud in town.
Now, there are many different public and private cloud offerings: platforms like GoGrid, VMWare, Citrix/Xen, and Cisco UCS, with recent additions being OpenStack, Cloud.com, and Microsoft Azure. Frameworks like JClouds have been developed to make cloud portability an easier goal to reach.
As a result, cloud portability is not only a possibility, but it's easily done with the right constraints in mind.
Having said that, there are still too many options from the standpoints of API standardization, portable virtual machines, abstraction frameworks, orchestration frameworks, etc. None of them fully address the challenge and therefore a solution has to combine these options into one cohesive unit. Finding the right combination of features is a pretty tricky challenge and involves lots of trial and error, as the chances that you’ll pick the wrong ones are still pretty high, as we experienced ourselves.
In this and the next few posts, I wanted to share some of our experiences when developing Cloudify , our unified cloud deployment/management tool. I will start by covering five common misconceptions people have.
Five Common Misconceptions on Cloud Portability
1. Cloud portability = Cloud API portability
This is perhaps the most important observation in this entire discussion. When people think of Cloud portability, the immediate thing that comes to mind is Cloud API portability - a standard API, or a common abstraction, that maps all the different API’s into one common façade, right? Well... wrong.
What you're really looking for is the ability to run your application on different cloud platforms without any code change. Having a common or standard API is one way to achieve that goal but not necessarily the most practical one, given the speed in which cloud APIs evolve. This brings me to the first observation: Application Portability != Cloud API portability. Let me explain:
Most applications don’t need to interact with the cloud-specific API from the application itself. Most of the cloud API deals with stuff that happens outside of the application code, like starting new virtual machines or services, or providing elasticity. In many cases, systems management tools are the primary specific aspect for a given cloud platform; most provide support for managed middleware like MySQL (RDMS), Tomcat, memcached, or even map/reduce (through Hadoop, for example.) The mechanisms for using these are standard and common, but the APIs for managing the services are not. (There are exceptions, such as SimpleDB and SQS, which do use proprietary and localized APIs. I believe that over time, you'll see less and less of this as there will be enough nonproprietary options to restrict their use.)
The problem of dealing with application portability between clouds is vastly different than the problem of dealing with API portability. API portability is easy; cloud API portability is not.
I’ll spend more time on how we can use this realization to simplify the task of achieving cloud portability in a follow-up post.
2. The main incentive for Cloud Portability is - Avoiding Vendor lock-in
Vendor lock-in is one of the major barriers to cloud adoption as indicated in a recent survey by 451, GigaOM and North Bridge Venture - Future of cloud survey shows significance of open source
Interoperability and vendor lock-in were ranked the next most significant challenges, both with 25% of response. Though we saw vendor lock-in fade a bit as a concern for customers two to three years ago, it has risen again as a major issue in cloud computing. We believe this is in part because of the early nature of cloud computing and a desire from users to avoid getting stuck with a cloud vendor, framework or technology.
Many (myself included) view cloud portability as a mean to address vendor lock-in as the main drive for Cloud portability. If we can run our application on any cloud without any code change, we can in fact avoid vendor lock-in.
This brings me to the second realization. Cloud portability is more about business agility than it is about vendor lock-in. Let me explain:
It is not very likely that you would switch entirely between one cloud provider to the other, at least not frequently. If you are going to change cloud providers, its going to be more of a one-off event than something that you would practice on a regular basis.
On the other hand, it is probably more likely that you would use cloud portability to choose the right cloud for the job. A common example would be to run your test on the cloud and production on your own private cloud. Another example would be to run your demos and trials on the “cheapest” cloud and production with a cloud provider that offers a higher service level. A more advanced scenario would be to use cloud portability for cloud bursting.
In other words, cloud portability is about business flexibility. It gives us more choices between cost, SLA, and security tradeoffs between the different cloud offerings and product lines. Portability between our local data center and the cloud would also help in making the transition to the cloud more smooth.
3. Cloud portability isn’t for startups
If you're a startup, when the discussion on cloud portability comes up the immediate reaction is often “hmm.. interesting but not for me.” Being part of a startup company myself, I think that I can relate to that reaction. As I mentioned before, when I faced the choice for dealing with cloud portability a few years back, cloud portability was one of the first items that I took off of my list of TODOs to meet our time-to-market goal, and settled for Amazon.
Recently, I came across different cases that shows that dealing with cloud portability is often forced on us even when we don’t plan for it.
The first case is with MixPanel, a web analytics company who switched from RackSpace to Amazon:
When MixPanel started up, cost mattered as they were running out of their own pockets. In this case, every dollar counted and therefore the right choice at the time was Slicehost. As their business grew, Mixpanel switched from Slicehost to Linode and later to Rackspace due to a Ycombinator deal. As their business grew, they switched to Amazon who happened to provide more features and greater flexibility .
First was Slicehost back when everything was on a single 256MB instance.. Second was Linode because it was cheaper (money mattered to me at that point). Lastly, we moved over to the Rackspace Cloud because they cut a deal with Ycombinator... Even with all the lock in we have with Rackspace (we have 50+ boxes)..it’s really not about the money but about the features and the product offering, here’s why we’re moving”
source: Amazon vs Rackspace
The second case is BeaTunes that switched from Amazon to EQ8 Hetzner
In the case of BeaTunes, the trigger for switching off of Amazon was an Elastic Block Store (EBS) crash. During that time, I had to rely on basic support that Amazon provided. Given that BeaTunes is a small startup, they had almost no leverage to escalate their issue fast enough for the business need and had to hope that their particular issue would hit enough users in the forum to attract Amazon's support staff's attention.
They ended up choosing EQ8. Being a smaller hoster than Amazon made EQ8 a better fit for BeaTunes, a small startup itself. Even a small startup can run mission critical software, and when there is a failure, it is even more important to get a timely response as your entire future may rely on this. Unlike big companies, you often don’t have the means to survive such failures easily. It also happened that by switching to EQ8 BeaTunes could choose better hardware configuration that fit their needs and at half the price of the original services!
Amazon: “two servers $380/month. Add roughly $220/month for backup snapshots and EBS for the (pretty large) database and your total yearly costs are $,7200”
EQ8: “two 24GB RAM, i7-920 Quad-Core, 2x1.5TB HDD (Software-RAID 1) servers for $3,530/year at a hosting company that you can actually call
Source: Goodbye Amazon EC2 See you later cloud
These are only two samples, representative of a much bigger trend in the industry at large.
The interesting thing in both MixPanel's and BeaTunes' histories is that their choice of cloud changed over time due to changes in their maturity as a company, as well as changes in the business requirements which involve cost, support level, flexibility and feature set, etc. At each point in time, the right cloud happened to be a different cloud.
Personally, I found that issue around supportability in the case of BeaTunes quite interesting as quite often we tend to go for the brand thinking that it’s the “safest bet” where perhaps the right choice would be to choose the one that fits our size (and stage).
The main point in both examples is that even during a relatively short period of time, the two startups found themselves switching from one cloud to another. In the case of MixPanel, this happened four times during their company lifetime.
The thought that you can avoid such a move between cloud platforms is likely to be too optimistic, even if you’re a startup. Given that there are more options to deal with cloud portability today, and the effort is not as a great as was the case few years back, I would encourage every startup that is expecting rapid growth to re-examine their deployments and plan for cloud portability rather than wait to be forced to make the switch when you are least prepared to do so.
4. Cloud portability = Compromising on the least common denominator
By definition, standards tend to lag behind implementations. Standards are therefore a compromise on the least common denominator. In a dynamic environment, such as the one we're experiencing with cloud today, compromising on the least common denominator is a choice which may come with great cost on “reinventing” things that are already available outside of the standard or common API.
If we think of application portability, we don’t need to compromise on the least common denominator as most of the interaction with the cloud API happens outside of our application code anyway, to handle things like provisioning, setup, installation, scaling, monitoring, etc.
There are orchestration tools such as Chef and Puppet that can provide higher levels of abstraction for automating those processes between clouds without relying on a common cloud API. Frameworks like JClouds provide common abstractions to the common set of APIs (such as Compute and Storage APIs) while still allowing the application to interact with the underlying cloud-specific APIs and thus minimizing the areas of differences between clouds that would require specific handling.
5. The effort for achieving cloud portability far exceed the value
Many would argue that cloud portability comes with a cost and rightly so. Indeed, there is no such thing as free lunch, and cloud portability is no exception.
However, the cost isn’t as great as many of us think, especially now that there are more tools and frameworks available.
The effort to achieve cloud portability is far less than it used to be, in most cases, making it a greater and more valuable priority (with less investment) than it used to be.
Final notes
The term "cloud portability" is often considered a synonym for "Cloud API portability," which implies a series of misconceptions.
If we break away from dogma, we can find that what we really looking for in cloud portability is Application portability between clouds which can be a vastly simpler requirement, as we can achieve application portability without settling on a common Cloud API.
As in the case of MixPanel and BeaTune, choosing the right cloud for the job may vary over time. The right cloud when we create a new startup can be different than when we grow, and if we're very successful, we may find out that managing our own cloud is the right choice.
If we focus only on what’s needed to ensure application portability between clouds, we may find that cloud portability can be easier than it seems at first glance. If done correctly, it can result in greater flexibility for our businesses:
-
Choose the right cloud for the Job
-
Reduce vendor lock-in
-
Enable advanced deployments such as a hybrid cloud, and cloudbursting
In the next post I’ll touch on what it takes to turn cloud portability into a practical reality.
References
