Randy Bias CTO of Cloud Scaling sparked an interesting debate around the need of OpenStack to align itself with major public cloud offering, namely Amazon and Google clouds. According to an interview on GigaOM the timing behind Randy's proposal is a result of the concern that Rackspace is heading to a different and potentially opposite direction with their view of Amazon as a rival to OpenStack and also from the thought that relying on AWS API could be too restrictive from an innovation perspective.
Portability Defined
While the debate was sparked around the use of the so called Native (Nova) API according to the twitter thread between Randy Bias @randybias, Dave Neary @nearyd, and Simon Wardley @swardley, it appears that the three look at portability in its broader sense (not just about around Nova API):
@nearyd: “@randybias user confidence that they can easily move work off one OpenStack to another"
@nearyd: “@randybias Also ability to burst off private cloud to a public cloud will be important.”
According to this definition what were really looking for is application portability from non-OpenStack environments to OpenStack.
API Interoperability != Application Interoperability
I think that most of the community would agree with the general sentiment behind Bias’ call that OpenStack should align itself with that of major public clouds. Now zero-in on the Nova API as the means for portability – this is where the debate lies and where I feel that Randy may be wrong in his assessment for the following reasons:
- The complexity is underplayed - as Adrian Cockcroft from Netflix pointed out:
- The permutations and complexity of doing this across many vendors has sunk many previous attempts over the last decade.
- Randy’s reference to CloudStack AWS API bridge is actually a reference to why API bridges aren't that useful after all. Based on my experience, most if not all CloudStack users do not really use the CloudStack AWS bridge as it doesn't expose many of the CloudStack features, it’s not well tested and doesn't always work well simply because it's not considered a main path. I also heard similar feedback from Adrian Cole, the founder of the JClouds project - a popular cloud abstraction framework.
- API Portability != Application Portability - API portability covers only a small part (and not necessarily the important part) of what is needed to get true application portability, as noted in Chief Engineer at eBay Subbu Allamaraju's post OpenStack is not Cloud and also in OpenStack COO Mark Collier’s response:
"I do agree that APIs sometimes get more credit than they deserve. Sometimes people oversimplify the concept of a platform and the compatibility of a platform based on just saying, 'If you've got the API, you've got compatibility.' The reality is that an application architecture depends on the behavior of the whole system it's interacting with. The API is simply the way it talks to that system."
What We Can Learn from the Android vs IOS Experience on Cloud Portability
Apple IOS and Google Android are actually good references for achieving fairly good application portability between two environments without necessarily agreeing on the same API.
Can you imagine Google adopting the same IOS API?
This shows us that there must be alternative approach to portability.
Alternative Path for Portability
Subbu Allamaraju suggested that portability should be done through glue code.
“AWS API compatibility is what an operator should worry about above all these? Nah. You can fix API incompatibility with glue code.”
Jeremy Jarvis, Co-founder of Brightbox, suggested a similar idea in his comment to Randy's post:
"Anyone writing code to interact directly with APIs of popular clouds is probably doing something wrong (and certainly wasting effort). There are a number of cloud abstraction libraries which are doing a great job in providing a unified interface to multiple clouds. Fog (Ruby) is a great example - sponsored by Heroku no less - of how a community/ecosystem is developing around multi-cloud interaction."
Ben Kepes pointed to orchestration framework as an alternative in his post On OpenStack and the AWS API – Internal Divisions Surface
“If the future is, as I suspect, made up of enterprises using lots of different cloud offerings (not so much in a hybrid/cloud migration model but more of a discrete multi cloud approach) then there is less of an urgent need to resolve to one common API set. So long as there are products that give visibility and management over these discrete resources (and this is where Dell, with its recent acquisition of enStratius and dropping of its own public cloud play, is thinking) the actual interoperability or commonality of the different products API sets is secondary."
My Take
Given the dynamic nature of the cloud environment, there is no doubt in my mind that portability is important - not just between AWS, GCE and OpenStack, but also between different OpenStack providers, as well as between traditional data-center environments and OpenStack as I noted in my previous post To OpenStack or Not to OpenStack? Moving Enterprise Applications onto OpenStack Today. Having a native cloud API that is compatible with AWS and GCE is wishful thinking and, unfortunately, makes it less practical at this point in time.
Application deployment is better described in recipes, which cover a wider spectrum of application deployment and behavior - such as the tiers of the application, the metrics, the scaling policies, the configuration etc. The OpenStack Heat project, which is actually driven from its AWS equivalent CloudFormation, can get us much closer to the desired portability, even if the underlying Cloud API doesn't conform to the same API. If done right, this means that we could execute our application recipes in a similar way on both AWS and OpenStack.
Having said that, in order for OpenStack Heat to become the solution for cloud portability there is more work to be done in designing the framework in a way that will be more loosely coupled with the underlying OpenStack infrastructure and also to adopt richer deployment recipes that can cover more aspects of our application deployment in a declarative way. It looks like there are already plans in this direction, but the progress seems to be fairly slow. Therefore, I think that it would be both simpler and less disruptive for the community to align its effort in maintaining portability around the Heat project rather than re-writing the Nova API.
Thoughts From Not so Long Past..
While I was writing those lines I came across one of my previous talks Making Cloud Portability a Practical Reality from 2011. I thought that many of the points from this presentation are still relevant… Enjoy.
References
- OpenStack’s Future Depends on Embracing Amazon. Now.Embrace major public cloud APIs (Randy Bias blog)
- To OpenStack or Not to OpenStack? Moving Enterprise Applications onto OpenStack Today
- On OpenStack and the AWS API – Internal Divisions Surface (by Ben Kepes)
- Is Rackspace holding OpenStack back? (GigaOM)
- OpenStack is Not a Cloud (Subbu Allamaraju's Blog)
- Making Cloud Portability a Practical Reality
- COO Mark Collier discusses OpenStack roadmap, API fit with AWS
- Portability defined- a Twitter thread
