The Promise of a Cloud Application Marketplace
The cloud has clearly emerged as the new world. Today, we find ourselves in an era of land-grabbing in which everyone wants the bigger piece of this new world.
Amazon has the advantage of being the first to grab. Others would like to follow in Amazon’s footsteps, but are met with the challenge of not being able to out-pace Amazon.
For those of us who aren’t Amazon, forming a coalition of some sort could be a good strategy.
One way of forming a coalition is through a marketplace, a system that enables faster growth through ecosystem partners.
A good example of how such a dynamic will work can be seen in today's world of mobile applications. Both Apple and Google were able to create strong marketplaces that grow much faster than they would have organically and add significant stickiness to their platforms.
Challenges of Building a Marketplace - Lessons from the Mobile Application Marketplace
A marketplace can be an excellent strategy for forming a coalition of partners and growing your offering significantly faster than you would if you were to try to grow organically.
Having said that, there is a significant advantage to being the first. As we have seen in the mobile application world, others who have tried to build a similar marketplace to that of Google and Apple are facing significant difficulties in attracting partners to their marketplaces (See Microsoft Surface as an example).
The main reason for these challenges is the Chicken vs. the Egg problem. In other words, for any marketplace to work, two essential factors must be in place: suppliers and consumers. Apple and Google already had their consumer-base and, therefore, it was easier for them to attract suppliers. Microsoft, on the other hand, does not yet have the mobile consumer-base on its side and is therefore facing a hard battle to bring in the suppliers.
A Cloud Application Marketplace
A Cloud Application Marketplace is similar in concept to that of mobile applications, but aims at cloud data-center services such as Database services, Application Containers, Messaging, CMS, CRM etc which are deployed on a cloud.
Amazon Cloud Marketplace
The marketplace concept is nothing new to Amazon and is already running a successful marketplace for its eCommerce business.
It only made sense for Amazon to latch on that model and use it as a basis for building its Cloud Application Marketplace, as described here.
The Limitations of Amazon Marketplace
While the promise of the a Cloud Marketplace as an ecosystem play is a no-brainer, an over-simplified implementation of it can yield a diminishing return as noted in Janakiram MSV article AWS Marketplace – An Oversimplification of the Cloud? Janakiram points specifically to the following limitations:
· Persistent Services Limitation:
By default, the database and other persistence-dependent services are confined to the ephemeral storage or instance storage of the EC2 instance. This is a very risky proposition as the data will be lost completely in the case of an instance failure. I wouldn’t be comfortable even running a non-mission-critical application like a WordPress blog on EC2 without attaching an EBS volume.
· Not Designed for Real Production:
Launching a LAMP or Tomcat stack with 1-Click is great for a small proof-of-concept or a demo, but no one would attempt this for a production application. Moving to the Cloud is not just about forklifting an app from on-premise server or a host to a VM. It is definitely much more than that! AWS Marketplace squarely focuses just on compute by launching an EC2 instance with pre-configured stack without considering any of the essential building block services like storage, networking and databases.
· Loadbalancer Limitations:
For example, the Amazon EC2 instances launched through the Marketplace cannot be added to an Elastic Loadbalancer. I attempted to launch 2 instances of the free Tomcat 6.0/MySQL Stack provided by OpenLogic and adding them to an Elastic Loadbalancer. ELB threw an error and refused to add the instances. This is a huge limitation for any real-world web application that intends to run on a Tomcat instance launched via the Marketplace. I wouldn’t be comfortable running my web application in just one instance without any possibility of scaling out in the future.
The Limitations of the VM-Centric Approach
Cloud applications are often packaged and run on VMs. It seems only natural that if applications are packaged on VMs, a marketplace would be a repository of pre-packed VMs, as in the case of the AWS marketplace.
Having said that, there are some basic flaws in this assumption - cloud applications and services are much more complex. Many of the services and applications need to run on high-availability mode, which often means that they need to run in more than one availability zone with inter-connectivity between them. In addition, many of the services, such as database services or web-services, need to run in a cluster deployment. In order to do so, multiple instances of VMs must be deployed. As real life applications tend to be multi-tier, there needs to be a way to deploy a set of VMs with some level of interdependency between them. When running an application on the cloud, we often need to think of it as a living organism that continues to change and evolve, rather than one-off deployment.
All these factors make the VM-centric approach a good trial and demo tool, but as Jankiram pointed out, a significant over-simplification of what cloud applications tend to be.
AWS Marketplace - Not a Fair Play
In order for a marketplace to be successful, it is essential that all players will get a fair shot to win consumers.
A marketplace owner in this context needs to take the role of "honest broker."
Amazon offers a long list of its own set of services as products outside the marketplace that don't face the limitations I listed above, in contrast to the applications that do run on the marketplace and are met with these challenges. This gives Amazon an unfair advantage over its own partners.
Building an Open Marketplace on OpenStack
The creation of an open marketplace would serve as an alternative to that of Amazon. A good substrate for that alternative approach would be to use open cloud frameworks like OpenStack. OpenStack already has a growing community of users and a growing support of cloud providers which provide an excellent starting point for the marketplace.
Below are some of the principles that I believe should serve as a foundation for building an open marketplace.
A Recipe-Based Marketplace
While the basic unit of deployment in many clouds tends to be based on VMs in many *real* deployments, the VM is used mostly as a generic container of the operating-system. The actual application services are deployed through a DevOps automation tool such as Chef, Puppet or Cloud Formation (Heat is the OpenStack alternative for CloudFormation on OpenStack) after the VM has been created. In this way, the deployment unit is the recipe rather than the VM. The recipe is used to describe the application services that will be deployed into the VM after it has been created. Unlike a VM, a recipe can be extended beyond the boundary of a VM and can describe fairly complex, multi-tier, clustered applications. It can also deal with the automation of post-deployment processes such as upgrades, monitoring, fail-over etc.
Based on that description, it only makes sense that we will use recipes as the unit of deployment instead of VMs. In this way, we can overcome many of the AWS marketplace limitations that are outlined above. As there is no standard recipe model as of yet, we'll need to be able to integrate with existing recipes such as Chef, Puppet and CloudFormation.
Other Considerations
Having a recipe-based marketplace is a good starting point. However, there is much more to building a marketplace than to choosing the recipe-based engine upon which to run an application. The main challenge is around the user experience and mastering the balancing act of keeping the marketplace attractive for both the consumers and suppliers.
Below are some of the key principles that I believe should be taken into consideration when building an open marketplace:
- Open source - An open source marketplace would enable different cloud providers to offer and share similar applications and services across different cloud providers.
- Popular frameworks – The marketplace should be based on popular open frameworks such as Chef, Puppet or CloudFormation/Heat templates.
- Reduce the barrier to entry - Make the investment of porting the application to the marketplace worth the effort beyond the marketplace. To reduce the barrier of porting applications into the marketplace we need to design it in such a way that we could use the same elements for deploying our application outside the marketplace.
- Support both multi-tier and cluster deployments
- Extensible - Users can pick a service from the marketplace and easily customize it to their needs.
- Fair play - Applications that are running through the marketplace should have the exact same capabilities as any application that is running outside of the marketplace.
If You Build It, They Will Come - Doesn't Work
These days I'm seeing many cloud providers trying to mimic Amazon’s success and launch their own marketplace. As in the case of Amazon, they also start with a VM-centric approach and thus, inherit many of its limitations. Furthermore, the main problem that I see with many of those attempts is that they seem to be taking the "if you build it, they will come" approach, not realizing that they don't have the advantage of being first and, more importantly, are not at the same point where Amazon was when it launched its marketplace ( i.e. with a huge consumer-base running on their platform).
As a result, I personally believe that many of those attempts are doomed to fail.
The real opportunity IMHO is to take a more open approach that could work outside the boundaries of a specific cloud provider. OpenStack could serve as a great starting point for forming this ecosystem.
To illustrate the points from this post on what an Open Marketplace could look like, in the next post I'll refer to how we use Cloudify as an open source framework for building your own cloud application marketplace on OpenStack.
