Nati Shalom's Blog

Going Hybrid Cloud with OpenStack

OpenStack Silicon Valley is just a few short weeks away and we are looking forward to talking OpenStack with the local community.

A main focus for Team Cloudify at the event will be around the topic of portability AKA hybrid cloud, not only between environments, but also between OpenStack and non-OpenStack environments. In July we announced the release of Cloudify 3, a complete rewrite and redesign of Cloudify.  This change provides an intelligent orchestration and management platform that is even more compatible with the open source world of OpenStack. The platform is designed to integrate with any tool chain and provide interoperability across stacks and environments.

Why the focus on portability? It is our core belief that for the foreseeable future, supporting hybrid clouds is going to be important for many OpenStack users. Here are a couple of reasons why:

1. Transitioning to OpenStack is a long journey - Many enterprises are based on VMware as their core infrastructure. Often, those organizations begin their journey to OpenStack by creating a dev environment and gradually moving more of their workload onto OpenStack as they become more accustomed to the new environment. This process typically spans the course of months and sometimes years. It is clear that during this period of transition, customers will continue to use vSphere and vCloud in parallel with OpenStack. For those users, having a common platform with which to manage their app deployment across those two environments can ease the transition and reduce the risk of hitting a roadblock if one of the environments does not meet their needs.

1. The public cloud market is going to be dominated by non-OpenStack clouds - While OpenStack is on a trajectory to dominate the private cloud market, the public cloud market is led by AWS with other major clouds emerging, such as GCE and Azure. Many organizations would not want to host their entire data center on a private cloud and would, therefore, need to take a hybrid cloud approach. In order to support hybrid cloud environments, an abstraction layer is needed to enable integration with clouds other than OpenStack.

While we strongly believe in the necessity of supporting hybrid deployments, the integration with OpenStack was a major consideration when redesigning the Cloudify platform. By switching to Python code, developing compatibility with Heat and supporting the TOSCA specification, Cloudify is now more than ever more deeply integrated with OpenStack’s core services.

This deeper integration with OpenStack technology makes Cloudify even more equipped to handle the task of multi-cloud deployment and management between OpenStack and not-only-OpenStack environments.

Open Cloud Deconstructed. What does it mean to be OpenStack Native?

OpenStack is just not like any other cloud.

OpenStack is an open source cloud and that makes it quite unique in comparison to many of the closed source public clouds generally available today, the most common being AWS, Azure GCE, and others.

The fact that it is open source, allows us to integrate with the lower level of the stack and know exactly how each element of the infrastructure is designed and implemented.

As with any open source project we have the option to contribute and influence how the OpenStack cloud ultimately shapes out.

Given the above background, when we're designing an integration with OpenStack, we should approach it differently than we approach other clouds. A more native integration with OpenStack involves two main angles:

• Integration with OpenStack core and infrastructure services, which usually includes this full OpenStack component list - as well as those that are continually being developed and added with every release.

• Follow the OpenStack design principles - OpenStack is written in  Python and uses a certain stack for messaging, database and such . To be OpenStack native you should align yourself with the OpenStack design principles as closely as possible.

Current options to be a part of the OpenStack ecosystem

In addition to the general guidelines outlined above, there are couple of additional options to be part of OpenStack. These options mostly fit in with those looking to take an OpenStack *only* approach, and require heavy involvement in the OpenStack community project.

• OpenStack Core - be a part of the OpenStack distribution

• OpenStack StackForge is an incubation framework that allows you to add new projects into the OpenStack ecosystem. StackForge uses the same set of tools and frameworks that are used to develop the core OpenStack services, and as such it allows smoother transition from an incubation project into the core services. That makes this option more relevant for projects that are likely to be included as an OpenStack core service once they've successfully passed the incubation stages.

• OpenStack Marketplace - The marketplace is a web directory that lists the various ecosystem players and their OpenStack solutions. Currently the marketplace is focused on OpenStack distros, training, consulting and drivers, and doesn’t cover solutions on top of OpenStack.

What’s missing?

While there is a fairly clear process, and tool chain for handling core and incubated services on OpenStack, these options fit mostly with services that are primarily designed to only run on OpenStack. What’s missing is a clear certification process that will allow users to find solutions in the ecosystem, outside of OpenStack, and find out quickly what their level of integration and support for OpenStack is. This should include information such as version support, and platform the solution was certified on, which of the OpenStack services are integrated, how much of the stack of the solution is compatible with that of OpenStack’s, and other such important information. A good and simple starting point could be the OpenStack marketplace, however this is just the first step. What’s needed is a set of frameworks similar to StackForge but much more permissive that will provide a means for validating the compatibility with OpenStack.

Why should you care?

As with any popular framework there are many tools that claim support for OpenStack, however in many cases the integration mostly includes basic integration with the OpenStack compute API (Nova). A good example for this is Cloud Foundry, or other tools such as Chef, Puppet, Docker, just to name a few. While these tools do have support for OpenStack, the level of integration can vary quite dramatically. Quite often, each of these  solutions comes with a parallel stack for handling security, monitoring and other issues, which isn’t compatible with the OpenStack infrastructure. As an end user, it is important that to be able to find out quickly which of these products comes with basic support for OpenStack, and which one takes a more native approach to OpenStack. There are many implications to the level of complexity and also to the level vulnerability of those services based on the support level.

OpenStack should attract more ecosystem projects rather than competing with them

In my opinion, there has been too much focus developing more alternative projects for the OpenStack project in comparison to the focus that has been placed on projects that exist within the broader ecosystem. The main reason for this, is the lack of native support for OpenStack. A good example of this, is project Solum and its ecosystem equivalents Cloud Foundry and OpenShift. 

I think that it’s time that OpenStack realign its focus from growing all around the stack, to supporting and attracting more ecosystem projects, that focus on adding native support to OpenStack. In order to do this, there needs to be more emphasis on creating certification, compatibility and testing projects. Additionally, the marketplace initiative needs to receive more attention and investment in order to make it more than a web-based directory service, similar to the AWS marketplace.

I would like to propose a discussion on that subject during the next summit in Paris. Do you think this is a good or bad idea? I’d appreciate it if you could share your thoughts.