Nati Shalom's Blog

Big Data on OpenStack

Big Data systems by their very nature tend to be...Big.

Big in the amount of data, in the number and size of infrastructure behind it, etc. Cloud based infrastructure can be a good fit as a cost effecive instrastrucutre for running those Big Data systems. While this may sound obvious, many of the Big Data deployments that I've encountered run outside of a cloud environment. The main reason is the performance overhead that is often associated in running on a virtualized cloud environment. Let me explain..

The Performance Overhead of Running Big Data on the Cloud

A recent benchmark done by Petersenna over various virtualization solutions, such as XEN, VMWare, HyperV etc, shows that on average the performance overhead associated with virtualization can lead to 2.4 times slower disk latency and 25% slower network I/O.

How does the performance overhead translate into cost?

Based on the various benchmarks, I think that it would be fair to assume that running I/O intrusive workloads such as Big Data on a virtualized infrastructure would require 3X more resources than its Baremetal equivalent.

As we are talking about Big Data infrastructure - 3x means a lots of wasted resources. The operational costs coupled with the complexity associated with running a significantly bigger system yields a fairly substantial overhead and cost. In addition, the Network and Disk overhead are not deterministic and can vary quite substantially when the utilization gets higher. This leads to not just performance overhead, but also non-deterministic overhead. In Big Data terms, that means that a query for particular data can take 10msec one time or 30msec another time. Quite often, running Big Data analysis requires a sequence of those operations. Therefore, if we pile this overhead in a sequence of 10 queries, the variance in the response time can vary quite substantially.

While the choice of running Big Data on the cloud holds a lot of promise, the performance and non-deterministic behavior makes that choice limited to more of a niche scenario where this overhead  becomes less significant in comparison to the elasticity benefit of the cloud. For example, for sporadic worklads in which we run our analysis for a certain period of time and then can release the resources, using on-demand infrastructure is still a better choice, as running 3x the amount of resources for an hour is significantly cheaper than having a third of that infrastructure allocated 24/7.

OpenStack Bare Metal Cloud

The analysis above shows that for I/O intensive workloads, virtualized infrastructure isn't such a good fit.

Cloud is often viewed as an infrastructure on top of virtualization. Therefore, by definition, a cloud-based infrastructure inherits the benefits and limitations of virtualization.

Is the coupling between cloud and virtualization really mandatory?

If we think of a cloud as an infrastructure for getting compute, storage and network resources on-demand, then there is nothing  to necessitate the coupling of the cloud with virtualization. It became common to pair the two mainly because of the complexity involved with provisioning non-virtualized resources and the limitations in enabling partitioning of a given bare metal machine.

As I noted in one of my previous posts Bare Metal Cloud/PaaS,  there are ways today to provision a baremetal machine from an image and partition it just like we would do with a hypervisor based VM.  There are already cloud providers that offer a choice of bare metal machines as part of their cloud infrastructure.

A new bare metal project in the OpenStack Grizzly release takes this a step further. It allows us to use the same compute API (NOVA) and allocate a bare metal machine just as we would with a virtualized machine.

All we need to do to make the switch is to change the image type of our target machine and the cloud infrastructure will know to map that request and allocate a bare metal image instead of a virtualized instance.

With this option we can now run our entire Big Data workload on the cloud and not worry about switching environments depending on the workload happen to be sporadic or I/O intesive.

HubSpot OpenStack – Bare Metal Case Study

During the last OpenStack Summit, Jim O’Neill CIO at HubSpot showed how using OpenStack with a combination of a public virtualized cloud and private bare metal cloud enabled a 4X increase in their infrastructure efficiency:

 

“We took this single image, picked it up from public cloud into a Rackspace-powered private cloud and saw a 4X increased efficiency running that workload.”

 

Moving from Existing Data Centers to the Cloud

Many existing Big Data and BI systems run on traditional data center environments. Moving those systems into an OpenStack-based environment isn't going to be a walk in the park. This is where automation frameworks, such as Cloudify with the combination of Chef and Puppet, make this transition smoother.

In this approach, we can automate the deployment of our existing Big Data/BI systems in a way that will be abstracted from the underlying infrastructure. We can later use this abstraction to run Big Data systems in our existing data center, and when ready, we can use the same deployment framework on an OpenStack-based environment without re-doing any of that investment.

 

 

Learn more through hands-on experience - a Real Life Experience on HP OpenStack

As it often happen in this sort of conceptial discussions the points and arguments from this post may often sound artificial and not easy to grasp. To make it more down to earth, we've put a NoSQL datastore such as Couchbase, MongoDB, Cassandra, ElasticSearch and Big Data applications available on-demand on HP OpenStack cloud services. You can use this reference to launch any of those applications and then use the management console to browse through the recipes and customize it to your environment as needed.

It is also worth pointing out that the recipes behind this project are available on Github and can be easily deployed on your cloud or in your data center or even desktop in pretty much the same way. If you are interested in more details, please post a comment on this post or in the Cloudify forum.

For more on Big Data on OpenStack, come check out my presentation at Cloud Expo East on Tuesday, June 11th at 8:15am in the “Cloud Computing and Big Data” Track.

References

• Bare Metal Cloud/PaaS
• OpenStack Baremetal Project
• Big Data in the Cloud
• Big Data in the Cloud using Cloudify
• Putting Hadoop On Any Cloud (A video presentation)
• In Memory Computing (Data Grid) for Big Data
• Using the Cloudify Player as an Open Source Framework for Building Your Own Cloud Application Marketplace on OpenStack
• Going native: The move to bare-metal cloud services
• New bare metal cloud offerings emerging
• How much overhead does x86/x64 virtualization have?
• Amazon EC2 versus Bare Metal and KVM? The inside story on what you thought you knew about EC2

Using the Cloudify Player as an Open Source Framework for Building Your Own Cloud Application Marketplace on OpenStack

In the previous post, I outlined the main lessons learned from the Amazon Cloud Marketplace and suggested an alternative approach for building an open marketplace.

In this post I'll refer to a specific implementation of that alternative approach through a new Cloudify Service called the Cloudify Player.

Introduction to the Cloudify Player

Cloudify Player is a new patent-based service that makes the experience of trying and deploying even the most complex, multi-tier big data applications as simple as playing a video on YouTube. The new service currently supports OpenStack-based clouds and is available as a free online service from HP Cloud Services and GigaSpaces’ Cloudify. The source code for the service is also available as an open source project on Github. The Player was developed in collaboration with HP Cloud Services, GigaSpaces’ cloud partner and leading provider of OpenStack-based public cloud services.

Cloudify Player an End User Perspective

The Cloudify Player provides a user-friendly and hassle-free user experience in which applications are "played" in similar fashion to the way you play a video online. Users can choose and browse applications from public catalogs that are based on Github. Similarly to YouTube, users can share the applications or embed them easily on their blog or website.

Users can choose to play any application as a free trial or for full production under their account.

Technical Overview

The Cloudify Player consists of the following components as outlined below:

Web Front End

The Web Front End is based on two main interfaces: a widget interface which allows users to browse through a list of applications and deploy them, and a widget management form is provided to allow users to easily publish new applications.

Users can use one of the two deployment options - a trial deployment provides a free non-gated experience for running the selected application on a single machine. Registered users can provide their cloud credentials and route the application into their cloud environment. In this way, they can also monitor the deployed instances through their cloud management console.

Multi-Tenant Orchestrator and Monitoring

The multi-tenant orchestration and monitoring service is responsible for parsing the selected application recipe and for executing the deployment of that application based on the description provided in the recipe. The web-based management allows users to monitor and manage the deployed application.

Application Analytics

Application activities are sent into an analytics service that produce real-time usage reports on the most-active application, as well as the user experience activities.

Service Catalogue Repository

The Service Catalogue is based on a Github clone.

Users use the repository to store their application recipes, track versions, manage history of updates, etc.

Cloud Pools

Cloudify can be integrated with various public and private clouds using out-of-the-box plug-ins.

There are basically two main cloud pools that are used in this architecture:

1. Free Trial Pool - The free trial pool is used to speed up the deployment experience during the trial stage. This pool is limited for time and is initially set for a single VM per deployment to avoid quick exhaustion of resources from the free pool.

2. Production User Cloud - Users that pass their credentials could use the same interface for deploying application under their account. In this case they are not limited in time nor the number of nodes per application.

Product Overview for IT Managers and DevOps

The Cloudify Player was designed to fit into a DevOps environment in which applications are expected to be modified on a continuous basis after they have been deployed. Users can add new applications easily by wrapping them with a simple script, called a recipe. Recipes can be based on either Chef or Puppet.  Recipes can be modified after they have been published by each end user before the deployment takes place.

Users can monitor and manage their applications using a single management dashboard. The management dashboard allows users to monitor the application topology and behavior as well as view real time logging and even open SSH console if needed.

Summary of the Main Features

• Out-of-the-box support for 100s of products (Leveraging the existing Chef/Puppet support)
• Supports full life cycle of user experience from simple free trial to full production
• Specific support for Big Data clusters, including Database as a Service, Hadoop as a Service, MongoDB as a Service, etc.
• Can run clusters rather than just individual VMs
• Application can run outside of the marketplace using the same tools
• Users can easily customize the services from the catalog for their own purposes, including the possibility to run their own version of each item in the catalog. Advanced users can leverage the built-in Chef & Puppet support for advanced configuration management.
• Built in multi-tenant application management so that each user can monitor and manage all of the products under his/her account using a single application-centric dashboard
• Built-in integration with analytics tools to support tracking of user activity

Using Cloudify to Build Your Own Marketplace

The Cloudify Player was designed to simplify the deployment of applications in the cloud. The open nature of the Cloudify Player provides many of the features required for building a marketplace in your own environment under your brand and with the application of your choice. In addition, we’ve put a lot of emphasise on making the user experience simple while ensuring a smooth transition from a hassle-free trial to full production usage.

You can try out the OpenStack version that we launched in collaboration with HP at www.cloudifysoure.org.

For more information about the Cloudify Player and how it works, please see Uri Cohen’s blog post Introducing the Cloudify Player.

Amazon Marketplace - Not a Fair Play, Thoughts on an Open Marketplace Alternative with OpenStack

The Promise of a Cloud Application Marketplace

The cloud has clearly emerged as the new world. Today, we find ourselves in an era of land-grabbing in which everyone wants the bigger piece of this new world.

Amazon has the advantage of being the first to grab. Others would like to follow in Amazon’s footsteps, but are met with the challenge of not being able to out-pace Amazon.

For those of us who aren’t Amazon, forming a coalition of some sort could be a good strategy.

One way of forming a coalition is through a marketplace, a system that enables faster growth through ecosystem partners.

A good example of how such a dynamic will work can be seen in today's world of mobile applications. Both Apple and Google were able to create strong marketplaces that grow much faster than they would have organically and add significant stickiness to their platforms.

Challenges of Building a Marketplace - Lessons from the Mobile Application Marketplace

A marketplace can be an excellent strategy for forming a coalition of partners and growing your offering significantly faster than you would if you were to try to grow organically.

Having said that, there is a significant advantage to being the first. As we have seen in the mobile application world, others who have tried to build a similar marketplace to that of Google and Apple are facing significant difficulties in attracting partners to their marketplaces (See Microsoft Surface as an example).

The main reason for these challenges is the Chicken vs. the Egg problem. In other words, for any marketplace to work, two essential factors must be in place: suppliers and consumers. Apple and Google already had their consumer-base and, therefore, it was easier for them to attract suppliers. Microsoft, on the other hand, does not yet have the mobile consumer-base on its side and is therefore facing a hard battle to bring in the suppliers.

A Cloud Application Marketplace

A Cloud Application Marketplace is similar in concept to that of mobile applications, but aims at cloud data-center services such as Database services, Application Containers, Messaging, CMS, CRM etc which are deployed on a cloud.

Amazon Cloud Marketplace

The marketplace concept is nothing new to Amazon and is already running a successful marketplace for its eCommerce business.

It only made sense for Amazon to latch on that model and use it as a basis for building its Cloud Application Marketplace, as described here.

The Limitations of Amazon Marketplace

While the promise of the a Cloud Marketplace as an ecosystem play is a no-brainer, an over-simplified implementation of it can yield a diminishing return as noted in Janakiram MSV article AWS Marketplace – An Oversimplification of the Cloud? Janakiram points specifically to the following limitations:

· Persistent Services Limitation:

By default, the database and other persistence-dependent services are confined to the ephemeral storage or instance storage of the EC2 instance. This is a very risky proposition as the data will be lost completely in the case of an instance failure. I wouldn’t be comfortable even running a non-mission-critical application like a WordPress blog on EC2 without attaching an EBS volume.

· Not Designed for Real Production:

Launching a LAMP or Tomcat stack with 1-Click is great for a small proof-of-concept or a demo, but no one would attempt this for a production application. Moving to the Cloud is not just about forklifting an app from on-premise server or a host to a VM. It is definitely much more than that! AWS Marketplace squarely focuses just on compute by launching an EC2 instance with pre-configured stack without considering any of the essential building block services like storage, networking and databases.

· Loadbalancer Limitations:

For example, the Amazon EC2 instances launched through the Marketplace cannot be added to an Elastic Loadbalancer. I attempted to launch 2 instances of the free Tomcat 6.0/MySQL Stack provided by OpenLogic and adding them to an Elastic Loadbalancer. ELB threw an error and refused to add the instances. This is a huge limitation for any real-world web application that intends to run on a Tomcat instance launched via the Marketplace. I wouldn’t be comfortable running my web application in just one instance without any possibility of scaling out in the future.

The Limitations of the VM-Centric Approach

Cloud applications are often packaged and run on VMs. It seems only natural that if applications are packaged on VMs, a marketplace would be a repository of pre-packed VMs, as in the case of the AWS marketplace.

Having said that, there are some basic flaws in this assumption - cloud applications and services are much more complex. Many of the services and applications need to run on high-availability mode, which often means that they need to run in more than one availability zone with inter-connectivity between them. In addition, many of the services, such as database services or web-services, need to run in a cluster deployment. In order to do so, multiple instances of VMs must be deployed. As real life applications tend to be multi-tier, there needs to be a way to deploy a set of VMs with some level of interdependency between them. When running an application on the cloud, we often need to think of it as a living organism that continues to change and evolve, rather than one-off deployment.

All these factors make the VM-centric approach a good trial and demo tool, but as Jankiram pointed out, a significant over-simplification of what cloud applications tend to be.

AWS Marketplace - Not a Fair Play

In order for a marketplace to be successful, it is essential that all players will get a fair shot to win consumers.

A marketplace owner in this context needs to take the role of "honest broker."

Amazon offers a long list of its own set of services as products outside the marketplace that don't face the limitations I listed above, in contrast to the applications that do run on the marketplace and are met with these challenges. This gives Amazon an unfair advantage over its own partners.

 

 

Building an Open Marketplace on OpenStack

The creation of an open marketplace would serve as an alternative to that of Amazon. A good substrate for that alternative approach would be to use open cloud frameworks like OpenStack. OpenStack already has a growing community of users and a growing support of cloud providers which provide an excellent starting point for the marketplace.

Below are some of the principles that I believe should serve as a foundation for building an open marketplace.

A Recipe-Based Marketplace

While the basic unit of deployment in many clouds tends to be based on VMs in many *real* deployments, the VM is used mostly as a generic container of the operating-system. The actual application services are deployed through a DevOps automation tool such as Chef, Puppet or Cloud Formation (Heat is the OpenStack alternative for CloudFormation on OpenStack) after the VM has been created. In this way, the deployment unit is the recipe rather than the VM. The recipe is used to describe the application services that will be deployed into the VM after it has been created. Unlike a VM, a recipe can be extended beyond the boundary of a VM and can describe fairly complex, multi-tier, clustered applications. It can also deal with the automation of post-deployment processes such as upgrades, monitoring, fail-over etc.

Based on that description, it only makes sense that we will use recipes as the unit of deployment instead of VMs. In this way, we can overcome many of the AWS marketplace limitations that are outlined above. As there is no standard recipe model as of yet, we'll need to be able to integrate with existing recipes such as Chef, Puppet and CloudFormation.

Other Considerations

Having a recipe-based marketplace is a good starting point. However, there is much more to building a marketplace than to choosing the recipe-based engine upon which to run an application. The main challenge is around the user experience and mastering the balancing act of keeping the marketplace attractive for both the consumers and suppliers.

Below are some of the key principles that I believe should be taken into consideration when building an open marketplace:

• Open source - An open source marketplace would enable different cloud providers to offer and share similar applications and services across different cloud providers.
• Popular frameworks – The marketplace should be based on popular open frameworks such as Chef, Puppet or CloudFormation/Heat templates.
• Reduce the barrier to entry - Make the investment of porting the application to the marketplace worth the effort beyond the marketplace. To reduce the barrier of porting applications into the marketplace we need to design it in such a way that we could use the same elements for deploying our application outside the marketplace.
• Support both multi-tier and cluster deployments
• Extensible - Users can pick a service from the marketplace and easily customize it to their needs.
• Fair play - Applications that are running through the marketplace should have the exact same capabilities as any application that is running outside of the marketplace.

If You Build It, They Will Come - Doesn't Work

These days I'm seeing many cloud providers trying to mimic Amazon’s success and launch their own marketplace. As in the case of Amazon, they also start with a VM-centric approach and thus, inherit many of its limitations. Furthermore, the main problem that I see with many of those attempts is that they seem to be taking the "if you build it, they will come" approach, not realizing that they don't have the advantage of being first and, more importantly, are not at the same point where Amazon was when it launched its marketplace ( i.e. with a huge consumer-base running on their platform).

As a result, I personally believe that many of those attempts are doomed to fail.

The real opportunity IMHO is to take a more open approach that could work outside the boundaries of a specific cloud provider. OpenStack could serve as a great starting point for forming this ecosystem.

To illustrate the points from this post on what an Open Marketplace could look like, in the next post I'll refer to how we use Cloudify as an open source framework for building your own cloud application marketplace on OpenStack.

Real-Time Big Data with Storm, Cassandra, and XAP

Real-time Big Data is becoming more and more common in Big Data systems.

One of the most common frameworks used for running real-time Big Data system is Twitter Storm backed by a NoSQL database, as shown in the diagram below:

 

In this architecture, Storm is used for processing data as it comes in and the NoSQL data store is used as the output for that processing as well as for reference data storage.

Challenges

This architecture presents a couple of challenges:

• Performance - Storm runs in-memory, and is therefore set to process large volumes of data at in-memory speed. However, a typical storm architecture needs to interface with a data source for its input and a data source for its output. In this context, the overall performance is now determined not by Storm itself but by the data input and output sources. Quite often these interfaces rely on file-based message queues for streaming, and NoSQL for data storage. These interfaces are at least an order of magnitude slower than Storm itself, and therefore become the limiting factor.
• Complexity - Storm itself consists of several moving parts, such as a coordinator  (ZooKeeper), state manager (Nimbus), and processing nodes (Supervisor). The NoSQL data store also comes with its own cluster management. In addition, a typical Big Data system comes with more components such as the application front end, a reporting database, and more. This makes the process of managing the deployment, configuration, fail-over and scaling of such a system quite complex.

Meeting the Performance Challenge

Given that Storm itself runs in memory, it only makes sense that in order for Storm to run at maximum capacity the streaming and data store interfaces should be implemented in-memory as well, as shown below:

As you can see in the above architecture, we added two interfaces that rely on a built-in Storm plug-in - one for the data inputs and the second for data output. In both cases, the underlying implementation is memory-based, and thus removing the impedance that the previous architecture included.

You can read the full details on how this integration works, including a code example, in DeWayne Filppi's post on integration of in-memory computing with Cassandra.

The second optimization that can be added is to the NoSQL data store.

As most analytics applications tend to be read-mostly, we can speed up access to the NoSQL data store using an in-memory local cache. This architecture speeds up read performance between 10 to 1000 times as outlined by Shay Hassidim in his post on real-time Big Data performance.

Meeting the Complexity Challenge

To meet the complexity challenge, we will use a DevOps automation approach using Cloudify in conjunction with Chef or Puppet.

With this approach, we wrap every service with a deployment recipe that abstracts the underlying details on how to manage Storm, Cassandra, and our in-memory data store. Cloudify uses these recipes to automate the deployment of the entire stack. In this way, you only need to interact with Cloudify for the deployment, configuration, and also scaling and fail-over of your stack rather than separately manage each individual component. In addition, we use Cloudify to abstract the underlying infrastructure. This enables us to use the same deployment recipe across different environments. One for testing, another for production, etc. We can also use the same approach to deploy our apps based on the type of workload. We can use a public cloud for running sporadic workload, and thus leverage the elasticity of the cloud and also enable us to create and scale the environment as well as rip it off completely when done. At the same time, we can choose bare-metal machines for I/O intensive workloads, and so on.

You can read more on how you can set Storm using Cloudify on DeWayne's post on Storm and the cloud.

Cloudify comes with a rich set of built-in recipes for other Big Data services as well, making the integration process an out of the box experience. The main ones are listed below:

Final Notes - Optimizing without Code Change

Many real-time Big Data systems are now based on Twitter Storm and a NoSQL data store such as Cassandra.

In this post I tried to outline how we can optimize this architecture by addressing two areas: performance, and management.

The good news is that all this is possible to achieve seamlessly, without any code change. Let me explain:

In the case of Storm we used built-in plug-ins - Spout for streaming and Trident as the data store interface. In this way we can simply plug our memory-based plug-ins under these two integration points. All our existing Storm business logic would work pretty much the same.

We use the same plug-in approach to integrate our in-memory data store with our NoSQL data. This integration makes the data flow between our real time streaming to our NoSQL storage fairly transparently. In addition, it allows us to plug in different NoSQL data stores such as MongoDB, CouchBase, etc., giving us another degree of flexibility.

The same applies to our management layer. Existing Storm and NoSQL deployments are wrapped into plugged in recipes, and don't require any specific code changes.

Not Mutually Exclusive

We don't have to implement all of the optimizations to gain the benefit of this architecture. Each of the optimization points can be plugged in  independently of the others. For  example, if your main pain point is complexity, you can start by only by adding the DevOps automation first. At the same time if your main pain point is performance, you can use the memory-based plug-ins to speed up processing.

The other advantage of the combined architecture that I haven't discussed in this post is that it provides more flexibility in the degrees of consistency and availability in which you set your system, as I outlined in one of my recent posts on the subject In Memory Computing (Data Grid) for Big Data.

References:

• Big Data Real-Time Performance
• GigaSpaces and Storm Part 2: XAP Integration
• GigaSpaces And Storm Part 1: Storm Clouds
• In Memory Computing (Data Grid) for Big Data
• Check out how GigaSpaces XAP and Cassandra integration works

Carrier Grade Cloud/PaaS

Cloud is becoming the next-generation backbone for running the web.

As the technology for building and running cloud infrastructure matures, it is starting to spread out into more industries and revolutionizing how even the most conservative organizations are running their entire operations.

One of the areas undergoing a transformation is the carrier backbone services. For those who are not familiar, carrier backbone services includes services like cell and network services (DHCP, DNS, …), content serving (SMS, MMS, …), activation services, CRM, call centers, etc. Moving these critical carrier services from the existing environment tends to be labor intensive and proprietary. Selecting to move applications into a more open and virtualized environment such as cloud could yield a significant cost saving. An open cloud environment also enables Carrier organizations to reduce their time to market for delivering new services.

At time where the Telco/Carrier market is increasingly competitive, moving to a cloud-based carrier backbone can be more than a cost saving initiative. It can be a differentiator from the competition and is critical for survivability and success of the business.

That said, running carrier grade services requires special care to meet the required SLAs in terms of  latency, deterministic behavior, performance, location awareness, etc. These challenges are unique enough not to fit into your mom-and-pop cloud.

The purpose of the Carrier Grade Cloud and Carrier Grade PaaS is to address these gaps and challenges.

In this post I'll try to provide a more detailed overview on what Carrier Grade Cloud and PaaS actually means. I will use examples based on GigaSpaces’ joint work with Alcatel. Alcatel-Lucent recently launched a new product in this space named CloudBand which is using Cloudify for its Carrier PaaS layer.

What does Carrier Grade mean?
Learning from the Weather Chanel experience during the Sandy super-storm

The Weather Channel’s experience during Sandy is an excellent example of the need for carrier grade services. Below is a list of some of the key statistics during Hurricane Sandy:

• 1000% - The Weather Company’s traffic increase during Hurricane Sandy
• 110 GB- The amount of data, served every second during Sandy
• 170,000 - Peak number of simultaneous streams of video served during Sandy
• 1 - The amount of data centers that went bust during the storm

To address this demand during the storm, the Weather Chanel was running from 13 Data Centers managed by Verizon across North America, all with load balancing between them. During the storm Verizon increased their bandwidth capacity to meet the peak demand.

This sort of increased traffic behavior wasn't unique to the weather channel, as noted below:

So what can be learned from this process? What makes a service Carrier Grade?

Learning from the Weather Channel experience we can define a Carrier Grade service as a service with the following attributes:

• Critical to the business function
• Designed for massive scale
• Designed to deal with major usage spikes
• Location sensitive
• Designed to provide deterministic response during extreme condition

This is obviously a fairly simplistic definition, but for the sake of this discussion I think it will suffice.

What Makes a Cloud/PaaS Carrier Grade?

There are various attributes that makes a Cloud/PaaS carrier grade, as I noted earlier. The two most important attributes IMO are the network and multi-site deployments. Let me explain why:

The Network

One of the main elements that is extremely important in in a Carrier Grade environment is the ability to assert control over the network.

That include the control over:

• Isolation
• Bandwidth
• Latency

Cross-Cloud/Data Center Deployments:

Another critical element of a successful Carrier environment is the multi-site deployment. As seen with the Weather Channel’s use of 13 sites, multi-site deployment is important to address continues availability and scaling. Optimizing the latency by surveying the content closer to the location of the end user also helps to deal with challenges of data delivery.

So how are things done today?

The current Carrier backbone runs on physical appliances which basically maps to lots of irons. In this environment scaling capacity means buying more appliances. While this model works, it has two main drawbacks:

1. cost (infrastructure/operation)
2. lack of agility (i.e. it takes months and sometimes years to launch new service in this environment).

Alcatel CloudBand -- Carrier Grade IaaS/PaaS

Alcatel CloudBand is a new platform  that let Telco apps easily leverage the carrier cloud services.

It is comprised of a few main elements.

Multi node/site IaaS -- a multi-site/Cloud infrastructure. The CloudBand infrastructure is essentially a policy based management on large numbers of cloud nodes. Each cloud node can run either an OpenStack or CloudStack-based infrastructure. These nodes can live in many disparate data centers. Alcatel CloudBand glues all of the disparate nodes together into a single big cloud that is accessible through an OpenStack API.

CPaaS -- Stands for Carrier Grade PaaS, which is essentially the framework enabling the on-boarding of the carrier services into the CloudBand infrastructure via a simple click and run user interface. Cloudify is integrated into this as an integral part of the CloudBand offering.

CloudBand's Unique Approach: Putting Network and Application Together

One of the unique aspects of the CloudBand architecture is its holistic approach to Network and Application. Standard cloud infrastructures tend to look at the two pieces as separate black boxes that run one on top of the other.

What does this new approach to Network and Applications really means?

Two example scenarios that I often use to describe the value of putting network and application together is in the areas of Disaster Recovery and Cloud Bursting. In today’s cloud, DR involves lots of wiring in which i need to explicitly point a segment of the application into a particular cloud zone and the other to another zone. Beyond the complexity of setting these zones up, it also means that there is a good degree of manual intervention required to handle a recovery or a scaling process in this environment.

Taking an automated SLA-driven approach to IaaS

Instead of identifying explicitly the zones in our cloud, with automated SLA we can simply ask the cloud to figure out the right zone for the job based on our application SLA. For example, a user could simply say something like "deploy RingTone service" where continuous availability=true redundancy=3 and distance between sites=100km". Most of that information is known to the CloudBand management at the time of deployment and it can therefore allocate machine instances not solely on image ID and zone ID, but also based on those SLA requirements.

Integrating the PaaS with the network

Many of the current PaaS solutions were designed to work with a simple cloud infrastructure.

If we design our PaaS solutions to work on top of a more intelligent infrastructure, like CloudBand, that can accept SLA-driven calls to coordinate infrastructure management, a revolution will happen. We can start looking at  offloading some of the responsibility for allocating the right machine instance to a particular application tier to the infrastructure. The infrastructure could be made aware that we're deploying a data service and would therefore ensure that the nodes of that database don't reside on the same physical machine or even the same data center. Another area where the responsibility could be delegated to the infrastructure is the network isolation. Instead of dealing with security groups, the system can attach a particular network for a given application or a tier within that application and the infrastructure will make sure that any machine that is allocated for this service would be attached to this network.

Final Words

For years there has been discussion on the missing piece in the cloud puzzle - the network. Today we're at a point where this gap is starting to be filled up by projects like Quantum in OpenStack. In addition to OpenStack the Telco industry is also launching a new initiative titled NFV which stand for Network Function Virtualization. NFV was born in October of 2012 when AT&T, BT, China Mobile, Deutsche Telekom and many other Telcos introduced the NFV Call to Action document. It basically aims to combine new network API with virtualization and thus provide a standard model for a virtulized Carrier Cloud.

While it seems that the industry is moving in the right direction toward the Virtualization of the backbone systems, most of the effort seem to be focused on standardisation at the lower level of stack. Very little has been done to draw the real end game i.e. how would an end to end Carrier backbone would look like given that new virtualized infrastructure in place. More importantly we haven't yet began to think of what would be the implication of that infrastructure change on the application and services ontop of it.

This is what excites me in the CloudBand project. CloudBand doesn't just end up with yet another fancy infrastructure piece that we don't necessarily know how and what do with it. It actually takes the holistic approach and maps those fancy features into a real end to end solution which at the most basic level maps to the fact that setting up data and network clusters, disaster recovery or cloud bursting scenarios can now be fully automated in a much simpler fashion than in most of the current cloud infrastructure environment.

At a more strategic level that means that Carrier can now rely on the the cloud as an infrastructure that could manage their backbone services and thus leverage the cloud economics to meet their cost and business challenges.

References

• Carrier Grade PaaS presentation by Uri Cohen
• Network Function Virtualization or NFV Explained
• Alcatel & Gigaspaces Joint partnership
• GigaSpaces Cloudify
• Alcatel Carrier Cloud

What is Cloudify? an End User Perspective

I just finished a great call with Joseph Weitekamp

Joe got introduced to Cloudify by one of my friends and shared with me his perspective on what is Cloudify. I realy liked the way he was able to put this togather and thought that the rest of you would be intrested in his perspective as well, so here it is..

What is Cloudify:

Cloudify is a software runtime lifecycle management system for cloud-hosted services.

-          It manages in an automated fashion:

• The provisioning of cloud infrastructure
• The installation and configuration of services on that infrastructure
• Monitors those services for health and parameters.  Respond to SLA violations.
• The uninstallation of service software and deprovisioning of cloud infrastructure
• Provide customized ways to interact with the running system that hide its cloudy deployment.

Summary

-          Cloud-provider agnostic through abstraction layer.   Public or private, ie, Amazon, Rackspace…

-          Recipe-based definition of deployments: end-state and means for achieving it

• Recipes are parameters + app lifecycle event handlers written in Groovy, a Python-like JVM-hosted language.
• Recipes for complex suites of services can be declared, which manages the coordinated configuration of these services.
• Service interdependencies declarable and respected.  Services don’t start until those they depend one are up.
• Recipe parameterization and inheritance (generalization / specialization) allow re-use and standardization

-          Custom Commands can be defined in Groovy and run interactively in a shell or web browser

-          Probes can be defined to surface telemetry on the services.  Standard probes for standard telemetry are provided.  Custom probe can be coded in Groovy or Java

-          Web Service Dashboard (meters and time-series graphs) can be declared to monitor service health/performance

-          Automated SLA Assurance can be declared to assure required number of service instances are always running.  Additional service instances are provisioned, installed and started as necessary

-          Auto-scaling based on telemetry can similar drive the provisioning, installation and startup / shutdown of service instances.

 

 

A Groovy Kind of Java

Today's application stack is a much more complex environment than the traditional JEE one. Moving from a Spring/JEE background to the cloud can be a complicated transition.

In talk given at the San Francisco Java Meetup Group i tried to outline how Groovy can help you map and translate an old JEE evironment to a new cloud-based one using  Cloudify in a way that anyone can adapt to their own needs. You can find the slides from the talk here.

Living with cloud outages as a way of life...and being ok with it

Learning from the experience of others has always been a great source for many of the posts in this blog.

I happened to have had a meeting with Ron Zavner, Applications Architect at GigaSpaces when he reviewed his planned presentation for the AWS Meetup this week in London.

I thought that the information that Ron gathered for this purpose could be extremely beneficial for everyone who is either running, or plans to run their application in the cloud.

In this post I tried to capture in words, the content of Ron’s presentation on the lessons from recent cloud outages.

Recap - History of Cloud Outages

• 21 April 2011 - Some parts of Amazon Web Services suffered a major outage. A portion of volumes utilizing the Elastic Block Store (EBS) service became "stuck" and were unable to fulfill read/write requests. It took at least two days for service to be fully restored. Reddit, one of the better-known sites to go down due to the error, said it has 700 EBS volumes with Amazon. Sites like Quora and Reddit were able to come back online in "read-only" mode, but users couldn't post new content for many hours.
• 29 June 2012 - Several websites that rely on Amazon Web Services were taken offline due to a severe storm of historic proportions in the Northern Virginia area where Amazon's largest data center is located.
• 22 October 2012 - A major outage occurred, affecting many sites, again  such as Reddit, Foursquare, Pinterest, and others. The cause was a latent bug in an operational data collection agent.
• Christmas Eve 2012 - Amazon AWS again suffered an outage, causing websites such as Netflix instant video to be unavailable for some customers, particularly in the Northeastern US. Amazon later issued a statement detailing the issues with the Elastic Load Balancing service that led up to the outage.

Cloud outages are not the sole property of Amazon – they’re everywhere

While most of the more notable failure events happened to be related to Amazon AWS, failures tend be in direct proportion to the usage of the infrastructure, and right now Amazon is probably running the biggest workloads on their cloud, and is growing pretty fast. With these statistics, it’s very likely that their failure would be more notable than others simply because they simply have a wider impact.

As we have experienced, failure has dropped a visit to other cloud infrastructure providers as well.

Microsoft Azure outage

• 28 December 2012 -  some owners of Microsoft's XBox 360 gaming console were unable to access some of their cloud-based storage files.
• 26 July 2012 - Service for Microsoft’s Windows Azure Europe region went down for more than two hours
• 29 February 2012 - The ultimate result was service impacts of 8-10 hours for users of Azure data centers in Dublin, Ireland, Chicago, and San Antonio.

Main take way

Looking at all of these failures, it becomes apparent that they don't quite follow a common pattern.

Failure tends to happens when and where you least expect it.

Rather than relying on the infrastructure to prevent such failure from happening, we need to learn how to cope with them as a way of living.

What does 99% availability mean anyway?

Quite often when we talk about availability, we’re referring to % of uptime.

In this context, for most people 99% uptime sounds good enough. Is it?

Let's examine what that means in days:

• 99% - 3.65 days downtime
• 99.9% - 8.76 hours downtime
• 99.99% - 53 minutes downtime
• 99.999% - 5.26 minutes downtime

99% downtime means that we need to be ready to tolerate almost 4 days of downtime, and no one can ensure how those 4 days will be spread across the year.

The impact of cloud outages

Although AWS went offline for a few hours only, the downtime experience did have an impact on customers’ businesses. There is no known data for the number of people affected by a cloud computing service outage. It is estimated that the travel service provider Amadeus loses $89,000 per hour during any cloud computing outage, while Paypal loses around $225,000 per hour.

How to survive cloud outages - (Lessons from RightScale & Netflix)

A good source for learning how to survive cloud outages is Netflix and RightScale who have had a good track record for surviving many of the previously mentioned cloud outages.

Below is a summary of the main takeaways.

• Make sure to have a dedicated expert to manage your disaster recovery (DR) architecture, processes and testing.
• Define what your target recovery time and recovery point is.
• Be pessimistic and design for failures – (assume everything will fail and design a solution that is capable of handling it).
• Avoid single points of failures – all parts of your app should be highly available (different AZ / regions / cloud) – load balancers, app servers, web servers, message bus, database.
• Use monitoring and alerts for failover processes and for every change in state.
• Document your DR operational processes and automations.
• Try to “break” different parts in your application. From unplugging the network, to turning off machines…then try it again.

Netflix didn't just provided its share of advice but has started to open source many of the tools it uses internally. The first one of them is  ”Chaos Monkey.”  This tool was designed to purposely cause failure in order to increase the resiliency of an application in Amazon Web Services (AWS.)

Designing your application to survive failure – It’s all about redundancy

Netflix provides an excellent toolset for surviving outages on Amazon on the operational level.

In this section, I wanted to zoom-in more on the design implications on our application.

The core principle for surviving failure is actually fairly simple, and in fact, applies to any system - not just cloud -  whether they happen to be airplanes, missiles, cars. At the end of the day, it’s all about redundancy. The degree of tolerance is often determined by how many alternate systems or parts of the system we have in our design, and how separate they are from one another. The degree of tolerance is also determined by how fast we can detect the broken part in our system and make the switch.

In software terms, the common parts that comprise our system are built out of two main groups - the business logic and the data.

Making a redundant software application that can survive failure is often based on setting up clones for two of those  parts of our system.

Cloning your data

There are two models for creating redundant data systems.

     1. Use database replication

     2. Use generic replication as in the case of CDN

For database replication - each database tends to have a different replication scheme.  Amazon RDS is based on a read replica that can take over when the master node fails. More modern databases, such as Cassandra and MongoDB, tend to have more flexibility and control for setting up the database replication. So the first choice that you need to make for setting up your data redundancy, is choosing the right database.

Quite often, using database replication tends to be good enough within a certain geography but can be too fragile for geo-redundancy. A model that has proven itself for replicating data across WAN is CDN.

What’s more, CDN is not tied to a specific data source, and therefore can use a generic service for replicating data from multiple sources that doesn't necessarily reside within a certain database.

Having said that, CDN is fit mostly to read data and doesn't ensure the consistency of the data. For that purpose, we need to use a generic replication service that could plug into any data source and replicate it to other locations in a way that will allow us to control the replication route, the latency, and consistency.

Cloning your application

To clone our application business logic, we need to be able to ensure that all parts of our system run the exact same version of all of our software components. That includes not just the binaries, but also the configuration, the scripts that run our application, and more importantly that all our post-deployment procedures such as failover, scaling and monitoring are also kept consistent.

Quite often the things that makes the cloning of our business logic complex, are due to the fact that the information on how to run our application is often scattered across many different sources such as scripts, as well as the mind of the people that runs these apps.

To make the job of cloning our application much simpler, and thus more consistent we need to be able to capture all parts of the information for running our apps in the same place.

Configuration management tools such as Chef, Puppet and in the case of Amazon CloudFormation can help in this regard.

Making it simpler through Cloudify

To make the work of setting all this up simpler, we tried to bake all those patterns into ready-made tools scripted into out of the box recipes.

Cloudify recipes include:

• Database cluster recipes with support for MySQL, MongoDB, Cassandra, Postgress...
• Integration with Chef and Puppet
• Automation of failover, scaling and continuous maintenance of our application.
• Application recipes that allow you to capture all the aspect of running your application, including the post-deployment aspects such as failover, scaling and monitoring.

This is a good example for setting a redundant web application between sites using Cloudify.

Final words

Cloud brings lots of promise for making our business more agile.

Cloud has also become a huge shared infrastructure in which every failure has a much more significant impact on our business worldwide.

The experience in the past year has taught us that even a robust cloud infrastructure such as Amazon can fail.

Through this experience we've learned that rather than relying on the infrastructure for preventing failure we need to design our system to cope with failure and get used to failure as away of life.

Having said that, the investment required to build a robust application can be fairly large and not something that everyone can afford.

Using tools like Cloudify, Chef, Puppet and if you’re a pure Amazon shop Netflix  could help to greatly reduce this effort by pre-baking a lot of these patterns into recipes.

 

References

• Cloudifying High Availability
• AWS Outage - Moving from Multi-Availability-Zone to Multi-Cloud
• Lessons From The Heroku Amazon Outage

 

 

Follow Nati on Twitter!

2013 The year of a major cloud world shakeout

Cloud has been around for couple of years now. Following the technology lifecycle, you would expect that we should now be entering the maturity lifecycle where we can expect to see less disruption and more consolidation in the market.

The reality shows that quite the contrary is happening, as we enter the land grabbing stage where everyone is competing for market share, and the incentive for major disruption is higher than ever before.

As a result of this, I expect during 2013 to see some major disruptions and shakeouts that will have a dramatic effect on the cloud world as we know it.

I've listed the main three below:

1. The IaaS Shakeout - So far, the IaaS world has been dominated by Amazon who is by far the leader in the market, where Rackspace its closest competitor comes in a far second. In 2013, we’re expecting to see Google entering the game, and Microsoft – who is not new to this game – to now finally start making the right moves, and with these is expected grab more market share from its existing customer base.  The other big disruption is the open source coalition behind OpenStack which was lacking the maturity needed to present a true viable alternative. With Rackspace, HP GA-ing its OpenStack-based cloud offering, and IBM  expected to follow suit as well, and with real production references such as Cisco’s Webex, I expect that we will now be seeing a far greater adoption of OpenStack, which to me currently looks like the main viable alternative to Amazon. The war on who rules the cloud is not going to be just a pricing war as some predict. One of the areas where there is huge potential for disruption is the network, as well as the software-based data center, which is a completely uncharted territory, with lots of potential for new innovation with how applications utilize the infrastructure.

2. The PaaS Shakeout - I've seen lots of predictions that talk about 2013 as the year of PaaS. They miss one major thing though. The definition of PaaS is undergoing a major disruption as well. PaaS was built initially to serve simple applications over the web. We’re now seeing a completely new category of PaaS coming mostly from open source initiatives, such as Cloud Foundry, which makes it possible to run PaaS in your local data center or even desktop. I'm also expecting us to see more convergence between PaaS and configuration management frameworks such as Chef, as in the case of Cloudify. If we follow Amazon, one can also note that the line between IaaS and PaaS is blurring, as Amazon has started to move up the stack. In addition to this, open source IaaS frameworks such as OpenStack and CloudStack make it possible to break down the wall between these two layers, and share a large part of the infrastructure between the IaaS and PaaS stack, blurring the boundaries between the two layers more than ever, as I pointed out in this post (PaaS as Infrastructure).

3. Big Data Shakeout - The Big Data world has primarily been dominated by Hadoop. The Hadoop project has since gone through major fragmentation with many different and incompatible versions of the Hadoop distribution such as the one coming from MapR, Horton Works, EMC, IBM, and more.  This gives a rise to some of the NoSQL vendors such as DataStax, the company behind Cassandra, and 10Gen the company behind MongoDB to come out with their own alternatives in this space. In addition, the move to real time makes the use of Map/Reduce less popular than it used to be. Frameworks like Twitter’s Storm and other stream processing frameworks are gaining popularity. With all this going on, the future of Haddoop as we know it remains fairly uncertain IMO, and while the Big Data market is expected to grow quite substantially over the next coming years, it is still unclear which technology will be the one to drive this change. One thing that has becomes quite clear though, is that it doesn’t appear as if there is going to be one solution that covers them all, making the challenge of integration and management of the various framework more critical than before.

So all in all, I think that 2013 is going to hold a lot of surprises and disruptions, and we are still pretty far from the point where we’re going to see consolidation.

PaaS as Infrastructure

The blurring line between PaaS and IaaS 

The core of IaaS services is centered around  Compute, Storage Network however over the past years the infrastructure moved up the stack and includes more application services such as Database as a Service, Hadoop Map/Reduce, and application lifecycle services. 

On the other hand, PaaS services offer similar services through the PaaS platform itself with the main difference being that quite often those services were designed to fit only the application that runs on that particular PaaS.

So rather than the traditional view, where we draw a clear line between the IaaS and PaaS layer we need to view them differently, where the PaaS and IaaS layers overlap each other on the software services layer.

 

IaaS and PaaS share a similar technology stack

A close look into the OpenStack infrastructure shows that even on the infrastructure level running the compute, storage and network services relies on messaging services such as RabbitMQ, databases such as Postgress, a load balancer such as Nginx and other software services. Running these software services for our infrastructure is not much different than running our apps.  With both the infrastructure and application we would need to address the scalability and high availabilityof these services and manage them through our infrastructure.

If that is the case wouldn't it make sense to share the same underlying technology to run our infrastructure and our apps?

Sharing the software services between your IaaS and PaaS

A typical public PaaS platform is built out of two main parts - a self-service portal, and a backend system that runs our apps.

With many of the public PaaS solutions such as GAE, Heroku the backend that runs our PaaS is treated as a blackbox.

Amazon Elastic Beanstalk does things differently in that respect - as it took a bottom-up approach. It uses the same AWS infrastructure to run Elastic Beanstalk and the infrastructure - this provides a lot of flexibility to its users, and equally as important, consistency between the way they run and manage their app and infrastructure. VMware took even a bigger step in this regard with CloudFoundry and BOSH and made their entire PaaS backend open source which gives users even greater flexibility. With Cloudify we took this idea a step further by introducing the concept of recipes into the core of the Cloudify infrastructure, and recently introduced our integration with Chef in a way that will enable users to run their Chef cookbooks as part of Cloudify. Chef has already gained popularity as a configuration and automation framework for setting up many of the OpenStack cloud deployments. Having Chef tightly integrated into the PaaS layer, simply put, makes perfect sense.

Example - Building your own RDS with Cloudify

To illustrate the idea of how we can use share the same deployment framework between our IaaS and PaaS layer I'll pick one of these software services to demonstrate the idea with. In this particular case, I've chosen to use a database as a service. Database as a service is often a good reference for a service that fits in both our infrastructure and application. Here is how we can easily set a database as a service with Cloudify:

1. Directly through the use of Cloudify recipes

If you allready developed your scripts for setting your database you can use cloudify recipe to wrap those script into a service. 

 Here a snippet of how that would look like:

service {
	name "mysql"
	icon "mysql.png"
	type "DATABASE"
	elastic true
	numInstances 1
	minAllowedInstances 1
	maxAllowedInstances 3	
	compute {
		template "SMALL_LINUX"
	}	
	lifecycle{
 
		install "mysql_install.groovy"
		start "mysql_start.groovy"
		startDetectionTimeoutSecs 900
		startDetection "mysql_startDetection.groovy"
		stopDetection {	
			!ServiceUtils.isPortOccupied(jdbcPort)
		}
        }	

You can refer to the full MySQL recipe here.

2. Through the use of Chef cookbooks

If you are already using Chef you can plug-in to the Chef MySQL cookbook by extending the Cloudify recipe to use Chef, as you can see in the snippet below:

service {
    extend "../../../services/chef"
    name "mysql-vanilla"
    type "DATABASE"
    numInstances 1
…
}

In addition to that you could attach Chef attributes and run parameters into the Cloudify recipe that will tell Chef which cookbook and recipe to run:

runParams = [
  "mysql": [ "bind_address": "0.0.0.0",
             "tunable": ["query_cache_limit": "2M"]  
  ],
  "run_list": ["recipe[mysql::server]"]]

You can read the full details on how to run Chef with Cloudify in our documentation.

Running through bare metal environment

Even in the case where we share the same software services through our infrastructure and application, we tend to run them differently.

Quite often infrastructure services will run on bare-metal environments, as we don't want to create dependencies between our virtualiation layer and the underlying infrastructure that runs it.  

To enable this level of flexibility, we need to abstract the underlying compute layer from our service orchestration layer. In this way, we can use the same recipe and deploy it on either a bare-metal environment or a virtaulization layer. (Read: Configuring a Traditional Data Center [BYON])

Final notes

Up until recently, PaaS and IaaS were treated as two separate and closed entities and the integration between the two was done pretty much in a blackbox approach. The fact that both the IaaS and PaaS underlying infrastructure were closed was the primary reason behind this.

Now with the growing adoption of open source IaaS such as OpenStack and CloudStack, and similarly with PaaS such CloudFoundry and Cloudify we have the opportunity to change this paradigm. There isn't much of a reason why we shouldn't be able to share the same orchestration, and configuration layers between our IaaS and PaaS layers. Amazon Elastic Beanstalk is a good reference why this makes a lot of sense.

Special thanks

Special thanks to Boris Renski who was the main inspiration behind this post.

References:

• Presentation on the Chef Meetup Group
• Online presentation