Nati Shalom's Blog

Do I Need PaaS if I Use Docker?

In my recent post on OpenStack  “Do I Need OpenStack if I Use Docker?” I covered the confusion that has emerged around one area of disruption that Docker has created. In this post, I’d like to segue into another similar topic that has emerged around the same concept where I sense similar disruption and confusion: the use of PaaS in its traditional form vs using a straight Docker approach.

Background

PaaS in its simplest form is a platform that enables developers to develop and deploy web applications into production fast by abstracting many details of the underlying infrastructure, whereas Docker provides a generic container for packaging software stacks in a portable way.

A Traditional PaaS Stack

Many of the traditional PaaS implementations use Docker or LXC as the underlying container to enable multi-tenant applications on a shared infrastructure. Usually PaaS relies on web containers that are specific to a certain language (Tomcat, Jetty/Java, modephp/PHP, Rails/Ruby, Django/Python,NodeJS/JavaScript…..) to host the user’s code, and Docker or LXC containers as lightweight VMs that hold these web containers, and provides the required isolation level between the various web containers. This architecture lends itself nicely into simple web applications which are often built out of a web front-end, and a database as a backend persistent model. As far as I know, Docker actually initially emerged from one of the PaaS’ provided by DotCloud, and evolved into its current incarnation.

The introduction of Docker as a simple abstraction to containers, has turned Docker into a portable software container that isn’t tied to a specific language or framework. It can not only be used as a web container, but also as a more general purpose software container for all sorts of applications such as microservices, big data analytics, stream processing, as well as legacy applications.

Some PaaS implementations, such as CloudFoundry, have already started to offer support for running Docker containers as one of the software components that are supported by the platform.

Having said that, PaaS provides a fairly opinionated model for the way we should model and package our applications. Often times, our packaging is too limiting as we start to grow to more scalable models, such as microservices.

This causes many users today to question whether there is enough added value in PaaS in order to justify these limitations, when users can simply run their software directly on Docker.

Using Docker as an Alternative to PaaS

Using Docker with a combination of orchestrations engines such as Kubernetes (in the case of GCE), Heat in the case of OpenStack, or Cloudify (for multiple clouds such as OpenStack, VMware, AWS, as well as bare metal)

can provide a relatively simple way to package, deploy and manage applications in an automated fashion.

Unlike the traditional PaaS stack - it doesn’t force a specific opinionated architecture for deploying apps although it does lend itself very nicely into both web applications and microservices architecture.

My Take

The main promise of PaaS is in the speed of development and simplicity. Traditional PaaS implementations achieve that goal through extreme abstraction of the underlying infrastructure, as well as by promoting an opinionated architecture, which provides a fairly optimized and consistent deployment experience for applications that fit into that model. The combination of the two ultimately does help achieve this promise, but for a fairly limited set of applications.

Using Docker coupled with orchestration as an alternative provides a more general purpose platform for achieving a similar goal. Unlike the traditional PaaS approach, this use of Docker with an orchestration engine does expose the user to more knobs and infrastructure details, which makes the user experience for developing and deploying simple web apps more complex than with the PaaS alternative, however it does provide the added benefit of flexibility - like with the onset of OpenStack, and open source cloud in general..

The right way to think of PaaS and Docker

In my opinion, the right way to think of the two is in a layered approach in which both the PaaS containers and Docker fit together as outlined in the diagram below.

As seen in the diagram above,  PaaS becomes one use case for deploying simple web apps. For more complex application deployments, it makes more sense to use Docker directly. In this context orchestration becomes a central piece for managing both use cases. Orchestration also provides a layer of abstraction between the application components and the underlying infrastructure.

The important realization is that in this context the role of PaaS as we know it becomes more of a niche role, yet still remains a fairly common and important use case. It is also important to note that the two models are not mutually exclusive. An application that runs on a PaaS container could interact with services that are managed directly through a Docker container, just as you would use other cloud services such as database as a service, load balancer as a service or any other on-demand resource.

Another important note is that the combination of Docker and orchestration opens a new path to manage dynamic workloads not just on an IaaS environment, but also directly on bare metal. I’m starting to see more use cases where this option becomes more popular, specifically for managing dev and testing environments where the use of a full blown cloud infrastructure may just  be overkill.

Final Note

When we started the journey into the cloud world a few years ago, the underlying cloud infrastructure was very complicated. As a result, the only way possible to simplify the application deployment experience was to take extreme measures in the level of abstraction, as well as by forcing an opinionated architecture to ensure that the application fits into typical cloud architecture. As the underlying infrastructure and tooling evolves,  those extreme measures are no longer needed for all use cases. What we’re now seeing is basically a shift of of pieces in the PaaS architecture that was previously embedded within the "PaaS Box" such as containers and orchestration that have becomes more generic, along with independent services that have their own right to exist.

The good news in all this, is that as users we are finally beginning to receive the simplicity of PaaS, that was previously limited only to greenfield web applications, for use with a  much broader spectrum of applications and architecture styles.

So to answer the question from the opening note, do i need PaaS if i use Docker? The answer is, yes. It makes sense to have the two combined, but the role of PaaS as we know it has narrowed down significantly.

Our Open Source Cloud (Re) Architecture with OpenStack + TOSCA

Cloudify OpenStack Native approach - The Story behind the Scenes

In the previous post I outlined what I mean by native-approach to the OpenStack cloud. In a nutshell, a native approach represents out of the box tighter integration with OpenStack, all without being limited only to OpenStack.

In this post I want to share some of the stories behind the design of Cloudify 3.0 and the approach that we have taken to make Cloudify *native* to OpenStack.  I would first start by outlining our motivation behind this approach.

OpenStack-only or Not-only-OpenStack?

It is our core belief that for the foreseeable future supporting hybrid cloud is going to be important for many of the OpenStack users for the following reasons:

1. Transitioning to OpenStack is a long journey - Many enterprises have VMware-based stacks as their core infrastructure. Where many of these organizations have often started their journey toward OpenStack by creating a dev environment, and then gradually moving more workloads into their OpenStack environment as they get more comfortable with it. This process usually spans over months and possibly even years. It became apparent to us, that during this time these customers would still be using VSphere and VCloud in parallel to OpenStack. For such users, having a common platform to manage their app deployment across two environments can smoothen the transition and also reduce the risk of getting stacked if one of the environments doesn’t meet their needs.
2. The public cloud market is going to be dominated by non-OpenStack clouds - while OpenStack is on a trajectory to dominate the private cloud market, the public cloud market looks quite different with AWS as a strong and long-standing leader and other major clouds emerging such as GCE and Azure - hoping to close in. Many organizations wouldn’t want to host their entire data center on a private cloud, and would therefore need to take a hybrid cloud approach. In order to support hybrid cloud, we had to include an abstraction layer that will allow us to integrate with clouds other than OpenStack.

At the same time we wanted to have a more intimate integration with OpenStack in order to leverage the fact that it is open source and allows for a deeper integration with its core services. Through that we were able to enable smooth and simple integration of Cloudify into OpenStack.

Extending or Rewriting?

Cloudify 2.x already took a step toward deeper integration with OpenStack and we were one of the first to announce our support for OpenStack. Recently, we also added support for OpenStack Neutron.

We realized that if we want to make Cloudify fit natively into the OpenStack infrastructure it is not enough to have integration with the OpenStack components.  We also need to ensure that the design behind Cloudify will be consistent with the way other projects and services of OpenStack are implemented. This led us to the conclusion that we wouldn’t be able to be a first class citizen with OpenStack unless we underwent a major redesign of Cloudify.

Going Python?

Our first step in this direction was to move from Java to Python. Moving to Python wasn’t as painful as you would think for us, because many of our developers already knew the language. This decision quickly proved itself, as it became much simpler not just to develop the new version of Cloudify faster, but it also enabled us to join other OpenStack projects as well as reuse some of the same components and frameworks that are already used by OpenStack.

Integrating with Core OpenStack Services

Many of the products that declare support for OpenStack mostly integrate with the compute API (Nova), and quite often come with their own security, messaging, configuration, and other such considerations. We felt that this is not enough, and doesn’t fit in with the way other services are built within OpenStack. Hence, we decided to deepen our integration with other services such as Keystone, Neutron, Heat and later with Ceilometer, Mistral and others.

Going TOSCA?

Cloudify 2.x was designed around a proprietary Groovy-based domain specific language (DSL) for defining application topology and configuration. It became apparent that the DSL had become a central piece of Cloudify architecture. As a result, maintaining a proprietary DSL would only serve as a barrier going forward. TOSCA (Topology Orchestration Specification for Cloud Applications led by the Oasis Foundation) defines a standard templating language. TOSCA was similar and richer in concept to our previous 2.x specification. Because of this, using TOSCA seemed like a natural evolution.

At the time when we decided to use TOSCA, the specification was provided in a fairly complex XML. Our first step for adopting TOSCA was to come with a simplified YAML based version which better fits into the spirit of OpenStack. After the OpenStack summit in Hong Kong, when we first introduced the idea, it became apparent that members from the Oasis group were already thinking along this line.We decided to join forces and join the Oasis organization. We started working with the team to incorporate TOSCA as an official part of OpenStack. During the Atlanta summit, it was decided to integrate the TOSCA project into the Heat project and now there is an official definition of the TOSCA 2.0 specification based on YAML. Our goal is to use the TOSCA 2.0 specification as the official templating language for Cloudify instead of our current TOSCA like DSL.

Integrating with OpenStack Heat

OpenStack is growing, and with this the services that are included as part of the core OpenStack project continue to grow up the stack. One of these services is OpenStack Heat, which started as the equivalent of Amazon Cloud Formation, and is becoming more of a general purpose infrastructure orchestration these days.

We realized that since Heat is closely integrated with OpenStack it will follow the OpenStack release cycles and API, and therefore would provide a useful tool for setting up the OpenStack infrastructure. The approach that we have taken with Heat is an “overlay approach”. This means that users can use Heat as they do today to set up their infrastructure. Then, Cloudify integrates with Heat in a way that will allow it to discover any resource that was provisioned through Heat, and ultimately add the monitoring, logging and software stack on top of that environment. 

Integrating with Neutron - Network Orchestration

Networking becomes a core service in any cloud deployment. Networking refers to an element such as security-group, private IP, floating IP as well as routers, DNS, vLans, load balancers, and such.

Cloudify 2.7 included basic integration with Neutron that allows users to attach floating IPs and set the availability zone configuration.

With Cloduify 3.0, we included support for all the networking elements and can now create vLans, security groups and more, as part of the application deployment.

Design for Scale

Application orchestration often requires intimate and continuous integration with the application so that it can detect failure in sub-seconds and take corrective actions as needed. This often imposes a more complex scalability challenge.

Cloudify 2.x uses a hierarchy of managers, where each manager controls 100+ nodes as a scaling architecture. One of the things that we have experienced over the last year is that there is a growing class of services such as  Big Data and NFV. With these services, a single service can be comprised out of more than 100s and potentially even 1000s of instances. This is why the manager of managers approach didn’t fit well to these use cases.

To enable scaling of 1000s of nodes per manager, we decided to use a message broker approach based on AMQP as well as to separate the provisioning, logging and real-time monitoring tasks into separate services that can scale independently. This allows us to also control the level of intimacy between the Cloudify manager and the application services. As an example, we can tune Cloudify to handle the provisioning and logging only, and use the real-time information that has already been gathered by the infrastructure. This is especially important in the context of OpenStack, as we expect that a large part of the metrics can be gathered through the infrastructure itself.

Integrating your DevOps toolchain of choice

One thing that has become clear through the various OpenStack statistics based on their annual survey, is that many of the OpenStack users are using a toolchain that is comprised of tools available through OpenStack infrastructure, as well as tools that are not specific to OpenStack such as Docker, Chef, Puppet, Ansible, JClouds and others.

Integrating all these tools as part of a deployment system becomes a fairly complex task that can takes weeks and even months.

We realized that if we want to provide a full lifecycle management of the application, it wouldn’t be right to force a particular stack or toolchain on a user, but rather provide an open pluggable architecture that could enable you to easily integrate your choice of tools into the same deployment.

With Cloudify 2.x we included support for Chef, Puppet and also had a cloud driver that allows us to integrate with various cloud infrastructures, as well as with tools like JClouds.

With Cloudify 3.x we created a more generic plugin architecture that allows users to plug in almost any element of their application, starting from the cloud plugin to configuration management, orchestration, monitoring etc.

To make this simple we included a Bash plugin that provides a simple framework for integration with external tools, just by pointing the plugin to the relevant Bash script.

Final Notes

In this post I thought sharing the story behind the scenes that we went through to make Cloudify OpenStack native, and the rationale and decision process behind them, would be relevant to other users that are considering their OpenStack strategy.  Especially for those still deciding how deep they should integrate with OpenStack.

The good news is that, it is quite possible to integrate natively with OpenStack without limiting yourself only to OpenStack. Having said that, the initial investment required to get there is quite big. I hope that this lesson from our own experience will save others time.

The next big thing in big data: fast data

The big data movement was pretty much driven by the demand for scale in velocity, volume, and variety of data. Those three vectors led to the emergence of a new generation of distributed data management platforms such as Hadoop for batch processing and NoSQL databases for interactive data access. Both were inspired by their respective predecessors Google (Hadoop, BigTable) and Amazon (Dynamo DB).

As we move to fast data, there’s more emphasis on processing big data at speed. Getting speed without compromising on scale pushes the limits in the way most of the existing big data solutions work and drives new models and technologies for breaking the current speed boundaries. New advancements on the hardware infrastructure with new flash drives offer great potential for breaking the current speed limit which is bounded mostly by the performance of hard drive devices.

Why using existing RDBMS, NoSQL on top of flash drive is not enough

Many of the existing databases — including more modern solutions such NoSQL and NewSQL — were designed to utilize the standard hard drive devices (HDD). These databases were designed with the assumption that disk access is slow and therefore they use many algorithms such as Bloom filters to save access to disk in case the data doesn’t exist. Another common algorithm is to use asynchronous write to commit-log. A good insight into all the optimizations that are often involved in a single write process on NoSQL databases is provided through the Cassandra Architecture. Let’s have a look at what that entails below:

 

The Cassandra write path

 

What happens when disk speed is no longer the bottleneck?

When the speed of disk is no longer the bottleneck, as in the case of flash devices, then a lot of the optimization turns into overhead. In other words, with flash devices it will be faster and simpler to access the flash device directly for every write or read operation.

Flash is not just a fast disk

Most of the existing use cases use flash devices as faster disk. Using flash as a fast disk was a short path to bypass the disk performance overhead without the need to change much of the software and applications. Having said that, this approach inherits many unnecessary disk driver overhead. So to exploit the full speed potential of flash devices, it would be best to access the flash devices directly from the application and treat flash devices as a key/value store rather than a disk drive.

Why can’t we simply optimize the existing databases?

When we reach a point in which we need to change much of the existing assumptions and architecture of the existing databases to take advantage of new technology and devices such as flash, that’s a clear sign that local optimization isn’t going to cut it. This calls for new disruption.

The next big thing in big data

Given the background above, I believe that in the same way that demand for big data led to the birth of the current generation of data-management systems, the drive for fast data will also lead to new kinds of data management systems. Unlike the current set of databases, I believe that the next generation databases will be written natively to flash and will use direct access to flash rather than the regular disk-based access. In addition to that, those databases will include high performance event streaming capabilities as part of their core API to allow processing of the data as it comes in, and thus allowing real-time data processing.

 

Fast data in the cloud

Many of the existing databases weren’t designed to run in cloud as first-class citizens and quite often require fairly complex setup to run well in a cloud environment.

As cloud infrastructure matures, we now have more options to run big data workload on the cloud. The next generation databases need to be designed to run as a service from the get-go.

To avoid the latency that is often associated with such a setup, the next generation databases will need to run as close as possible to the application. Assuming that many of the applications will run in one cloud or another, it means that those databases will need to have built-in support for different cloud environments. In addition, they will need to leverage dynamic code shipping to pass code with the data, and in this way allow complex processing with minimum network hops.

In-memory databases and data grid are the closest candidates to drive next-gen flash databases

RAM and flash-based devices have much more in common than flash and hard drive. In both RAM and flash, access time is fairly low and wasn’t really the bottleneck; in-memory databases use direct access to the RAM device through key value interface to store and index data.

Those factors makes in-memory based databases more likely to fit into the next generation flash databases.

The combination of in-memory databases and data grid on top of flash devices also allows the system to overcome some of the key capacity and cost per GB limitations of an in-memory based solution. The integration of the two will allow an increase in the capacity per single node to the limit of the underlying flash device rather than to the limit of the size of RAM.

The overall architecture of the integrated solution looks as follows (Source: Gartner):

 

 

As can be seen in the diagram above, the IMC (in-memory computing) layer acts as the front end to the flash device and handles the transactional data access and stream processing, while the flash device is used as an extension of the RAM device from the application perspective.

There are basically two modes of integration between the RAM and flash device.

LRU Mode – In this mode, we use the RAM as a caching layer to the flash device. The RAM device holds the “Hot,” i.e. the most recently used, and the flash device holds the entire set.

Pros: Optimized for maximum capacity.

Cons: Limited query to simple key/value access

Fast Index Mode – In this mode, all the indices in RAM and the data in flash holds the entire set.

Pros: Supports complex querying, including range queries and aggregated functions.

Cons: Capacity is limited to the size of indexes that can be held in RAM.

In both cases, the access to flash is done directly using key/value interface and not through disk drive interface. As both RAM and flash drives are fairly fast, it is simpler to write data to the flash drive synchronously and in this way avoid potential complexity due to inconsistency.

It is also quite common to sync the data from the IMC and flash devices into an external data store that runs on a traditional hard drive. This process is done asynchronously and in batches to minimize the performance overhead.

Direct flash access in real numbers

To put some real numbers behind these statements, I wanted to refer to one of the recent benchmarks that was done using an in-memory data grid based on GigaSpaces XAP and direct flash access using a key/value software API that allows direct access to various flash devices. The benchmark was conducted on several devices as well as on private and public cloud-based services on AWS. The benchmark shows a ten times increase in managed data without compromising on performance.

 

 

 

For full details of the benchmark, see the following white paper:

XAP MemoryXtend -Massive Application Storage Capacity for Real-Time Applications

 

 

 

OpenStack by the Numbers

One thing that I like about working with an open source community like OpenStack is that you get direct access not just to the source code itself, but to its users and ecosystem as well.  This allows you to be less exposed to marketing spins, and actually analyze our actions based on real numbers.

The OpenStack Foundation conducts a very insightful survey twice a year which helps to measure how users are using the technology. This is an important and useful feedback loop that goes to the PTL of the various projects, and can help product companies to analyze their go to market plans and such. You can find the source

I found the original version a bit too verbose, so I took the liberty to put together an executive summary of the main points from the survey, mostly targeted at decision makers. I hope that you’ll find it useful, any feedback would be highly appreciated.

Main Reasons for Choosing OpenStack

The main reason cited for choosing OpenStack is Open Technology and Avoiding Lock-in.

Interestingly enough Cost Saving comes only 4th!

 

Size of Deployments

The large majority of the current deployments are under 100 nodes.

Around 30% of the deployments run relatively large scale deployments between 500-1000.

QA/DEV	Production

Production seems to follow a similar trend but lower in numbers to QA, which is reasonable as I believe that some of the QA deployments represent a transition stage to production.

 

User Type

The majority of the users seem to be service providers and cloud operators.

There is also a good number of ISVs and ecosystem partners, which is a strong indication that OpenStack is indeed an ecosystem.

There's a very strong presence of OpenStack in the Telco industry. Finance seem to be lagging behind the adoption curve with only three financial organizations declaring their use of OpenStack.

The majority of the deployments run on private or hosted private clouds. About 20% run on public or hybrid clouds. OpenStack also seems to attract mostly small to mid-size companies.

 

Private/Public Clouds	Size of Organizations

 

Deployment Tools

The survey includes two categories for deployment tools, one for those that are used for deploying the OpenStack infrastructure, and the other that are used for application configuration (covered in the application survey).

According to the two surveys users tend to use different sets of tools for the infrastructure deployment and for application configuration management. Puppet seems to be the most popular choice for OpenStack users in both categories. 

One of the interesting results is to see that Bare Metal and Docker are starting to appear in more deployments. This is fairly impressive given that the two are not yet officially supported by OpenStack.  VMware is around 10%-13% of KVM of Virtualization similar to XEN.

 

OpenStack Deployment Tools

 

Applications on top of OpenStack

As noted above, Puppet is popular for application configuration, as well as infrastructure deployment. Heat and Docker take the 2nd and 3rd place respectively,  putting Chef behind in the 4th place.

Given the popularity of Heat during the last summit I wouldn’t be surprised if Heat and Docker will surpass the use of Puppet in this category.

 

 

Popular Frameworks

Interestingly enough, JClouds seems to be gaining popularity amongst OpenStack users. This may be an indication that ~ ¼ of the users are interested in frameworks that are not tied only OpenStack. Not surprisingly, Python is far more popular than any other language, however Java seems to have a good and dominant presence more than other frameworks. One possible explanation is that the type of users that are attracted to OpenStack are more of the traditional enterprise organizations, and less of the new web-facing companies.

Operating Systems

Ubuntu and CentOS are the most popular choice. Windows adoption amongst OpenStack users is between 10-20%.

<

Application Survey	Production Survey

 

Compatibility / Portability

EC2 is a relatively popular choice which indicates that there is an increasing desire among the OpenStack community to maintain portability with EC2. This also correlates with the popularity of portability frameworks such as JClouds.

 

QA/DevTest	Production Survey

 

References

• OpenStack user survey presentation
• OpenStack user survey online session from OSS Atlanta 2014

Facebook’s vs Twitter’s Approach to Real-Time Analytics

Last year, Twitter and Facebook have released new versions of their real-time analytics systems.

 In both cases, the motivation was relatively similar — they wanted to provide their customers with better insights on the performance and effectiveness of their marketing activities. Facebook’s measurement includes “likes” and “comments” to monitor interactions. For Twitter, the measurement is based on the effectiveness of a given tweet – typically called “Reach” – basically a measure of the number of followers that were exposed to the tweet. Beyond the initial exposure, you often want to measure the number of clicks on that tweet, which indicate the number of users who saw the tweet and also looked into its content.

Facebook’s vs Twitter’s Approach to real-time Analytics 

Facebook Real-Time Analytics Architecture – Logging-Centric Approach:

  Relies on Apache Hadoop framework for real-time and batch (map/reduce) processing. Using the same underlying system simplifies the maintenance of that system.

• Limited real-time processing — the logging-centric approach basically delegates most of the heavy lifting to the backend system. Performing even a fairly simple correlation beyond simple counting isn’t a trivial task.

• real-time is often measured in tens of seconds. In many analytics system, this order of magnitude is often more than enough to express a real-time view of what is going on in your system.

• It is suitable for simple processing. Because of the logging nature of the Facebook architecture, most of the heavy lifting of processing cannot be done in real-time and is often pushed into the backend system.

• Low parallelization — Hadoop systems do not give you ways to ensure ordering and consistency based on the data. Because of that, Facebook came up with their Puma service that collects and inputs data into a centralized service, thus making it easier to processes events in order.

• Facebook collects user click streams from your Facebook wall through an Ajax listener which, then, sends those events back into the Facebook data centers. The info is stored on Hadoop File System via Scribe and collected by PTail.

• Puma aggregates logs in-memory, batching them in windows of 1.5 seconds and stores the information in Hbase. 

• The Facebook approach puts a huge limit as to the volume of events that the system can handle and have significant implications over the utilization of the overall system.

 

Twitter Real-Time Analytics Architecture – Event-Driven Approach:

• Unlike Facebook, Twitter uses Hadoop for batch processing and Storm for real-time processing. Storm was designed to perform fairly complex aggregation of the data that comes through the stream as it flows into the system, before it is sent back to the batch system for further analysis.

• Real-time can be measured in milliseconds. While having second or millisecond latency is not crucial to the end user — it does have a significant effect on the overall processing time and the level of analysis that we can produce and push through the system. As many of those analyses involve thousands of operations to get to the actual result.

• It is suitable for complex processing. With Storm, it is possible to perform a large range of complex aggregation while the data flows through the system. This has a significant impact on the complexity of the processing. A good example is calculating trending words. With the event-driven approach, we can assume that we have the current state and just make the change to that state to update the list of trending words. In contrast, a batch system will have to read the entire set of words, re-calculate, and re-order the words for every update. This is why those operations are often done in long batches.

• Extremely parallel – Asynchronous events are, by definition, easier to parallelize. Storm was designed for extreme parallelization. Ultimately, it determines the speed level of utilization that we can get per machine in our system. Looking at the bigger picture, this quite substantially adds to the cost of our system and to our ability to perform complex analyses.

Final Words

 Quite often, we get caught in the technical details of these discussions and lose sight of what this all really means.

If all you are looking for is to collect data streams and simply update counters, then both approaches would work. The main difference between the two is felt in the level and complexity of processing that you would like to process in real-time. If you want to continuously update a different form of sorted lists or indexes, you’ll find that doing so in an event-driven approach, as is the case of Twitter, can be exponentially faster and more efficient than the logging-centric approach. To put some numbers behind that, Twitter reported that calculating the reach without Storm took 2 hours whereas Storm could do the same in less than a second.

Such a difference in speed and utilization have a direct correlation with the business bottom line, as it determines the level and depth of intelligence that it can run against its data. It also determines the cost of running the analytics systems and, in some cases, the availability of those systems. When the processing is slower there would be larger number of scenarios that could saturate the system.

References:

• Slides: real-time Big Data with Storm, Cassandra and In Memory Computing
• Video: Real-time Analytics for Big Data: A Facebook Case Study
• Video: C* Summit 2013: Real-Time Big Data with Storm, Cassandra, and In Memory Computing
• Nathan Marz (Twitter) on Storm

Real-Time Big Data with Storm, Cassandra, and XAP

Real-time Big Data is becoming more and more common in Big Data systems.

One of the most common frameworks used for running real-time Big Data system is Twitter Storm backed by a NoSQL database, as shown in the diagram below:

 

In this architecture, Storm is used for processing data as it comes in and the NoSQL data store is used as the output for that processing as well as for reference data storage.

Challenges

This architecture presents a couple of challenges:

• Performance - Storm runs in-memory, and is therefore set to process large volumes of data at in-memory speed. However, a typical storm architecture needs to interface with a data source for its input and a data source for its output. In this context, the overall performance is now determined not by Storm itself but by the data input and output sources. Quite often these interfaces rely on file-based message queues for streaming, and NoSQL for data storage. These interfaces are at least an order of magnitude slower than Storm itself, and therefore become the limiting factor.
• Complexity - Storm itself consists of several moving parts, such as a coordinator  (ZooKeeper), state manager (Nimbus), and processing nodes (Supervisor). The NoSQL data store also comes with its own cluster management. In addition, a typical Big Data system comes with more components such as the application front end, a reporting database, and more. This makes the process of managing the deployment, configuration, fail-over and scaling of such a system quite complex.

Meeting the Performance Challenge

Given that Storm itself runs in memory, it only makes sense that in order for Storm to run at maximum capacity the streaming and data store interfaces should be implemented in-memory as well, as shown below:

As you can see in the above architecture, we added two interfaces that rely on a built-in Storm plug-in - one for the data inputs and the second for data output. In both cases, the underlying implementation is memory-based, and thus removing the impedance that the previous architecture included.

You can read the full details on how this integration works, including a code example, in DeWayne Filppi's post on integration of in-memory computing with Cassandra.

The second optimization that can be added is to the NoSQL data store.

As most analytics applications tend to be read-mostly, we can speed up access to the NoSQL data store using an in-memory local cache. This architecture speeds up read performance between 10 to 1000 times as outlined by Shay Hassidim in his post on real-time Big Data performance.

Meeting the Complexity Challenge

To meet the complexity challenge, we will use a DevOps automation approach using Cloudify in conjunction with Chef or Puppet.

With this approach, we wrap every service with a deployment recipe that abstracts the underlying details on how to manage Storm, Cassandra, and our in-memory data store. Cloudify uses these recipes to automate the deployment of the entire stack. In this way, you only need to interact with Cloudify for the deployment, configuration, and also scaling and fail-over of your stack rather than separately manage each individual component. In addition, we use Cloudify to abstract the underlying infrastructure. This enables us to use the same deployment recipe across different environments. One for testing, another for production, etc. We can also use the same approach to deploy our apps based on the type of workload. We can use a public cloud for running sporadic workload, and thus leverage the elasticity of the cloud and also enable us to create and scale the environment as well as rip it off completely when done. At the same time, we can choose bare-metal machines for I/O intensive workloads, and so on.

You can read more on how you can set Storm using Cloudify on DeWayne's post on Storm and the cloud.

Cloudify comes with a rich set of built-in recipes for other Big Data services as well, making the integration process an out of the box experience. The main ones are listed below:

Final Notes - Optimizing without Code Change

Many real-time Big Data systems are now based on Twitter Storm and a NoSQL data store such as Cassandra.

In this post I tried to outline how we can optimize this architecture by addressing two areas: performance, and management.

The good news is that all this is possible to achieve seamlessly, without any code change. Let me explain:

In the case of Storm we used built-in plug-ins - Spout for streaming and Trident as the data store interface. In this way we can simply plug our memory-based plug-ins under these two integration points. All our existing Storm business logic would work pretty much the same.

We use the same plug-in approach to integrate our in-memory data store with our NoSQL data. This integration makes the data flow between our real time streaming to our NoSQL storage fairly transparently. In addition, it allows us to plug in different NoSQL data stores such as MongoDB, CouchBase, etc., giving us another degree of flexibility.

The same applies to our management layer. Existing Storm and NoSQL deployments are wrapped into plugged in recipes, and don't require any specific code changes.

Not Mutually Exclusive

We don't have to implement all of the optimizations to gain the benefit of this architecture. Each of the optimization points can be plugged in  independently of the others. For  example, if your main pain point is complexity, you can start by only by adding the DevOps automation first. At the same time if your main pain point is performance, you can use the memory-based plug-ins to speed up processing.

The other advantage of the combined architecture that I haven't discussed in this post is that it provides more flexibility in the degrees of consistency and availability in which you set your system, as I outlined in one of my recent posts on the subject In Memory Computing (Data Grid) for Big Data.

References:

• Big Data Real-Time Performance
• GigaSpaces and Storm Part 2: XAP Integration
• GigaSpaces And Storm Part 1: Storm Clouds
• In Memory Computing (Data Grid) for Big Data
• Check out how GigaSpaces XAP and Cassandra integration works

Real Time analytics for Big Data: Facebook's New Realtime Analytics System

Recently, I was reading Todd Hoff's write-up on FaceBook real time analytics system. As usual, Todd did an excellent job in summarizing this video from Engineering Manager at Facebook Alex Himel, Engineering Manager at Facebook.

In this first post, I’d like to summarize the case study, and consider some things that weren't mentioned in the summaries. This will lead to an architecture for building your own Realtime Time Analytics for Big-Data that might be easier to implement, using Facebook's experience as a starting point and guide as well as the experience gathered through a recent work with few of GigaSpaces customers. The second post provide a summary of that new approach as well as a pattern and a demo for building your own Real Time Analytics system..

The Business Drive for real time analytics: Time is money

The main drive for many of the innovations around realtime analytics has to do with competitiveness and cost, just as with most other advances.

For example, during the past few years financial organizations realized that inter-day risk analysis of their customers' portfolios translated to increased profit as they could react faster to profit and loss events.

The same applies to many of the online ecommerce and social sites. Knowing what your users are doing on your site in real time and matching what they do with more targeted information transforms into better conversion rate and better user satisfaction, which means more money in the end.

Todd provides similar reasoning to describe the motivation behind Facebook's new system:

Content producers can see what people like, which will enable content producers to generate more of what people like, which raises the content quality of the web, which gives users a better Facebook experience.

Why now?

Real time analytics goes mainstream

The massive transition to online and social applications makes it possible to track user patterns like never before. The correlation between the quality of data that providers track and their business success is closely related: for example, e-commerce customers want to know what their friends think about products or services, right in the middle of their shopping experience. If sites cannot keep up with their thousands of users in real-time, they can lose their customers to sites that can.

So while risk analytics in financial industry was still a fairly small niche of the analytics market the demand for real time analytics in Social, eCommrce and SaaS applications brought the demand for real time analytics to main-stream business under massive load.

No one has time for batch processing anymore.

Technology advancement

Newer infrastructures and technologies like tera-scale memory, nosql, parallel processing platforms, and cloud computing, provide new ways to process massive amount of data in shorter time and with lower cost.  As most of the current analytics systems weren’t built to take advantage of these new technologies and capabilities, they haven't been able to adopt to real time requirements without massive changes.

Hadoop Map/Reduce doesn’t fit the real time business

One of the hottest trends in the analytics space is the use of Hadoop as almost the de-facto standard for many of the batch processing analytics application. While Hadoop (and Map/Reduce in general) does a pretty good job in processing massive logs and data through parallel batch processing, it wasn’t designed to serve the real time part of the business.

Strong evidence for that can be seen from the moves of those who were known as the “poster children” for Map/Reduce: Google and Yahoo both moved away from Map/Reduce. Google has moved to  Google Percolator for its indexing service. Yahoo came-up with a new service S4 which was designed specifically for real time processing.

It is therefore not surprising that Facebook reached the same conclusion as it relate to Hadoop:

Hadoop/Hive..Not realtime. Many dependencies. Lots of points of failure. Complicated system. Not dependable enough to hit realtime goals.

 Facebook's Real Time Analytics system

According Todd,  Facebook evaluated a fairly long list of alternatives including as MySQL, an in-memory cache, Hadoop Hive/Map Reduce. I highly recommend reading the full details from Todd's post.

I tried to outline Facebook architecture based on Todd's summary in the diagram below:

Every user activity triggers an asynchronous event, through AJAX – this event is logged in a tail log using Scribe.  Ptail is used to aggregate the different individual logs into a consolidated stream. The stream is batched in 1.5 sec groupings by Puma which stores the event batch into HBase. The real time logs are kept for a certain period of time and than get cleared from the system.

Obviously this description is a fairly simplistic view – the full details are provided in the original post.

Evaluating the Facebook Architecture

Facebook reasoning behind their technology evaluation seems acceptable for the most part, although there are some obvious concerns.

There were two things that caught my attention:

Memory Counters

(Facebook) felt in-memory counters, for reasons not explained, weren't as accurate as other approaches. Even a 1% failure rate would be unacceptable. Analytics drive money so the counters have to be highly accurate.

It sounds to me that the evaluation was based on memcached. By default, it's not highly available; a failure would result in loss of data. Obviously, that doesn’t apply to some other memory based solutions such as In Memory Data Grids (for example, GigaSpaces and Coherence both were designed for high resiliency.)

Cassandra vs HBase

The choice of HBase over Cassandra was also very interesting mainly since Cassandra was developed by another Facebook team to address write scalability.  The choice had to do with the write rate differences between the two alternatives at the time of evaluation:

HBase seemed a better solution based on availability and the write rate. Write rate was the huge bottleneck being solved.

Eric Hauser  posted a comment on this analysis which seem to indicate that this issue has been addressed with Cassandra:

When Facebook engineers started the project 6 months ago, Cassandra did not have distributed counters which is now committed in trunk. Twitter is making a large investment on Facebook for real-time analytics (see Rainbird). Write rate should be less of of bottleneck for Cassandra now that counter writes are spread out across multiple hosts. For HBase, every counter is still bound by the performance of single region server? A performance comparison of the two would be interesting

Eric's comment is indicative of how dynamic the NoSQL space is. I’d be interested in how different the technology selection would be now.

The Architecture

There are a few common principles that drive the architecture for this type of system:

• Events logging needs to be extremely fast, to minimize the latency impact on the site.
• Events need to be reliable, otherwise the entire system's accuracy is questionable and the data is devalued.
• The real time data in the form of logs is kept for a certain period of time (x hours or x days)
• Write scalability is key.
• Post processing can happens in batches, the size of the batch depends on how *real-time* we need this data to be.
• Writes to a backend database need to be done asynchronously.

Facebook seem to follow those same principles in their architecture while keeping the system at scale at a fairly impressive rate.

There are a few questions that are still open as I don’t have full visibility into their system:

• Are Ptail and Puma centralized components? If so, don't they pose a potential bottleneck? Based on Todd's summary and Alex' presentation, it seems that the way Facebook scaled their system is by splitting the PTail stream by categories of events so each type can be handled by a different data center.
• Puma batches event logs in memory before it writes them into Hbase – what happens if Puma fails before the batch is written?
• The solution seemed to be limited to handling simple counters. This seem to be a fairly severe limitation, as many systems need to produce more complex relationships even during the real time part of the system, as also indicated as part of the future enhancements, as shown:

"(We need to know) how many people across a time window liked a URL. Easy to do in MapReduce, hard to do with a naive counter solution.”

• In one point, it is noted that Facebook chose not to rely on memory for counters. However, throughout the description it seems that there is still strong reliance on keeping data within memory boundaries:

“(We) write extremely lean log lines. The more compact the log lines the more can be stored in memory..”

“(We) batch for 1.5 seconds on average. Would like to batch longer but they have so many URLs that they run out of memory when creating a hashtable”

Looking backward – wouldn’t it be better to store the data in-memory in the first place? Why add the extra architecture components, if you're able to make memory work for you? This is a crucial question, of course, because it focuses attention on memory availability. As mentioned, though, there are in-memory data grids that are designed for just this kind of situation.

• It is noted that Puma writes it batches to Hbase sequentially:

“(We) wait for last flush to complete for starting new batch to avoid lock contention issues...”

What if this rate is lower than the actual write rate?

Realtime.Analytics.next

Interestingly enough, I was asked to draw a solution for similar challenge for a voice recording system: collect lots data from various sources and process them in real time. The good news is that it's doable, as shown by Facebook's success.

The better news is that it's actually fairly easy. We can add rich query capabilities, elastic scaling, database and platform neutrality, evolution of data, and more without making things unnecessarily difficult - it's not so much that this is the next generation of realtime analytics as much as map/reduce and the hbase approach used by facebook is the previous generation.

I'll describe how it can be done more simply in the next post.

References

• Video: Realtime Analytics for Big Data: A Facebook Case Study
• Real Time Analytics for Big Data: An Alternative Approach
• Big data in real time is no fantasy (GigaOm)
• Facebook's New Realtime Analytics System: HBase To Process 20 Billion Events Per Day
• Read/write scale without complete re-write
• How Map/Reduce and the Cloud are Affecting Analytics Applications
• Is MapReduce going mainstream?
• Memory is the New Disk for the Enterprise

My JavaOne 2010 Plans

JavaOne 2010 is coming next week. This year will be the first time Oracle is running the show, so it'll be interesting to see how well (or badly) Oracle treats the community, as there has been lots of speculation recently on how Oracle plans to monetize the technology. The recent lawsuit against Google and the public departure of James Gosling the father of Java didn’t helped to dismiss that fear.

My plan for this year is sort of a continuation of last year. Last year, I was mostly focused on the emergence of Cloud/PaaS technologies and how they will shape the next-generation application server stack (see relevant posts here and here). This year I'm going to drill down more into the components that make up this new PaaS/Cloud stack. My session at JavaOne (NoSQL Alternatives: Principles and Patterns for Building Scalable Applications) is going to deal specifically with data scalability patterns as they have emerged through the various NoSQL implementations. In this talk, I'll try to focus on the common principles and patterns behind the various approaches, such as Sharding, Eventual Consistency, Query models, Map/Reduce, and Transaction Processing. I’ll also discuss how the two models SQL/NoSQL can live together using the examples from Google JPA/BigTable, HBase/Hive and GigaSpaces/SQL and the document model.

For those who don’t plan to attend JavaOne, or for those who are just interested in open discussion around PaaS/Cloud and Dev/Ops, we'll be holding two meetups in collaboration with Adrian Cole from JClouds on Tuesday and Wednesday. Both meetups will take place at 6:30 PM at the Joyent headquarters (345 California Street Suite 2000, 15 min walk from Moscone Center) who were kind enough to host the event.

• In the PaaS showdown: Java vs Ruby meetup Wesley Beary and Ed Muller from EngineYard are going to reveal some of the architecture behind their PaaS as well as their experience in moving to Terremark. I’ll expand upon the patterns and ways to build your own multi-tenant PaaS on any cloud. This meeting will also be an interesting opportunity to get both the Ruby and Java camp together and learn and compare between the Java and Ruby stack.
• In the DevOps meetup were going to have few lightning talks on production stories, strategies, and tradeoffs, including decisions such as native Java vs. the rest APIs, pull vs. push provisioning, configuration, and integration, agentless(ssh) vs. agent OS control. Following the lightning talks we'll have an open discussion and Q&A panel based on your feedback.

** Note that the room for these meetups is limited so if you plan to attend please RSVP ASAP so we can plan accordingly.

I marked other sessions and BOFs (33 in total), mostly around my favorite scalability, NoSQL, and Hadoop topics during the JavaOne conference. If you have other suggestions or comments, you're welcome to post them here, and I’ll update the list accordingly.

See you next week…

ID	Title	Type	Presenter(s)	Company
S313726	Performance and Debugging Advancements in OpenJDK	BOF	Eric Caspole; Tom Deneau; Vladimir Kozlov	Advanced Micro Devices; Advanced Micro Devices; Oracle
S314364	Standard Dependency Injection (DI) with @Inject and JSR 330	BOF	Costin Leau	SpringSource
S314377	Performance Tuning with Cheap Drink and Poor Tools	BOF	Kirk Pepperdine	Kodewerk Ltd.
S313904	Comparing Apache CXF and Axis2 Web Service Frameworks	BOF	Thomas Liou; Jim Smith	PDX; PDX
S314196	A New Era for In-Memory Data Grids	BOF	Manik Surtani	Red Hat, Inc.
S314190	The Better RESTful Web Services Framework: Jersey or Spring Framework	BOF	Hien Luu	Netflix
S314196	A New Era for In-Memory Data Grids	BOF	Manik Surtani	Red Hat, Inc.
S314563	Hadoop vs. Relational Database: Shout-out Between a Java Guy and a Database Guy	BOF	Karthik Shyamsunder; Chethan Thippeswamy	VeriSign Inc.; VeriSign
S313430	Chillin' with ICEfaces	BOF	Ted Goddard	ICEsoft Technologies
S313988	The Groovy and Grails BOF, with Live Grails Podcast Recording	BOF	Glen Smith	Bytecode Pty Ltd
S314360	Dependency Injection (DI) Flavors in Spring 3	BOF	Costin Leau	SpringSource
S314648	Give Me a Proper Visual JavaServer Faces Editor or Else . . .	BOF	David Konecny	Oracle
S313773	Going Beyond Memory Leaks: Debugging Java from Dumps, Using Memory Analyzer	Conference Session	Chris Bailey; Trent Gray-Donald	IBM; IBM
S313977	Functional Programming in Java: Lessons Learned by GridGain	Conference Session	Nikita Ivanov	GridGain, Inc.
S314432	Multiple Languages, One Virtual Machine	Conference Session	Brian Goetz; John Rose	Oracle; Oracle
S313223	Practical Big Data Processing with MapReduce and Hadoop	Conference Session	Shevek M	Karmasphere, Inc.
S314192	Measuring Performance and Looking for Bottlenecks in Java-Based Data Grids	Conference Session	Manik Surtani	Red Hat, Inc.
S314235	The Cassandra Distributed Database	Conference Session	Jonathan Ellis	Rackspace Hosting
S314496	Top 10 Lessons Learned from Deploying Hadoop in a Private Cloud	Conference Session	Rod Cope	OpenLogic, Inc.
S313637	What's New in Grails 1.3~	Conference Session	Graeme Rocher	SpringSource
S313876	JSR 299 (CDI), Weld, and the Future of Seam	Conference Session	Dan Allen	JBoss by Red Hat
S314404	A Lean, RESTful Java Architecture for Building Rich HTML5 Web Applications	Conference Session	Keith Donald; Jeremy Grelle	VMware; VMware
S314445	Scale Java Persistence API Applications with OpenJPA Slice	Conference Session	Pinaki Poddar	IBM
S314492	Java Persistence API (JPA) 2.0 with EclipseLink	Conference Session	Doug Clarke; Shaun Smith; Gordon Yorke	Oracle; Oracle; Oracle
S312989	Comparing Groovy and JRuby	Conference Session	Neal Ford	ThoughtWorks Inc.
S312998	NoSQL Alternatives: Principles and Patterns for Building Scalable Applications	Conference Session	Nati Shalom	GigaSpaces, Inc
S314142	Groovy and Concurrency	Conference Session	Paul King	ASERT
S314154	Writing Domain-Specific Languages (DSLs), Using Groovy	Conference Session	Paul King	ASERT
S314226	SomeSQL: Combining NoSQL Technologies with Existing RDBMS Architectures	Conference Session	Joshua Tuberville	eHarmony, Inc.
S314782	Choosing the Right NoSQL Database	Conference Session	Tobias Ivarsson	Neo Technology
S314413	Extracting Real Value from Your Data with Apache Hadoop	Hands-on Lab	Michal Bachorik; Aaron Kimball; Daniel Templeton	Oracle; Cloudera; Oracle
S313963	Script Bowl 2010: A Scripting Languages Shoot-out	Panel	Rich Hickey; Guillaume Laforge; Raghavan Srinivas; Dick Wall; Roberto Chinnici	Clojure; SpringSource; Self; Navigenics, Inc.; Oracle

Elasticity for the Enterprise -- Ensuring Continuous High Availability in a Disaster Failure Scenario

In the past few weeks I've had many interesting discussions on our new Elastic Middleware and how it fits into the current enterprise world. One of the interesting comments that was made is that in order to make use of elasticity in general you need to have a spare pool of resources -- while the reality is that most enterprises don’t have many spare resources in their data center. Moreover, taking a mission-critical application that needs to provide deterministic behavior and moving it around in a production environment can be a scary idea for the operations team.

I think that the best way to answer these concerns is through an example: In this example I will describe how a top firm managed to maintain the high availability of their production application under an extreme failure scenario. The purpose of the example is to illustrate that elasticity applies to lots of areas where we have complex procedures that must be automated, and it is basically used to ensure that our application meets a given SLA in a continuous basis and provide enough control points so that you can control where and how those automated functions happen.

The Application: Real-Time Web Application

The application itself is a real-time web application that enables users to subscribe to different news items in real time. The application itself targets millions of subscribes and has to manage x100GB of data. The application uses a fairly standard stack -– load balancer and web container (Tomcat in this specific case) at the front, and a GigaSpaces In-Memory-Data-Grid front-ending an Oracle database at the back. The application uses the In-Memory-Data-Grid for managing session high availability and as the system of record that enables fetching the data at memory speed for the company's online subscribers. 

The Challenge: Ensuring Application Continuous Availability During Planned and Unplanned Downtime

To ensure the system's continuous high availability, it needs to tolerate both planned and unplanned downtime scenarios including the extreme case of a complete data center failure.  

Planned downtime happens as part of an organization's standard procedure for hardware refresh as well as a preemptive measure to clean their system. Unplanned downtime, as the name suggests, can happen anytime, anywhere. An extreme scenario would be a complete failure of the data center.

Ensuring High Availability During Planned Downtime

The following steps ensure high availability of the data during planned downtime:

1. Bring the target machine down.
2. Down-scale the system to the currently available set of machines to ensure that existing users connected to the system will continue to be served.
3. Launch a new alternative machine.
4. Synchronize the machine with the existing live cluster: While the machine was down, the system continued to serve user requests; so before we bring a new machine to the pool we need to ensure that it contains the current state of the cluster.
5. Re-balance the resources so that the load is spread with the newly available machine.

One thing that immediately pops up when I look at all those steps is: the fact that were dealing with planned downtime doesn't make the actual recovery procedure any simpler. In fact it would have been exactly the same procedure as with un-planned downtime. The only thing that is different is that we may control the time it happens and be there to in case something goes wrong.  Now it is clear that if we need to manually follow all those steps the chances that something will go wrong is fairly high. In addition to that if we have lots of application in our data center that needs to follow this procedure then the cost and complexity of following through that procedure is going to be extremely high up to the point where we it becomes unmanageable.

Automating planned downtime recovery using the elastic middleware

With elastic middleware, an agent runs on each machine and reports the machine's availability and its state. In addition, we can interact with the machine and instruct it to do almost anything we want through an API. In our specific case, when we bring a new machine into the system we detect its availability and then relocate the relevant resources from the existing set of machines into the new machine in order bring the system back to its normal state.

Ensuring Continuous High Availability During a Disaster

In a post-9/11 world, every serious enterprise maintains a disaster recovery site to ensure its continuous availability in case of a disaster failure, where a complete data center could become out of reach for a long period of time.

The challenge for the enterprise in our specific case was not just to ensure complete recovery, but to do so while the application is running, with no -- or with only minimal -- hiccups.

In this case, we had the machines of both the disaster recovery site and the primary site connected through a 10 msec latency network as if it was a single cluster.To deal with the latency, we grouped each site into what we refer to as zones. With zones, we can split an entire cluster into sub-clusters and apply policies only to a specific group based on their zone affinity. In this case, we used the elastic middleware to ensure that all backup and primary nodes would be evenly distributed between the two zones. If one of the zones went down, the available site would turn its backup nodes into primaries and would continue to serve the entire load of currently active users, so the users would hardly feel any hiccups. 

 

As soon as the site comes back on, the elastic middleware re-balances the resources and bring them back to the previous state.

Failures are Inevitable. Cope with It!

Many enterprises' high-availability architecture is based on the assumption that you can prevent failure from happening by putting all your critical data in a centralized database, back it up with expensive storage, and replicate it somehow between the sites. As I argued in one of my previous posts (Why Existing Databases (RAC) are So Breakable!) many of those assumptions are broken at their core, as storage is doomed to failure just like any other device, expensive hardware doesn’t make things any better and database replication is often not enough.

One of the main lessons that we can take from the likes of Amazon and Google is that the right way to ensure continuous high availability is by designing our system to cope with failure. We need to assume that what we tend to think of as unthinkable will probably happen, as that’s the nature of failure. So rather than trying to prevent failures, we need to build a system that will tolerate them.

As we can learn from a recent outage event in one of Amazon's cloud data centers, we can’t rely on the data center alone to solve this type of failure. The knowledge of how to manage failure must be built into our application:

"By launching instances in separate Availability Zones, you can protect your applications from failure of a single location," Amazon notes in a FAQ on its Elastic Compute Cloud service

Final Words

According to IDC, the total cost of downtime in 2009 was estimated at $400 billion, an average of $8,000 per server per year. We don’t need academic research to know that the chances of downtime grow exponentially in proportion to the number of moving parts. This becomes even more challenging as we start to build larger and larger systems to deal with the demand for scale. Under these conditions, providing built-in automation becomes a necessity rather than a luxury, and indeed during the poll that we conducted it turns out that 100% of the people in the poll thought that same way.

Update 1:

Pawel Plaszczak wrote an interesting post on The cost of High Availability (HA) with Oracle that provides an interesting insight into the actual cost associated with the current high availability models:

The original “DB v1” option was priced at $611K. After the license tuning exercise, the total for “DB v2" option came in at $518K - a saving of 93K.

For this type of project, the major cost considerations are hardware and software, while services and support are marginal. For "DB v1", cost breakdown is: 36% hardware, 40% software, 13% services, 11% support.

Note: 

The GigaSpaces Elastic middle-ware is built as an abstraction layer on top of the GigaSpaces Cluster Admin API which is in GA since 7.0 release and is already in use by many of the customers. The Elastic middleware implementation is already available for beta users as part of the 7.1 release.

    

References

• The interactive cloud
•  Why Existing Databases (RAC) are So Breakable
• Amazon S3: Architecting For Resieliency In The Face Of Failures By Jason McHugh
• Failure Trends in a Large Disk Drive Population - Google, Inc., February 2007
• Brief Power Outage for Amazon Data Center

The Missing Piece in the Virtualization Stack (Part 1)

This and the next post will discuss how virtualization and cloud computing, as we know it today, is only a small part of the solution for today’s IT inefficiencies. While new technologies and delivery models have made it much simpler to manage the infrastructure, this is not where our core inefficiencies lie. Virtualization principles must be extended to higher levels of the application stack, to make it easier for all of us to manage, tune and integrate applications. Otherwise we will continue to spend most of our time on things that don’t provide real value to the business.

What do we really spend our time on?

If you’ve been in the application development space for a while, I'm sure that you are all familiar with the current application development cycle. The diagram below shows a typical application development cycle. As you can see, we spend a large part of our time on things that don’t provide real value to our business.

Typical application development lifecycle

The continuous demand for scale and scalability has made things even worse – many of us are forced to repeat this cycle over and over again every time we are faced with new scaling requirements:

The promise of virtualization/cloud

Virtualization and cloud computing aim to solve a large part of the overhead involved in setting up the infrastructure (buying new hardware, setting it up, installing it, etc). Indeed, we can now start a new machine just by calling an API, we can lease a machine or even completely outsource our entire infrastructure to a public hosting provider.

Does this solve all of our problems?

As I outlined in the diagram above, setting up the infrastructure is only part of the challenge in the development of new business application. If you measure the complexity/effort required, plugging an application into the infrastructure isn’t necessarily the biggest challenge. Most of us spend most of our time maintaining our code, plumbing it to other services within our organizations and continuously maintaining and tuning it. In recent years, with the growth of data volumes on the one hand and the demand for better efficiency on the other hand, I found that most of the time (and cost!) is spent on dealing with these two contradicting requirements: Each demand for additional scaling forces us to go through a complete cycle of tuning, design and in some cases, through a complete product selection phase to meet the demand.

Last week alone, I found myself spending a good amount of time in discussion with a large telco ISV that built its solution through a combination of storage devices, databases, and so on. In the telco world many of these services face both an increase in the size of the data (per user) and an increase in the number of users. Imagine the increase in the size of pictures that you’re able to send through your phone. It started with few KB, is now up to 100KB and will soon get up to Megabytes of data per message, as the camera resolution grows. Multiply it by the number of users and messages per second and you get a classic scaling challenge. In this telco ISV’s specific case, it is fairly easy to partition the problem based on users (personally, I believe that this is only a temporary assumption, as I'm sure that with the likes of Twitter this assumption will no longer hold true). Now, they could have gone through the traditional way of scaling which is to duplicate their system several times, each unit dealing with smaller amount of users. Now that sounds easy, so why they are still reluctant to do that? The answer is fairly easy – cost. This is potentially an easy solution but fairly inefficient. Taking into account that there is only so much a customer would be willing to pay, that cost will come out of their pocket and effect their profit margin. It may even cost them their business, as at this point they might be beaten by a competitor who comes up with a more efficient solution.

The elephant in the room

Now here is the elephant in the room – would virtualization or cloud solve their problem? It might be a solution – but for only a small part of their challenge. And that’s my point. We spent a good deal of the past year talking about cloud and virtualization as the solution for all of our inefficiency problems, but we forgot that they cover only a small part – in some cases even a fairly insignificant part – of our challenge. Our main real-life challenge is not how to make our infrastructure more efficient but how to make our business more efficient!

To illustrate this gap, I like to use the three questions below:

Assuming that with cloud and virtualization you can easily create a new machine by a call of an API…

Q1: What would happen to your existing application when you add a new machine?
A1: Nothing – it wouldn’t even know that it exists if we wouldn’t tell it (through manual work).

Q2: Assuming that you addressed (1) – which part of your application would you run on that new machine?
A2: It depends... we need to measure and see  => meaning more manual work…

Q3: Assuming that you addressed (2) – what do you expect would be the impact of the new hardware capacity on your application, in terms of latency/throughput or concurrent users?
A3: We wouldn’t know until we measure it in real life => meaning lots of manual tuning, testing, optimization and in some cases redesigning your entire system.

The solution

The challenge I was trying to outline doesn’t necessarily point to a flaw in virtualization or even cloud computing, which is basically an outsourced version of virtualization. It is more to do with the fact that the IT world has applied the concept of virtualization only to the lower level part of the stack – the infrastructure, and expected that it will solve all its inefficiencies. Conceptually, I believe that virtualization is the way to go but it needs to be applied through the entire stack, as I outlined in one of my earlier posts (The Missing Piece in Cloud Computing – Middleware Virtualization). To learn how to apply the concept of virtualization through the entire stack, it must be better understood how virtualization works in other layers, above the infrastructure. If you examine different virtualization technologies such as storage virtualization, operating system virtualization and desktop virtualization, a pattern emerges:

The Virtualization pattern

1. Break big physical resources into smaller logical units

2. Decouple the application from the physical resources

3. Provide an abstraction that makes all the small units look like one big unit

Scaling pattern of a virtual resource

When you scale a virtualized resource, you basically plug in more small physical resources, and thus increase your capacity. The abstraction layer is responsible for detecting these new resources and adding the new resource to its pool. Since the application is decoupled from these resources, it “sees” the increased capacity without necessarily worrying about where those resources exist.

Making it more efficient through resource sharing and pooling

The way to make the solution more efficient, is to pool and share resources together among multiple instances of the application. This is often called multi-tenancy. The general idea is that you can pool the resources of multiple users of your application, and assume that none of them is going to require your full capacity, so you can put them on the same underlying hardware. Obviously, one of the the biggest challenges with multi-tenancy is isolation, i.e., how to let each user “feel” as if she is running on her own dedicated resource.

I know this is a fairly simplistic view of the concept, and obviously doing this for a mission critical application that is running in production is going to require much more thought. In part II of this post I’ll discuss in more depth how to apply those principles through the entire stack.