Nati Shalom's Blog

Achieving 100% Uptime and 139% Increase in Holiday Online Sales: Lessons Learned

 

Background

You don’t need to be an expert to realize that a failure of an eCommerce site during Black Friday or Cyber Monday is a disastrous event, leading to huge loss in revenue and reputation for the retailer. As the share of eCommerce accounts increases to more than 8% of total US retail sales this year, the impact of failure becomes more significant - not just to the site itself, but on the the overall economy. A study on the subject, compiled by Joyent and New Relic, showed that 86% of companies experienced one or more episodes of downtime last holiday season. At the same time, 58% of customers will not use a company’s site again after experiencing site errors.

 

Another study by Radware measured not just the impact of downtime on eCommerce sites, but also the impact of slowness - an even more common and less measured metric.  According to this study a one-second delay correlates to:

• 2.1% decrease in cart size

• 3.5-7% decrease in conversions

• 9-11% decrease in page views

• 8% decrease in bounce rate

• 16% decrease in customer satisfaction

• a 2.2 slowdown equals a 7.7% conversion rate hit.

Meanwhile, KISSmetrics illustrated how page loads longer than three seconds lead to a 40% bounce rate.

Obviously there is enough business incentive here to invest in handling both the downtime and latency issues. Meanwhile taking a look at a typical retailer traffic (source: Akamai) during this season, we notice that the traffic spikes increase by at least 500%:

 

In this post I will share our specific experience and lessons-learned from the 2014 holiday season which turned out to be very successful.  I believe that the results below speak for themselves.

 

2014 Results

 

 

How We Achieved These Results

• Taking a preemptive approach - rather than reacting after a failure occurred- prevented failure in the first place.

• Common Causes: Most failures are the result of misconfiguration or capacity planning guesswork.

• Planning: Proactive tuning with continuous system monitoring produces predictable improvements at scale, eliminating risks from incorrect provisioning.

• Knowledge & Experience: eCommerce applications are complex and built from many subsystems. In many cases, an eCommerce organization does not have the expert skill-set in each of the subsystems. Having an expert in the room helps to bridge this gap and builds the capabilities of business operations.

• Fast Feedback: When product-related issues are identified, we were able to provide the fastest path to protect the business and address concerns in a timely fashion.

To give you a bit more insight on this process I’ve added a section to this post called Stories from the War Room which illustrates a real-life incident and the action that was taken by our on-site engineer to resolve it.

• Using In-Memory Computing to buffer peak load access to shared data resources

• Typical eCommerce systems have shared data resources for managing inventory, orders, and catalog information.

• Putting the shared data resources in-memory provides faster and more efficient (parallel) access to this shared data.

• Data is mirrored back into the database in batches. In this way, peak load transactions are buffered so that database traffic does not crash the database back-end.

• The In-Memory Compute grid acts as a system of record. Failure in the underlying database can be saved without affecting the online users while the database is restored to a working state.

• Using a combination of In-Memory & SSD allows very large In-Memory data sets to be stored at a reasonable cost, while still ensuring fast recovery during failure.

• Self-Healing Systems recover from failure in real time

• Failures are inevitable: Keeping a backup copy in-memory enables zero-downtime systems to service user traffic without interruption, even if something does go wrong.

• Systems provisioned for failure handle failure by design.

• Automatic failover and provisioning eliminates the need to overprovision (costly) resources in case of failure. Traditionally, it’s common for retailers to provision resources for holiday season that are 5 times the capacity of non-holiday traffic infrastructure.

Two Examples

In one case, a Top 100 online retailer used XAP to provide access to its catalog and inventory data and achieved its first zero-downtime holiday season in several years. As a result, this retailer delivered a vastly improved customer experience from previous years (achieving an 18% improvement in customer satisfaction ratings) and generated a 139% increase over 2013 holiday sales.

In another case, a Top 30 US online retailer logged a record-setting peak sales day of $44 million. This was especially notable because that same day the retailer experienced system performance issues caused by an automated hardware failover condition. Fortunately, the retailer’s XAP implementation began automatically relocating application components to standby resources, keeping apps running despite the complications. As a result, consumers continued to shop—and buy—with minimal disruption.

Stories from the War Room

I’ve picked two issues that we identified as our engineers were working on-site with one of our top eCommerce customers. I thought that these two cases provides useful insight on how a preemptive support strategy and a short feedback loop works:

Issue #1: Sudden slow client response time

The quote below was taken from the direct on-site report:

 

GC Spikes is one of the common issues that we encounter for managing in-memory data clusters. As GC tends to compete with the same CPU resources that serves the user transaction, this often leads to overall slowness of the system. Fairly quickly, this slowness can pile up into a huge backlog which can break the system in unexpected areas.

The resolution was to split the cluster into more data containers (GSC in XAP terminology) as this will allow better spread of the load across the entire cluster. In addition, overall capacity (memory and CPU) that was allocated for the cluster was increased to meet the increasing capacity demand.

The diagram below provides a view of one of the clusters at the time the issue occurred.

 

As can be seen, around 23:00 the system started to hit its high CPU mark as a result of GC spikes. The system was gradually rebalanced after couple of hours without facing any downtime.  The preemptive action that was taken to handle this incident prevented it from piling up and causing a complete system failure.

 

Issue #2: Connection Issues

 

In this incident, the increase of concurrent client activities during peak load resulted in a large number of network connections that were opened at the same time. One of the nodes in the cluster was misconfigured with a low limit on the number of concurrent connections that could be opened simultaneously.  The resolution was to kill that faulty node and leverage the self-healing capability of XAP to force an immediate re-route of clients into the backup node while relocating the faulty node into another machine.

  

Final Notes

Peak load performance often tends to stretch any system behavior in areas that are least expected and thus are often hard to handle. Quite often, peak loads lead to unexpected downtime.

There are many cases in which this sort of peak load performance is known in advance, as is the case with Black Friday and Cyber Monday. Still, many eCommerce sites continue to experience downtime or slowness during such events that lead to huge loss of revenue and reputation.

As a software vendor, we have often found ourselves involved in the early architecture discussion phases, which usually take place as a result of failure in the previous year. Despite the fact that we are brought in to solve these peak load performance problems, we are still called during a fire drill when those failures occur. Often, the result of the failure was misconfiguration or a problem on another system that manifested itself as an issue in our product. Obviously the experience of handling fire drills is never be pleasant, neither for us nor for our customer and that is something that we wanted to avoid as we approached the 2014 holiday season.

This year we decided, together with our customers, to take a more preemptive approach by putting an engineer on-site to escort the customer team during the event itself.  This resulted in huge success, leading to 100% up time. Both teams learned much from the experience; the customer learned even better how to operate our product and what to look for to ensure that the system is running properly. We learned much about how the customer is using our product and were able to shorten the feedback loop between the customer and our product and engineering team.

With those lessons in hand I feel that both we and our customers are much more equipped to handle 2015.  I can’t wait to write about the lessons learned from Black Friday 2015.

 

 

 

References:

Avoid Blackout on Black Friday and Cyber Monday [Infographic]

The Real Cost of Slow Time vs Downtime [SLIDES]

How Loading Time Affects Your Bottom Line

Using SSD As A Foundation For New Generations Of Flash Databases

“You just can't have it all” is a phrase that most of us are accustomed to hearing and that many still believe to be true when discussing the speed, scale and cost of processing data. To reach high speed data processing, it is necessary to utilize more memory resources which increases cost. This occurs because price increases as memory, on average, tends to be more expensive than commodity disk drive. The idea of data systems being unable to reliably provide you with both memory and fast access—not to mention at the right cost—has long been debated, though the idea of such limitations was cemented by computer scientist, Eric Brewer, who introduced us to the CAP theorem.

The CAP Theorem and Limitations for Distributed Computer Systems

Through this theorem Brewer stated that it was impossible for any distributed computer system to be able to provide users with these three following guarantees simultaneously:

• Consistency (Every node will be able to view the same data at the same time)

• Availability (Every request will receive a response)
• Partition Tolerance (The system will continue to operate even if the system faces any arbitrary failures)

 We can evaluate different approaches for data management solution by the above three properties and the tradeoffs that each one of them faces when utilized over the other. For example, what is the tradeoff of putting more emphasis on consistency? The trade off will most likely be less availability or partition tolerance.

 Traditional RDBS solutions offered consistency over partition tolerance and cost. In-Memory caching solution such as memcache offered a different set of tradeoffs, for example speed over some degree of consistency (between the in-memory data and the data held in disk).In Memory Data Grid or In Memory Computing, as it is called today, extended the use of memory as the system of record and thus enabled higher degree of consistency by reducing the dependency on the underlying disk databases. The new generation of NoSQL databases took a different approach and offered speed and scale at low cost over consistency by utilizing distributed commodity storage and relying heavily on asynchronous data flow to speed up the data processing.

 Quite often there is a direct correlation between those tradeoffs and cost - for example high degree of consistency often relies on synchronous data-flow and replication operations which often come at a cost of speed. To overcome the speed limit we often need to use memory or other flavors of high speed storage which often translate to high cost. On the other hand if we can compromise on consistency, we can offer speed and scale using commodity resources (  without relying on high cost resources).

As technology advances and demands for cost effective and scalable, reliable data increase, new solutions are coming to light, with one of the most favored being a combination between two items we are already familiar with: RAM and flash.

 

Why SSD Is a Viable Storage Solution

Many of the underlying assumptions in the previously mentioned alternatives that  was built under the assumption that disk is the bottleneck. If disk is the bottleneck, we needed layers of optimizations to minimize the access to disk. SSD provides a high speed storage device, which gets rid of the idea that disk is the bottleneck. This is a fundamental change in one of core assumptions behind the design of many databases today, let me explain..

 For example if the disk is slow it makes sense to put various filters to minimize the access to disk. As disk gets faster all of those filters becomes an overhead. In many cases it would be faster writing directly to disk without those additional layers. In similar way the use of asynchronous operations need to change as well. When the disk is slow it makes sense to use asynchronous operations to deffer write operations. However, when the disk is no longer the bottleneck it would be faster and vastly simpler to access the disk directly. This also allows avoiding many of the consistency tradeoffs that i mentioned earlier that are often a result of those asynchronous operations.

 

Putting SSD and RAM Together using off Heap Storage

 RAM storage can be as big as necessary and it is incredibly quick—but it costs about ten times more than a flash disk. By putting SSD and RAM together we can optimize the cost/performance ratio. The main challenge in doing so, remain around consistency-i.e. how do we keep the data in-memory and flash in sync so that from an end user perspective this integration would look completely seamless?

One method of such integration is referred to as off-heap storage.  

Off Heap storage is often implemented as a plug-in that is used by the memory data-store to offload its data from the actual RAM into Flash and visa versa.

Traditionally Off Heap was implemented with shared memory which is basically a block storage that provides external access to the same RAM device. By doing so, we bypass the management overhead of managing data in RAM through JAVA. The limitation of that approach is that the capacity is still limited by the size of RAM. It also forces an external data management and garbage collection layer to manage this external heap.

With SSD, users can now have both the memory and the SSD device synchronized and regularly kept up-to-date, meaning storage is scalable and consistent.

Some of the SSD drivers provides a key,value interface which makes the integration vastly simpler as it takes care of its own data  management.  Having said that, most of the In-Memory implementation that uses SSD as off-heap storage provides fairly limited functionality in terms of query and transaction support. This is due to the fact that they( What is they?)  integrate with SSD through a disk storage interface.  

 

Using SSD as a Foundation for Flash DataBase

 To really make the best use of SSD and maximize its potential we need to think of SSD as foundation for database. This requires a more tight and native integration with SSD in order to overcome some of the limitations that exist with many of the current In Memory Data Grid and SSD implementations.

 The specific set of features that are needed for this sort of integration includes: 

• Portable and native Key/Value interface that works against any flash device.
• Using Flash as Durable Storage - Flash can be used as a persistent data-store and not just as extension to RAM. As such we can leverage the durability of flash to speed up the load time of data from the underlying flash device incase of a planned or un-planned recovery process.
• Support for batch operations - batch operations are a common way to speed up the access to flash by minimizing the number of cross boundaries calls between the RAM device and the underlying flash devices.
• Transactional - To support transactional access we need to extend the batch operations to fail, succeed as a single atomic operation.
• Query index support - to speed up the query and access time, the data needs to be indexed in a way that fetching a particular object by its IDcould point directly to its physical location on disk.

 

To make this a practical reality GigaSpaces partnered with SanDisk who through fusion I/O acquisition own the majority of the SSD market. As part of this partnership, we implemented a new version of our In Memory Data Grid, XAP MemoryXtend, which is now integrated with SanDisk ZetaScale interface. ZetaScale implements all of the above features as a general purpose flash disk interface. Through this integration we are now able to utilize the XAP RAM based storage for handling complex queries and analytics as well as transactional support. We can also leverage the XAP cluster support for integrating multiple flash devices that are mounted on various machines and make them work as one big transactional database ( as illustrated in the diagram below).

The XAP MemoryXtend cluster consists of a number of partitions each plugged into a local SSD drive.  Each partition can have at least one backup running on a separate machine to ensure  availability. The XAP client is a smart proxy that abstracts the underlying cluster and exposes all of the physical partitions through a single data-grid interface. The proxy routes the write or query requests to a particular partition in cases where we’re looking for a specific data item. In the case of agregated queries it will invoke a parallel query against all nodes and consolidate the results into a single result-set.

 

Flash DataBase as Service

As Flash devices are now being supported by cloud providers such as Amazon, it only make sense to leverage this capability and offer this sort of Flash based database on-demand.

This is where the XAP Deployment and Orechestration comes handy as it automates the process of deployment and management of XAP clusters across a variety of cloud infrastructures.

For more information, check out The next Big Disruption in Big Data.

The next big thing in big data: fast data

The big data movement was pretty much driven by the demand for scale in velocity, volume, and variety of data. Those three vectors led to the emergence of a new generation of distributed data management platforms such as Hadoop for batch processing and NoSQL databases for interactive data access. Both were inspired by their respective predecessors Google (Hadoop, BigTable) and Amazon (Dynamo DB).

As we move to fast data, there’s more emphasis on processing big data at speed. Getting speed without compromising on scale pushes the limits in the way most of the existing big data solutions work and drives new models and technologies for breaking the current speed boundaries. New advancements on the hardware infrastructure with new flash drives offer great potential for breaking the current speed limit which is bounded mostly by the performance of hard drive devices.

Why using existing RDBMS, NoSQL on top of flash drive is not enough

Many of the existing databases — including more modern solutions such NoSQL and NewSQL — were designed to utilize the standard hard drive devices (HDD). These databases were designed with the assumption that disk access is slow and therefore they use many algorithms such as Bloom filters to save access to disk in case the data doesn’t exist. Another common algorithm is to use asynchronous write to commit-log. A good insight into all the optimizations that are often involved in a single write process on NoSQL databases is provided through the Cassandra Architecture. Let’s have a look at what that entails below:

 

The Cassandra write path

 

What happens when disk speed is no longer the bottleneck?

When the speed of disk is no longer the bottleneck, as in the case of flash devices, then a lot of the optimization turns into overhead. In other words, with flash devices it will be faster and simpler to access the flash device directly for every write or read operation.

Flash is not just a fast disk

Most of the existing use cases use flash devices as faster disk. Using flash as a fast disk was a short path to bypass the disk performance overhead without the need to change much of the software and applications. Having said that, this approach inherits many unnecessary disk driver overhead. So to exploit the full speed potential of flash devices, it would be best to access the flash devices directly from the application and treat flash devices as a key/value store rather than a disk drive.

Why can’t we simply optimize the existing databases?

When we reach a point in which we need to change much of the existing assumptions and architecture of the existing databases to take advantage of new technology and devices such as flash, that’s a clear sign that local optimization isn’t going to cut it. This calls for new disruption.

The next big thing in big data

Given the background above, I believe that in the same way that demand for big data led to the birth of the current generation of data-management systems, the drive for fast data will also lead to new kinds of data management systems. Unlike the current set of databases, I believe that the next generation databases will be written natively to flash and will use direct access to flash rather than the regular disk-based access. In addition to that, those databases will include high performance event streaming capabilities as part of their core API to allow processing of the data as it comes in, and thus allowing real-time data processing.

 

Fast data in the cloud

Many of the existing databases weren’t designed to run in cloud as first-class citizens and quite often require fairly complex setup to run well in a cloud environment.

As cloud infrastructure matures, we now have more options to run big data workload on the cloud. The next generation databases need to be designed to run as a service from the get-go.

To avoid the latency that is often associated with such a setup, the next generation databases will need to run as close as possible to the application. Assuming that many of the applications will run in one cloud or another, it means that those databases will need to have built-in support for different cloud environments. In addition, they will need to leverage dynamic code shipping to pass code with the data, and in this way allow complex processing with minimum network hops.

In-memory databases and data grid are the closest candidates to drive next-gen flash databases

RAM and flash-based devices have much more in common than flash and hard drive. In both RAM and flash, access time is fairly low and wasn’t really the bottleneck; in-memory databases use direct access to the RAM device through key value interface to store and index data.

Those factors makes in-memory based databases more likely to fit into the next generation flash databases.

The combination of in-memory databases and data grid on top of flash devices also allows the system to overcome some of the key capacity and cost per GB limitations of an in-memory based solution. The integration of the two will allow an increase in the capacity per single node to the limit of the underlying flash device rather than to the limit of the size of RAM.

The overall architecture of the integrated solution looks as follows (Source: Gartner):

 

 

As can be seen in the diagram above, the IMC (in-memory computing) layer acts as the front end to the flash device and handles the transactional data access and stream processing, while the flash device is used as an extension of the RAM device from the application perspective.

There are basically two modes of integration between the RAM and flash device.

LRU Mode – In this mode, we use the RAM as a caching layer to the flash device. The RAM device holds the “Hot,” i.e. the most recently used, and the flash device holds the entire set.

Pros: Optimized for maximum capacity.

Cons: Limited query to simple key/value access

Fast Index Mode – In this mode, all the indices in RAM and the data in flash holds the entire set.

Pros: Supports complex querying, including range queries and aggregated functions.

Cons: Capacity is limited to the size of indexes that can be held in RAM.

In both cases, the access to flash is done directly using key/value interface and not through disk drive interface. As both RAM and flash drives are fairly fast, it is simpler to write data to the flash drive synchronously and in this way avoid potential complexity due to inconsistency.

It is also quite common to sync the data from the IMC and flash devices into an external data store that runs on a traditional hard drive. This process is done asynchronously and in batches to minimize the performance overhead.

Direct flash access in real numbers

To put some real numbers behind these statements, I wanted to refer to one of the recent benchmarks that was done using an in-memory data grid based on GigaSpaces XAP and direct flash access using a key/value software API that allows direct access to various flash devices. The benchmark was conducted on several devices as well as on private and public cloud-based services on AWS. The benchmark shows a ten times increase in managed data without compromising on performance.

 

 

 

For full details of the benchmark, see the following white paper:

XAP MemoryXtend -Massive Application Storage Capacity for Real-Time Applications

 

 

 

Notes from Big Data Business Challenges Panel Discussion

Most of the Big Data topics that I covered in the past were focused mostly on the technical aspects of running Big Data.

In this post, I want to take a different route and discuss the business challenges based on a panel discussion that I had the pleasure of moderating.

 

 

Is Big Data a Big Hype?

One of the first questions that came up in the discussion is whether Big Data is over-hyped due to over-use in marketing content. Many of those who claim to provide either a Big Data solution or implement Big Data are just using the term to re-brand their product or offering.

 There was consensus across the panel that while Big Data is probably over hyped like any big trend, it does represent a real change in both technology and business. Let me explain:

In a pre-Big Data world, Big Data lived in expensive and closed data warehouses, where many of the solutions were often tied to specific, high end hardware. In addition, the term “warehouse” in itself meant that most of the data had little exposure and impact on the business.  Today, Big Data is everywhere and can be done at a fraction of the cost of the traditional data warehouses. With the addition of cloud the barrier to running Big Data has become significantly lower and now every startup can run Big Data almost at the scale of Google.

Having said that, the actual adoption of Big Data tends to vary quite significantly per industry with telecommunications (67%), consumer goods (57%) and financial services (52%) industries as the leaders in recognizing that Big Data can greatly improve their understanding of customer needs according to a recent Hitachi Data Systems Corporation (HDS) study  .The same study indicates that more than 60% of the firms in the financial services and consumer goods industries haven't started any Big Data programs. Healthcare and life science are lagging further behind; 72% of them haven't started any Big Data programs. This means that the perspective of whether Big Data is real or just marketing hype tends to vary as well depending on the industry.

 

How do you make sense out of your Big Data? Do we need a new role for Chief Data Officer?

Many of the questions in the panel had to do with how to make sense of the data.

One of the common strategies to make sense of the data is to assign a data scientist or Chief  Data Officer, expecting them to deal with the data and offer useful insights. Another common strategy is to put all the data in something like Hadoop with the hope that having the data in hand will provide a variety of interesting insights that will change the way we operate.

Putting the data in Hadoop or assigning a specialized data analyst are often just the beginning of a process.

Eyal Veitzman from Wix described how they deal with the challenge. To really make sense of your data, you first need to change the organizational structure in which the business, developers and data analysts will be part of the same group. In this way you can turn insights into action and more importantly have a faster feedback loop that will allow you to iterate quickly.  

 

What is the business value behind Big Data?

The business value behind Big Data comes in different shapes and forms depending on where you apply it in your system. Insightera uses Big Data to capture user interaction and increase the conversion of users using personalized responses. Wix uses Big Data to optimize their user conversion and satisfaction.  Nice Systems uses Big Data to improve their call center user satisfaction. PayPal is using Big Data in all sort of shapes and forms to both detect fraud as well as to optimize their operations. Check Point uses Big Data to collect data from all their gateways and in that way keep their gateways protected from new virus attacks

So how much can we gain from applying Big Data practices in our organization?

According to the Hitachi Data Systems Corporation (HDS) study, Big Data can help companies' revenue by 25%. This applies mostly to companies that use Big Data to optimize their internal operations and products, like most of the companies in the panel. That is obviously more true for companies that sell Big Data insights as part of their business, as in the case of Insightera.

Is there a good visualization tool for Big Data?

Big Data visualization is a tough challenge and there is no simple answer or single framework that fits them all. Using infographic style can help to visualize data in a way that makes it easier to capture lots of data into a single, readable report using a variety of visualization dimensions, such as fonts, colors, graphics, etc. There are many open source frameworks, such as Jaspersoft and many SaaS-based frameworks, aimed to provide specialized reporting and visualization tools. A useful reference on that regard can be found here here: The 36 best tools for data visualization.

Having said that, the important thing is to decide what and how to measure. Quite often, the number of Key Performance Indicators (KPIs) must be limited to a fairly small number.

The second important thing is to realize that as humans, we are not designed to capture and process vast amounts of information and therefore, visualization tools provide only a small aid to a much bigger challenge.

In summary, there is a dangerous slippery slope with Big Data visualization; fairly quickly we can flood our users with many graphs and numbers to a point where what we actually get is a dimension return and more complexity and confusion than clarity. Those who had experience with analytics tools such as Google Analytics are probably familiar with that experience. On the other hand, narrowing down the key performance indicators into a small number of KPIs makes the visualization challenge simple.

 

Moving from Reports to Actions

The key in capturing most of the value of Big Data is to turn Big Data into actions. In the example of Insightera, actions means presenting a user with the most relevant information on a website. In the case of Nice Systems, actions would refer to routing the caller in a call center to the right agent based on the caller’s profile and history of behaviour. In the case of GigaSpaces, this would mean constantly adjusting our provisioning system to detect failures and auto-heal a production environment when something goes wrong.

To get to that level we need to apply automation practices on how we collect, analyze and execute actions out of our data. We also need to be able to process data in real time to reduce the time to action.

Final words: How do make our data instantly actionable?

It is quite clear that the end game of any Big Data initiative is to figure out a way to make it instantly actionable. One of the big challenges on that regard is that quite often we need to compromise between scale and speed. In fact, many of the Big Data systems (Hadoop included) use batch processing to be able to process large amounts information, meaning that the actual insights, and thus actions, from that data will be available only when the batch cycle has been completed.

To make Big Data instantly actionable we need to figure out a way in which we can process vast amounts of data in real time and still keep the cost down.

During the panel discussion, PayPal described their use of In-Memory based architecture to handle this challenge. Still, pure In-Memory-based solutions tend to be fairly expensive and limited in capacity comparing to disk based alternatives. So the next big thing IMO is to devise a way in which we can get In-Memory speed at a cost that is closer to Disk. This is where new advancements in SSD have the potential to disrupt the entire way in which we manage and process Big Data, by providing high capacity durable RAM based out of SSD that can reach terabytes capacity per single machine. The fact that it can be durable means that we could rely completely on In-Memory databases to manage and store larger chunks of our Big Data with no need to offload or reload the data to and from external disks. I also expect that this will lead to the rise of a new generation of In-Memory databases that will be designed primarily for this new kind of durable RAM devices.

 

About the Panel

The Big Data Business challenges panel  was led by the CICC and in collaboration with IGTCloud, focusing primarily on the business challenges side of Big Data. The event was held simultaneously in California and Israel and was hosted by eBay. The panel included a range of industry experts in Big Data from a wide range of companies and industries and included Allen Kamer, Chief Commercial Officer, Optum Analytic (Co-founder of Humedica that was acquired by Optum Health), Doron Simon, Head of Cloud, Business Development, Nice Systems, Mickey Alon, Co-Founder & CEO, Insightera (Acquired by Marketo), Dr. Nachum Shacham, Principal Data Scientist, PayPal, Eyal Veitzman, Director of Business Operations, Wix and Ron Davidson, Head of Security Services, Check Point.

I had the special honor to organize and moderate the panel together with Ido Sarig, General Manager, IoT Solutions Group while Wind River moderated the panel in California.

 

Useful References

• Big data can help companies' revenue by 25%

Telecommunications (67%), consumer goods (57%) and financial services (52%) industries are leaders in recognizing that big data can greatly improve their understanding of customer needs. However, more than 60% of the firms in the financial services and consumer goods industries haven't started any big data programs. Healthcare and life science are lagging further behind; 72% of them haven't started any big data programs. http://timesofindia.indiatimes....­

• Fast Data hits the Big Data fast lane

The usual factors: the same data explosion that created the urgency for Big Data is also generating demand for making the data instantly actionable. Bandwidth, commodity hardware and, of course, declining memory prices, are further forcing the issue: Fast Data is no longer limited to specialized, premium use cases for enterprises with infinite budgets. Not surprisingly, pure in-memory databases are now going mainstream: ..

..Potential use cases for Fast Data could encompass:

• A homeland security agency monitoring the borders requiring the ability to parse, decipher, and act on complex occurrences in real time to prevent suspicious people from entering the country
• Capital markets trading firms requiring real-time analytics and sophisticated event processing to conduct algorithmic or high-frequency trades
• Entities managing smart infrastructure which must digest torrents of sensory data to make real-time decisions that optimize use of transportation or public utility infrastructure
• B2B consumer products firms monitoring social networks may require real-time response to understand sudden swings in customer sentiment
• The 36 best tools for data visualization

A useful reference for popular visualisation frameworks 

ScaleDB: A NewSQL Alternative to Amazon RDS and Oracle RAC with 10x Performance Gain

In the world of relational databases, MySQL and Oracle RAC (Real Application Cluster) own a significant segment of the market. Oracle RAC owns the majority of the enterprise market, while MySQL gained significant popularity amongst many of the Web 2.0 sites.

Today, both of those databases are owned by Oracle (MySQL was acquired by Sun which was later acquired by Oracle).

The following diagrams show the enterprise database marketshare covered by Gartner and Cloud Database market share covered by Jalistic - a Java PaaS provider.

The acquisition of MySQL by Oracle raised concern over the future of the project due to the inherent conflict of interest between its two database products. Oracle RAC is the company’s main “cash cow” product, while MySQL competes for the same audience.

Shortly after Oracle’s acquisition of MySQL, the open source database was forked by one of its original founders into a new project named MariaDB. MariaDB was established to provide an alternative development and support option to MySQL and is now becoming the default database of choice of RedHat.

MySQL vs Oracle RAC Clustering Models

The two databases take a fairly different approach to scalability.

Oracle RAC is based on a shared storage model. With Oracle, the data is broken into strips that are spread across multiple devices, and multiple database servers operate concurrently and in sync over the (shared) data.  

MySQL, on the other hand, does not use a shared data model. With MySQL, a given data set is managed by a single server (a model called shared nothing). With MySQL, scaling and High Availability (HA) is achieved by managing copies of the data. As only a single machine can update the data, this mode can only scale-up by adding more capacity to the machine that owns the database. As machines have limits to capacity yet must keep up with large amounts of data or many users, the database needs to be broken into several independent databases (a process called sharding). However, sharding is a complex process, is not dynamic and requires assumptions on data distribution, data growth and the queries. For these reasons, sharding is not possible with many applications. In particular, a cloud deployment is expected to be elastic to dynamically support changing needs and user behaviors.

For High Availability, the MySQL shared nothing approach uses Primary/Backup model with a single master and a backup node (called slave) that manage a copy of the data. Each update to the master requires that the same update will be executed on the slave. The primary and slave nodes require some handshake protocol to determine who is the master and sync the changes of the data. The master node performs updates/writes to the persistent file-system and the level of High Availability is set by the DBA, who decides if the slave can lag behind or needs to confirm the duplicate updates within each transaction.

For scaling, this model can use the slave as a read replica (or make additional copies of the data), a method called horizontal scaling in which read requests are spread across the different copies of the data. (However, all the writes need to go to the master and then be reflected on the replicas/slaves.)

Relational Databases on the Cloud

The high-end position of Oracle RAC, the low-cost and open source nature of MySQL, along with the adoption of the cloud as the infrastructure platform led to a vastly different method of deployment of databases in the cloud.

Oracle RAC took an approach similar to the old mainframe: to produce a pre-engineered appliance (called Exadata) that comes with the software and hardware integrated. That approach was specifically aimed at existing customers of Oracle RAC who needed a quick resolution to their scalability needs without redesigning their applications. Plugging a more optimized stack helped to push the scalability bar without changing the existing applications that rely on the database.

Amazon launched RDS, which is basically an on-demand version of the MySQL database. This approach fits nicely with the existing users of MySQL who are looking for a more affordable way to run their database in the cloud.

The main limitation of the Amazon RDS approach is that it inherits the limitations of MySQL and thus provides a limited and relatively complex read-scalability and doesn't provide a good solution for write-scalability other than the scale-up approach.

ScaleDB - a NewSQL Alternative to Amazon RDS and Oracle RAC

ScaleDB is a NewSQL database that takes an approach similar to the read/write scalability model of Oracle RAC and implements it as a platform that transparently supports the MySQL (or MariaDB) database. As a result, existing MySQL/MariaDB applications can leverage the platform without any changes - they use MySQL or MariaDB as the front end engine which connects to the ScaleDB platform that provides a new and more concurrent database and storage services. This approach makes it possible to run multiple MySQL or MariaDB server instances against a distributed storage in a shared data model (similar to Oracle RAC).

The diagram below shows in high level how ScaleDB works.

Each Database Node runs a standard MariaDB database instance with ScaleDB plugged as the database engine and as an alternative storage and index device.

Scaling of the data and the index is done by processing requests concurrently from multiple database instances and leveraging the ScaleDB distributed storage tier (at the storage/file-system level) where data and index elements are spread evenly across multiple storage nodes of the cluster. Read more on how ScaleDB works.

ScaleDB vs Other NewSQL Databases

Most of the NewSQL databases are based on MySQL as the engine behind their implementation. The main difference between many NewSQLs and ScaleDB is that NewSQL databases brings the NoSQL approach into the SQL world, where ScaleDB takes the Oracle RAC shared storage approach to scale the database.

ScaleDB can deliver write/read scale while offering close to 100% compatibility, whereas in many of the alternative NewSQL approaches, scaling would often involve significant changes in the data model and queries semantics.

Benchmark - ScaleDB vs Amazon RDS

To demonstrate the difference between a standard cloud deployment of MySQL and a ScaleDB deployment and in order to find whether the ScaleDB approach can live up to its promise we conducted a benchmark comparing Amazon RDS scalability for write/read workloads with that of ScaleDB. We tested a dataset that does not fit to the memory (RAM) of a single machine and used  the largest machines offered by Amazon. We required that scaling would be dynamic and that all types of queries would be supported. These requirements made sharding a no-go option.

The benchmark is based on the popular YCSB - Yahoo Benchmark as the benchmarking framework.

The results of that benchmark are shown in the graphs below.

Both illustrate a relatively flat scalability with Amazon RDS and a close to linear scalability on the ScaleDB front.

Benchmark environment:

• Benchmark Framework - YCSB - Yahoo Benchmark

• Cloud environment: Amazon

• Machine Type: Extra Large

• Data Element Size (per row) - 1k

• Data Capacity: 50GB

• Zones - Single zone.

• RDS was set with 1000 provisional ios

• ScaleDB cluster setup - 2 database nodes, 2 data volumes (4 machines -data is striped over 2 machines and each machine had a mirror).

• ScaleDB MySQL engine - MariaDB

Running ScaleDB on OpenStack and Other Clouds with Cloudify

The thing that got me excited about the project is that it serves as a perfect fit for many of our enterprise customers who are using Cloudify for moving their existing applications to their private cloud without code change. Those customers are looking for a simple way to scale their applications and many of them run today on Oracle RAC for that purpose.

The move of enterprises to a cloud-based environment also drives the demand for a more affordable approach to handle their database scaling, which is provided by ScaleDB.

On the other hand, setting up a database cluster can be a fairly complex task.

By creating a Cloudify recipe for ScaleDB, we could remove a large part of that complexity and set up an entire ScaleDB database and storage environment through a single click.

In this way we can run ScaleDB on demand as we would with RDS and scale on both read and write as with Oracle Exadata, only in a more affordable fashion.

To try it now please follow the steps described in the ScaleDB Git project

References

1. Cloudify Recipe for running Scale DB

2. Yahoo Benchmark

3. ScaleDB Architecture white paper

4. ScaleDB user manual

Industry Survey Shows: Organizations Turn to Stream Processing for their Big Data Needs

In a stream processing model, data is processed as it arrives to the Big Data system. With a batch processing model, only once the data is stored does the system run a variety of batch analytics - typically through a map/reduce style of processing.

 Facebook and Twitter are good examples of stream processing models in use today, as I outlined in a previous post entitled Facebook’s vs Twitter’s Approach to Real-Time Analytics.

 According to a recent survey of approximately 250 respondents, there is a trend moving towards stream processing in order to speed up analytics, with a significant increase in the popularity of this model when compared with last year. The number of organizations planning to use stream processing in 2014 has more than doubled (24%) from last year’s amount (10%).

This goes hand in hand with the fact that real-time analytics is also becoming more mainstream, as I pointed out in my 2014 Predictions.

Another interesting data point from the survey is that roughly 43% of the respondents defined real-time as sub-second and 42% defined it as sub-minute. This difference is quite interesting and could lead to different approaches for implementing stream processing.

For example, real-time is defined differently by Facebook and Twitter. In the case of Twitter, they chose Storm as their event processing engine, allowing them to process events at a sub-second resolution. Meanwhile, Facebook defines real-time at a 30 second batch window, choosing a logging-based approach to fit their need.

Based on the survey, it appears that both approaches are valid and can be applied in correlation to the degree of real-timeliness of your analysis.

 

Facebook Log-Centric Stream Processing	Twitter Event-Centric Stream Processing

In-Memory Data Grids become a more popular choice for real-time processing

According to the survey, 64% of the respondents indicated that they plan to combine In-Memory-based solutions for delivering their real-time analytics processing. This is also consistent with last year survey done by Ventana Research.

 I believe that will see an even bigger movement in this direction, as the cost/performance ratio of In-Memory Data Grids and In-Memory Databases would go down significantly with the combination of RAM and Flash devices, which together provide a fairly compelling solution from both performance and cost ratio to that of other disk-based alternatives.

 

Big Data in the Cloud increased to 56%

New developments in cloud infrastructure, such as the support for bare metal in the new OpenStack releases, as well as support, high memory, flash disk, etc, are removing many of the technical barriers for running I/O intensive workloads like Big Data in the cloud.

Indeed, according to the survey, there is a significant shift towards the use of Big Data in the cloud with 55% of organizations either using or planning to run their Big Data in the cloud in 2014.

 

Where do we go from here? How does this effect GigaSpaces’ future roadmap?

There are multiple areas in the GigaSpaces roadmap that aim to address this demand. 

• Real-Time Processing through Storm Integration - In this project, we integrate Storm on top of a Memory backend for both stream processing and data-store.
• Support for Flash Disk - The integration with flash disk is planned for our upcoming XAP release and will include support for SanDisk, Fusion I/O and other flash disk devices.
• Big Data in the Cloud - through Cloudify - We've been continuously extending our Big Data portfolio support and recently added support for Storm and Cognos, adding to our existing support for Hadoop, MongoDB, Cassandra, ElasticSearch, etc.

 

References

• GigaSpaces Annual Big Data & Cloud Survey 2013
• Eight Cloud and Big Data Predictions for 2014
• GigaSpaces and Storm Part 2: XAP Integration
• GigaSpaces And Storm Part 1: Storm Clouds
• Big-Data-Real-Time-Performance - Enjoying both worlds with one Architecture!
• 2012 Big Data Survey Report
• In Memory Computing (Data Grid) for Big Data

Big Data on OpenStack

Big Data systems by their very nature tend to be...Big.

Big in the amount of data, in the number and size of infrastructure behind it, etc. Cloud based infrastructure can be a good fit as a cost effecive instrastrucutre for running those Big Data systems. While this may sound obvious, many of the Big Data deployments that I've encountered run outside of a cloud environment. The main reason is the performance overhead that is often associated in running on a virtualized cloud environment. Let me explain..

The Performance Overhead of Running Big Data on the Cloud

A recent benchmark done by Petersenna over various virtualization solutions, such as XEN, VMWare, HyperV etc, shows that on average the performance overhead associated with virtualization can lead to 2.4 times slower disk latency and 25% slower network I/O.

How does the performance overhead translate into cost?

Based on the various benchmarks, I think that it would be fair to assume that running I/O intrusive workloads such as Big Data on a virtualized infrastructure would require 3X more resources than its Baremetal equivalent.

As we are talking about Big Data infrastructure - 3x means a lots of wasted resources. The operational costs coupled with the complexity associated with running a significantly bigger system yields a fairly substantial overhead and cost. In addition, the Network and Disk overhead are not deterministic and can vary quite substantially when the utilization gets higher. This leads to not just performance overhead, but also non-deterministic overhead. In Big Data terms, that means that a query for particular data can take 10msec one time or 30msec another time. Quite often, running Big Data analysis requires a sequence of those operations. Therefore, if we pile this overhead in a sequence of 10 queries, the variance in the response time can vary quite substantially.

While the choice of running Big Data on the cloud holds a lot of promise, the performance and non-deterministic behavior makes that choice limited to more of a niche scenario where this overhead  becomes less significant in comparison to the elasticity benefit of the cloud. For example, for sporadic worklads in which we run our analysis for a certain period of time and then can release the resources, using on-demand infrastructure is still a better choice, as running 3x the amount of resources for an hour is significantly cheaper than having a third of that infrastructure allocated 24/7.

OpenStack Bare Metal Cloud

The analysis above shows that for I/O intensive workloads, virtualized infrastructure isn't such a good fit.

Cloud is often viewed as an infrastructure on top of virtualization. Therefore, by definition, a cloud-based infrastructure inherits the benefits and limitations of virtualization.

Is the coupling between cloud and virtualization really mandatory?

If we think of a cloud as an infrastructure for getting compute, storage and network resources on-demand, then there is nothing  to necessitate the coupling of the cloud with virtualization. It became common to pair the two mainly because of the complexity involved with provisioning non-virtualized resources and the limitations in enabling partitioning of a given bare metal machine.

As I noted in one of my previous posts Bare Metal Cloud/PaaS,  there are ways today to provision a baremetal machine from an image and partition it just like we would do with a hypervisor based VM.  There are already cloud providers that offer a choice of bare metal machines as part of their cloud infrastructure.

A new bare metal project in the OpenStack Grizzly release takes this a step further. It allows us to use the same compute API (NOVA) and allocate a bare metal machine just as we would with a virtualized machine.

All we need to do to make the switch is to change the image type of our target machine and the cloud infrastructure will know to map that request and allocate a bare metal image instead of a virtualized instance.

With this option we can now run our entire Big Data workload on the cloud and not worry about switching environments depending on the workload happen to be sporadic or I/O intesive.

HubSpot OpenStack – Bare Metal Case Study

During the last OpenStack Summit, Jim O’Neill CIO at HubSpot showed how using OpenStack with a combination of a public virtualized cloud and private bare metal cloud enabled a 4X increase in their infrastructure efficiency:

 

“We took this single image, picked it up from public cloud into a Rackspace-powered private cloud and saw a 4X increased efficiency running that workload.”

 

Moving from Existing Data Centers to the Cloud

Many existing Big Data and BI systems run on traditional data center environments. Moving those systems into an OpenStack-based environment isn't going to be a walk in the park. This is where automation frameworks, such as Cloudify with the combination of Chef and Puppet, make this transition smoother.

In this approach, we can automate the deployment of our existing Big Data/BI systems in a way that will be abstracted from the underlying infrastructure. We can later use this abstraction to run Big Data systems in our existing data center, and when ready, we can use the same deployment framework on an OpenStack-based environment without re-doing any of that investment.

 

 

Learn more through hands-on experience - a Real Life Experience on HP OpenStack

As it often happen in this sort of conceptial discussions the points and arguments from this post may often sound artificial and not easy to grasp. To make it more down to earth, we've put a NoSQL datastore such as Couchbase, MongoDB, Cassandra, ElasticSearch and Big Data applications available on-demand on HP OpenStack cloud services. You can use this reference to launch any of those applications and then use the management console to browse through the recipes and customize it to your environment as needed.

It is also worth pointing out that the recipes behind this project are available on Github and can be easily deployed on your cloud or in your data center or even desktop in pretty much the same way. If you are interested in more details, please post a comment on this post or in the Cloudify forum.

For more on Big Data on OpenStack, come check out my presentation at Cloud Expo East on Tuesday, June 11th at 8:15am in the “Cloud Computing and Big Data” Track.

References

• Bare Metal Cloud/PaaS
• OpenStack Baremetal Project
• Big Data in the Cloud
• Big Data in the Cloud using Cloudify
• Putting Hadoop On Any Cloud (A video presentation)
• In Memory Computing (Data Grid) for Big Data
• Using the Cloudify Player as an Open Source Framework for Building Your Own Cloud Application Marketplace on OpenStack
• Going native: The move to bare-metal cloud services
• New bare metal cloud offerings emerging
• How much overhead does x86/x64 virtualization have?
• Amazon EC2 versus Bare Metal and KVM? The inside story on what you thought you knew about EC2

Real-Time Big Data with Storm, Cassandra, and XAP

Real-time Big Data is becoming more and more common in Big Data systems.

One of the most common frameworks used for running real-time Big Data system is Twitter Storm backed by a NoSQL database, as shown in the diagram below:

 

In this architecture, Storm is used for processing data as it comes in and the NoSQL data store is used as the output for that processing as well as for reference data storage.

Challenges

This architecture presents a couple of challenges:

• Performance - Storm runs in-memory, and is therefore set to process large volumes of data at in-memory speed. However, a typical storm architecture needs to interface with a data source for its input and a data source for its output. In this context, the overall performance is now determined not by Storm itself but by the data input and output sources. Quite often these interfaces rely on file-based message queues for streaming, and NoSQL for data storage. These interfaces are at least an order of magnitude slower than Storm itself, and therefore become the limiting factor.
• Complexity - Storm itself consists of several moving parts, such as a coordinator  (ZooKeeper), state manager (Nimbus), and processing nodes (Supervisor). The NoSQL data store also comes with its own cluster management. In addition, a typical Big Data system comes with more components such as the application front end, a reporting database, and more. This makes the process of managing the deployment, configuration, fail-over and scaling of such a system quite complex.

Meeting the Performance Challenge

Given that Storm itself runs in memory, it only makes sense that in order for Storm to run at maximum capacity the streaming and data store interfaces should be implemented in-memory as well, as shown below:

As you can see in the above architecture, we added two interfaces that rely on a built-in Storm plug-in - one for the data inputs and the second for data output. In both cases, the underlying implementation is memory-based, and thus removing the impedance that the previous architecture included.

You can read the full details on how this integration works, including a code example, in DeWayne Filppi's post on integration of in-memory computing with Cassandra.

The second optimization that can be added is to the NoSQL data store.

As most analytics applications tend to be read-mostly, we can speed up access to the NoSQL data store using an in-memory local cache. This architecture speeds up read performance between 10 to 1000 times as outlined by Shay Hassidim in his post on real-time Big Data performance.

Meeting the Complexity Challenge

To meet the complexity challenge, we will use a DevOps automation approach using Cloudify in conjunction with Chef or Puppet.

With this approach, we wrap every service with a deployment recipe that abstracts the underlying details on how to manage Storm, Cassandra, and our in-memory data store. Cloudify uses these recipes to automate the deployment of the entire stack. In this way, you only need to interact with Cloudify for the deployment, configuration, and also scaling and fail-over of your stack rather than separately manage each individual component. In addition, we use Cloudify to abstract the underlying infrastructure. This enables us to use the same deployment recipe across different environments. One for testing, another for production, etc. We can also use the same approach to deploy our apps based on the type of workload. We can use a public cloud for running sporadic workload, and thus leverage the elasticity of the cloud and also enable us to create and scale the environment as well as rip it off completely when done. At the same time, we can choose bare-metal machines for I/O intensive workloads, and so on.

You can read more on how you can set Storm using Cloudify on DeWayne's post on Storm and the cloud.

Cloudify comes with a rich set of built-in recipes for other Big Data services as well, making the integration process an out of the box experience. The main ones are listed below:

Final Notes - Optimizing without Code Change

Many real-time Big Data systems are now based on Twitter Storm and a NoSQL data store such as Cassandra.

In this post I tried to outline how we can optimize this architecture by addressing two areas: performance, and management.

The good news is that all this is possible to achieve seamlessly, without any code change. Let me explain:

In the case of Storm we used built-in plug-ins - Spout for streaming and Trident as the data store interface. In this way we can simply plug our memory-based plug-ins under these two integration points. All our existing Storm business logic would work pretty much the same.

We use the same plug-in approach to integrate our in-memory data store with our NoSQL data. This integration makes the data flow between our real time streaming to our NoSQL storage fairly transparently. In addition, it allows us to plug in different NoSQL data stores such as MongoDB, CouchBase, etc., giving us another degree of flexibility.

The same applies to our management layer. Existing Storm and NoSQL deployments are wrapped into plugged in recipes, and don't require any specific code changes.

Not Mutually Exclusive

We don't have to implement all of the optimizations to gain the benefit of this architecture. Each of the optimization points can be plugged in  independently of the others. For  example, if your main pain point is complexity, you can start by only by adding the DevOps automation first. At the same time if your main pain point is performance, you can use the memory-based plug-ins to speed up processing.

The other advantage of the combined architecture that I haven't discussed in this post is that it provides more flexibility in the degrees of consistency and availability in which you set your system, as I outlined in one of my recent posts on the subject In Memory Computing (Data Grid) for Big Data.

References:

• Big Data Real-Time Performance
• GigaSpaces and Storm Part 2: XAP Integration
• GigaSpaces And Storm Part 1: Storm Clouds
• In Memory Computing (Data Grid) for Big Data
• Check out how GigaSpaces XAP and Cassandra integration works

Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack - My Take

Ruven Cohen published an article on Forbes pointing to Paypal's plan to move a large part of its servers from VMware to OpenStack.

This is aligned also with what seems to be a general trend where enterprises use OpenStack to reduce their VMware cost. They often start with a hybrid approach in which they move the less mission critical apps to OpenStack and keep the rest for VMware. I assume that this is only a transitional stage until we will see almost all the workload moving to OpenStack as soon as the organization will feel confident with the technology. 

Paypal is not the only one who's moving away from VMware; Koby Holzer from Live Person will be speaking on how Live Person uses OpenStack internally to reduce their VMware cost during the upcoming OpenStack Israel event. I'm sure that there are many others who have already taken that path, but were not as public about it.

While the article focuses on the general trend, I thought that there is a bigger opportunity here worth noting.

The Opportunity

Clearly a failure of one player (VMware in this specific case) is an opportunity for others.

The opportunity is for OpenStack providers, such as HP, Redhat, Rackspace and IBM, to take-on VMware directly rather than focus on winning on Amazon.

Why?

• VMware has a huge enterprise footprint
• Private and hybrid clouds are at the heart of enterprise cloud strategy
• OpenStack is well positioned for the private cloud, given its open source foundation
• Having organizations such as HP, Rackspace, Redhat, Mirantis and recently IBM behind OpenStack should also provide enough comfort that the gaps that still exist will be bridged shortly in a way that will fit the enterprise requirements.

Timing is Everything..

OpenStack was created initially by Rackspace and NASA as a response to the strong dominance of Amazon as the leader in the cloud space. At that time, Amazon was the benchmark and the reference for the OpenStack project.

Many things have changed since OpenStack was first born that, IMHO, need to change some of the fundamental strategies for many of the OpenStack players.

• Amazon was quick to realize the threat and moved remarkably fast up the stack, leaving enough of a gap to preserve its leadership position.
• VMware's response, on the other hand, was fairly poor, despite various attempts to move up the stack, including major acquisition of SpringSource in 400M$, followed by a few other failed attempts, such as VMforce which was later forked into CloudFondry which recently moved out to a new pivotal fork under EMC. Now VMware seems to be taking another turn with its continuously changing cloud strategy, but without real substance. This creates lots of skeptisim in the market over their ability to execute their plans, as noted in Bernard Golden's report: VMware Tantrum Shows It's Not Connecting With Cloud Buyers or Sellers

Like VMware, many of the traditional enterprise players are now threatened by Amazon and need to fight back to defend their current position in the enterprise space.

From a technology perspective, the gap between OpenStack and VMware's offering is much smaller than with AWS. Most of those gaps can be easily bridged by the likes of HP, Redhat, Ubuntu, IBM and Rackspace.

All this makes VMware the right target for OpenStack providers rather than Amazon, at least at this point in time. With that in mind, the right strategy for OpenStack providers is to offer a VMware alternative based on OpenStack and have both a private and public offering of OpenStack that can be easily plugged together as a single compute cloud.

References

• Comparing VMware and OpenStack
• Paypal To Drop VMware From 80,000 Servers and Replace It With OpenStack (Updated)
• Moving from VMware to OpenStack talk - Live Person 
• VMware Tantrum Shows It's Not Connecting With Cloud Buyers or Sellers
• VMWARE'S GROWTH STALLS – LAYOFFS COMING, BUT SO ARE NEW HIRES

NoCAP - Part II Availability and Partition tolerance

Abstract

Eric Brewer's CAP theorem is one of the foundations behind the design and architecture of many of the large scale systems architectures. According to the one form of the CAP theorem, “You either choose availability or consistency. You cannot choose both.” 

Is this our only choice?

In this post I'll try to provide an aggregated view of the different perspectives on this question and plant the seeds for an alternative approach.

Availability vs Partition tolerance.

One of the thing that comes up repetitively through the various debates is the lack of clarity behind the definition between Availability and Partition Tolerance.

To set the stage I’ll  start by referring to the original definitions by Gilbert and Lynch . I’ll use Coda Hale's You Can't Sacrifice Partition Tolerance - a "must-read" that provides a good summary.

Brief history  (By Coda Hale):

In 2000, Dr. Eric Brewer gave a keynote at the Proceedings of the Annual ACM Symposium on Principles of Distributed Computing¹ in which he laid out his famous CAP Theorem: a shared-data system can have at most two of the three following properties:Consistency,Availability, and tolerance to network Partitions. In 2002, Gilbert and Lynch²converted “Brewer’s conjecture” into a more formal definition with an informal proof.

The original definition by Gilbert and Lynch:

Availability

For a distributed system to be continuously available, every request received by a non-failing node in the system must result in a response. That is, any algorithm used by the service must eventually terminate … [When] qualified by the need for partition tolerance, this can be seen as a strong definition of availability: even when severe network failures occur, every request must terminate.

Partition tolerance

In order to model partition tolerance, the network will be allowed to lose arbitrarily many messages sent from one node to another. When a network is partitioned, all messages sent from nodes in one component of the partition to nodes in another component are lost. (And any pattern of message loss can be modeled as a temporary partition separating the communicating nodes at the exact instant the message is lost.)

Dr. Stonebraker’s "Consistency over Partition Tolerance"

Dr. Michael Stonebraker’s post Errors in Database Systems, Eventual Consistency, and the CAP Theorem argues that since partition failures are rare you might sacrifice partition tolerance for consistency and availability:

Obviously, one should write software that can deal with load spikes without failing; for example, by shedding load or operating in a degraded mode. Also, good monitoring software will help identify such problems early, since the real solution is to add more capacity. Lastly, self-reconfiguring software that can absorb additional resources quickly is obviously a good idea.

In summary, one should not throw out the [Consistency aspect] so quickly, since there are real error scenarios where CAP does not apply and it seems like a bad tradeoff in many of the other situations.

Henry Robinson (Cloudera) describes "CAP Confusion"

Henry Robinson, in his response to Dr. Stonebraker entitled Problems with ‘partition tolerance’, argues that failure are inevitable and therefore you can’t give away partition tolerance:

..Partition tolerance is not something we have a choice about designing into our systems. If you have a partition in your network, you lose either consistency (because you allow updates to both sides of the partition) or you lose availability (because you detect the error and shutdown the system until the error condition is resolved). Partition tolerance means simply developing a coping strategy by choosing which of the other system properties to drop. This is the real lesson of the CAP theorem – if you have a network that may drop messages, then you cannot have both availability and consistency, you must choose one..

Coda Hale : "You Can't Sacrifice Partition Tolerance"

Coda Hale also argues that you can’t give away partition tolerance in his response, You Can't Sacrifice Partition Tolerance. Mr. Hale suggest a more relaxed version of availability through graceful degradation that is based on the yield and harvest model from  Eric A. Brewer's ACM article, Lessons from Giant-Scale Services:

You cannot ... choose both consistency and availability in a distributed system.

Of the CAP theorem’s Consistency, Availability, and Partition Tolerance, Partition Tolerance is mandatory in distributed systems. You cannot not choose it. Instead of CAP, you should think about your availability in terms of yield (percent of requests answered successfully) and harvest (percent of required data actually included in the responses) and which of these two your system will sacrifice when failures happen.

Dr. Stonebraker - Partition tolerance and Machine failure are two separate things

Both Henry Robinson and Coda Hale reffed to a Machine failure as part of their argument that you can't avoid partition tolerance . Dr. Stonebraker provides clarification on that point on his response to Coda Hale: 

.., the dead node is in one partition and the remaining N-1 nodes are in the other one. The guidance from the CAP theorem is that you must choose either A or C, when a network partition is present. As is obvious in the real world, it is possible to achieve both C and A in this failure mode. You simply failover to a replica in a transactionally consistent way. Notably, at least Tandem and Vertica have been doing exactly this for years. Therefore, considering a node failure as a partition results in an obviously inappropriate CAP theorem conclusion.

Daniel Abadi - Problems with CAP

Daniel Abadi Assistant Professor of Computer Science at Yale University outlines four issues in the current definition of CAP in Problems with CAP, and Yahoo’s little known NoSQL system:

• ..The definition of CP looks a little strange --- “consistent and tolerant of network partitions, but not available” --- the way that this is written makes it look like such as system is never available --- a clearly useless system

• ..the roles of the A and C in CAP are asymmetric. Systems that sacrifice consistency (AP systems) tend to do so all the time, not just when there is a network partition

• ..What if there is a network partition? What does “not tolerant” mean? In practice, it means that they lose availability if there is a partition. Hence CP and CA are essentially identical

• Lack of latency considerations in CAP significantly reduces its utility.

Daniel suggest an alternative definition to CAP:

..CAP should really be PACELC --- if there is a partition (P) how does the system tradeoff between availability and consistency (A and C); else (E) when the system is running as normal in the absence of partitions, how does the system tradeoff between latency (L) and consistency (C)?

My Take

The main issue that I see with the current definition of CAP is that it describes each of the properties of CAP in absolute terms. In reality, however, each of the properties can be applied in various degrees. Eric Brewer's definition of yield and harvest is one example of a more relaxed version of availability. Interestingly enough the last note in Gilbert and Lynch is also an interesting recognition that it may be possible to achieve different tradeoffs that provides both Availability and Consistency:

..in partially synchronous models it is possible to achieve a 
practical compromise between consistency and availability. In particular, 
most real-world systems today are forced to settle with returning “most of 
the data, most of the time.” Formalizing this idea and studying algorithms 
for achieving it is an interesting subject for future theoretical research.

The same applies to Partition Tolerance – there are various degrees of partition tolerance. Probably the most extreme one are the one suggested by some of the commenters which is to do with WAN. As Dr. Stonebraker argues, even that is not necessarily a high likelihood scenario for most applications:

There is enough redundancy engineered into today’s WANs that a partition is quite rare. My experience is that local failures and application errors are way more likely. Moreover, the most likely WAN failure is to separate a small portion of the network from the majority. In this case, the majority can continue with straightforward algorithms, and only the small portion must block. Hence, it seems unwise to give up consistency all the time in exchange for availability of a small subset of the nodes in a fairly rare scenario.

So the question whether or not you can address partition tolerance shouldn’t be measured in absolute terms but against the most likely scenario for your application. The answer as to what is a most likely partition tolerance scenario really comes down to common sense and may vary over a course of time and application business needs. 

The introduction of the cloud may also change some of our core assumptions on how we deal with failure. With cloud, we can easily bring an alternate machine up to deal with a failure in matters of minutes, and we can also assume that we have a fairly robust underlying infrastructure with lots of built-in redundancy. That in itself changes the scenario where failures can happen as well as the way to deal with them, compared to the time when the original CAP Theorem was written. 

Another source for confusion is the use of Amazon, Google and Facebook as a reference to justify the eventual consistency model. What we often tend to forget is that Amazon, Facebook and Google face fairly unique challenges that are not that common and even those three still rely on strongly consistent systems for the majority of their applications. As noted in Will Scalable Data Stores Make NoSQL a Non-Starter?:

Facebook famously invented the NoSQL Cassandra database but still relies on the venerable MySQL-plus-memcached combination for the brunt of its critical operations.

The bottom line of my argument is that giving up consistency should be our last resort. Rather then giving up Consistency for Partition Tolerance we could consider a different set of tradeoffs that deals with various degrees of Consistency, Availability, and Partition tolerance that fit our business needs and also take the performance and latency tradeoffs into account.

In the next post on that subject I’ll outline how that can be achieved using the reference architecture that I outlined in the previous post.

Special thanks

I would like to thank Ron Pressler for the constructive debate and for the useful references 

References:

• Errors in Database Systems, Eventual Consistency, and the CAP Theorem (Dr. Michael Stonebraker)

• You Can't Sacrifice Partition Tolerance - (Coda Hale)

•  Will Scalable Data Stores Make NoSQL a Non-Starter?

• Problems with CAP, and Yahoo’s little known NoSQL system

• You Can't Sacrifice Partition Tolerance