Nati Shalom's Blog

Achieving 100% Uptime and 139% Increase in Holiday Online Sales: Lessons Learned

 

Background

You don’t need to be an expert to realize that a failure of an eCommerce site during Black Friday or Cyber Monday is a disastrous event, leading to huge loss in revenue and reputation for the retailer. As the share of eCommerce accounts increases to more than 8% of total US retail sales this year, the impact of failure becomes more significant - not just to the site itself, but on the the overall economy. A study on the subject, compiled by Joyent and New Relic, showed that 86% of companies experienced one or more episodes of downtime last holiday season. At the same time, 58% of customers will not use a company’s site again after experiencing site errors.

 

Another study by Radware measured not just the impact of downtime on eCommerce sites, but also the impact of slowness - an even more common and less measured metric.  According to this study a one-second delay correlates to:

• 2.1% decrease in cart size

• 3.5-7% decrease in conversions

• 9-11% decrease in page views

• 8% decrease in bounce rate

• 16% decrease in customer satisfaction

• a 2.2 slowdown equals a 7.7% conversion rate hit.

Meanwhile, KISSmetrics illustrated how page loads longer than three seconds lead to a 40% bounce rate.

Obviously there is enough business incentive here to invest in handling both the downtime and latency issues. Meanwhile taking a look at a typical retailer traffic (source: Akamai) during this season, we notice that the traffic spikes increase by at least 500%:

 

In this post I will share our specific experience and lessons-learned from the 2014 holiday season which turned out to be very successful.  I believe that the results below speak for themselves.

 

2014 Results

 

 

How We Achieved These Results

• Taking a preemptive approach - rather than reacting after a failure occurred- prevented failure in the first place.

• Common Causes: Most failures are the result of misconfiguration or capacity planning guesswork.

• Planning: Proactive tuning with continuous system monitoring produces predictable improvements at scale, eliminating risks from incorrect provisioning.

• Knowledge & Experience: eCommerce applications are complex and built from many subsystems. In many cases, an eCommerce organization does not have the expert skill-set in each of the subsystems. Having an expert in the room helps to bridge this gap and builds the capabilities of business operations.

• Fast Feedback: When product-related issues are identified, we were able to provide the fastest path to protect the business and address concerns in a timely fashion.

To give you a bit more insight on this process I’ve added a section to this post called Stories from the War Room which illustrates a real-life incident and the action that was taken by our on-site engineer to resolve it.

• Using In-Memory Computing to buffer peak load access to shared data resources

• Typical eCommerce systems have shared data resources for managing inventory, orders, and catalog information.

• Putting the shared data resources in-memory provides faster and more efficient (parallel) access to this shared data.

• Data is mirrored back into the database in batches. In this way, peak load transactions are buffered so that database traffic does not crash the database back-end.

• The In-Memory Compute grid acts as a system of record. Failure in the underlying database can be saved without affecting the online users while the database is restored to a working state.

• Using a combination of In-Memory & SSD allows very large In-Memory data sets to be stored at a reasonable cost, while still ensuring fast recovery during failure.

• Self-Healing Systems recover from failure in real time

• Failures are inevitable: Keeping a backup copy in-memory enables zero-downtime systems to service user traffic without interruption, even if something does go wrong.

• Systems provisioned for failure handle failure by design.

• Automatic failover and provisioning eliminates the need to overprovision (costly) resources in case of failure. Traditionally, it’s common for retailers to provision resources for holiday season that are 5 times the capacity of non-holiday traffic infrastructure.

Two Examples

In one case, a Top 100 online retailer used XAP to provide access to its catalog and inventory data and achieved its first zero-downtime holiday season in several years. As a result, this retailer delivered a vastly improved customer experience from previous years (achieving an 18% improvement in customer satisfaction ratings) and generated a 139% increase over 2013 holiday sales.

In another case, a Top 30 US online retailer logged a record-setting peak sales day of $44 million. This was especially notable because that same day the retailer experienced system performance issues caused by an automated hardware failover condition. Fortunately, the retailer’s XAP implementation began automatically relocating application components to standby resources, keeping apps running despite the complications. As a result, consumers continued to shop—and buy—with minimal disruption.

Stories from the War Room

I’ve picked two issues that we identified as our engineers were working on-site with one of our top eCommerce customers. I thought that these two cases provides useful insight on how a preemptive support strategy and a short feedback loop works:

Issue #1: Sudden slow client response time

The quote below was taken from the direct on-site report:

 

GC Spikes is one of the common issues that we encounter for managing in-memory data clusters. As GC tends to compete with the same CPU resources that serves the user transaction, this often leads to overall slowness of the system. Fairly quickly, this slowness can pile up into a huge backlog which can break the system in unexpected areas.

The resolution was to split the cluster into more data containers (GSC in XAP terminology) as this will allow better spread of the load across the entire cluster. In addition, overall capacity (memory and CPU) that was allocated for the cluster was increased to meet the increasing capacity demand.

The diagram below provides a view of one of the clusters at the time the issue occurred.

 

As can be seen, around 23:00 the system started to hit its high CPU mark as a result of GC spikes. The system was gradually rebalanced after couple of hours without facing any downtime.  The preemptive action that was taken to handle this incident prevented it from piling up and causing a complete system failure.

 

Issue #2: Connection Issues

 

In this incident, the increase of concurrent client activities during peak load resulted in a large number of network connections that were opened at the same time. One of the nodes in the cluster was misconfigured with a low limit on the number of concurrent connections that could be opened simultaneously.  The resolution was to kill that faulty node and leverage the self-healing capability of XAP to force an immediate re-route of clients into the backup node while relocating the faulty node into another machine.

  

Final Notes

Peak load performance often tends to stretch any system behavior in areas that are least expected and thus are often hard to handle. Quite often, peak loads lead to unexpected downtime.

There are many cases in which this sort of peak load performance is known in advance, as is the case with Black Friday and Cyber Monday. Still, many eCommerce sites continue to experience downtime or slowness during such events that lead to huge loss of revenue and reputation.

As a software vendor, we have often found ourselves involved in the early architecture discussion phases, which usually take place as a result of failure in the previous year. Despite the fact that we are brought in to solve these peak load performance problems, we are still called during a fire drill when those failures occur. Often, the result of the failure was misconfiguration or a problem on another system that manifested itself as an issue in our product. Obviously the experience of handling fire drills is never be pleasant, neither for us nor for our customer and that is something that we wanted to avoid as we approached the 2014 holiday season.

This year we decided, together with our customers, to take a more preemptive approach by putting an engineer on-site to escort the customer team during the event itself.  This resulted in huge success, leading to 100% up time. Both teams learned much from the experience; the customer learned even better how to operate our product and what to look for to ensure that the system is running properly. We learned much about how the customer is using our product and were able to shorten the feedback loop between the customer and our product and engineering team.

With those lessons in hand I feel that both we and our customers are much more equipped to handle 2015.  I can’t wait to write about the lessons learned from Black Friday 2015.

 

 

 

References:

Avoid Blackout on Black Friday and Cyber Monday [Infographic]

The Real Cost of Slow Time vs Downtime [SLIDES]

How Loading Time Affects Your Bottom Line

The Seven Most Popular APIs in Big Data – Part 2

Many APIs, Same Data In “The Seven Most Popular APIs in Big Data—Part One,”

I described the various forms of popular data management APIs out there and how they’re often used. As I noted toward the end of the post, currently each of the APIs is often tied with a specific data model and data store and is used by different APIs that serve other use cases. One of the main reasons behind this is that each API represents a specific optimization that requires a fairly different data structure. Common to all of the various techniques however is that they were written under the assumption that disk is a bottleneck. That led to various point optimizations, architecture, and algorithms that are used by each of the APIs to bypass the disk bottleneck.

Memory Based Data Management Is Ready for Flash

Here's the good news: this need not always be the case. Flash devices have become so advanced that they are able to completely remove the disk from causing this bottleneck, which then makes it possible for us to explore API and data structures that will serve them in a completely different way.
I would argue that there's no longer a need for point optimizations that are not compatible with one another and we can instead use a common data structure to serve each of the APIs.

Same Data, Many APIs

So unlike the majority of existing data management solutions which have been designed with the assumption that disk will be a bottleneck, In Memory Data Grids are based on RAM and therefore designed with the assumption that the data device is fast. Best of all, moving memory-based solutions into flash doesn't require any significant change to your current system. To illustrate this, let's look at our solution, XAP, and what it's able to do:

This example shows how one can write the same data as an object, read it as a document, query it through SQL via JDBC connection, navigate through its subfields as in object graphs, and so on. XAP even allows for the same data store to run both transactional data and analytics, vastly reducing complex data transformation.

Utopia vs. Reality

The ideal system would be one where we could rely on just one common data store that's been outfitted with a set of light-weight services that could expose different APIs and semantics to access data, as illustrated in the diagram below:
In reality however, the maturity and cost of a flash-based solution isn't yet ready for such a transformation. A more realistic scenario is for a flash-based device to be used as a data bus or hub that can act as a front-end for various databases and for more high performance and latency-sensitive use cases. In order to reduce transformation complexity, however, flash will need to include built-in synchronization with those databases and streaming technologies so that the area of complexity is handled implicitly.
Memory Based Data Management Systems Are More Suitable to Serve as High Speed Data Buses

Many of the existing databases were designed with the assumption that they serve as the main data store; therefore, moving from MySQL to Mongo for example often means a complete re-write. Memory Data Grids on the other hand were designed as an extension to the database that takes/holds the part of the data with high contention and serves it in-memory to speed up the access time and scalability. This approach is considered more complementary as it only affects part of the application that needs high speed access, where the remaining of the application continues to work with the same underlying database as if nothing had changed. As a result of that, most of the existing Data-Grid solutions already include fairly rich data synchronization plugins—hence why I would argue that they are more suitable to serve as a Bus or a Hub.

Example: Using High Speed Data Bus with Storm

I’ll leave you with one example of where this framework is effective: “Storm.” This stream processing framework that's used by Twitter includes a few plug-in external feeds (Spouts) while also still maintaining the state of processing (Trident State). The following illustration demonstrates how this integration can be used to handle real time processing of web page analytics by using Storm and XAP as the high speed data bus.

The next big thing in big data: fast data

The big data movement was pretty much driven by the demand for scale in velocity, volume, and variety of data. Those three vectors led to the emergence of a new generation of distributed data management platforms such as Hadoop for batch processing and NoSQL databases for interactive data access. Both were inspired by their respective predecessors Google (Hadoop, BigTable) and Amazon (Dynamo DB).

As we move to fast data, there’s more emphasis on processing big data at speed. Getting speed without compromising on scale pushes the limits in the way most of the existing big data solutions work and drives new models and technologies for breaking the current speed boundaries. New advancements on the hardware infrastructure with new flash drives offer great potential for breaking the current speed limit which is bounded mostly by the performance of hard drive devices.

Why using existing RDBMS, NoSQL on top of flash drive is not enough

Many of the existing databases — including more modern solutions such NoSQL and NewSQL — were designed to utilize the standard hard drive devices (HDD). These databases were designed with the assumption that disk access is slow and therefore they use many algorithms such as Bloom filters to save access to disk in case the data doesn’t exist. Another common algorithm is to use asynchronous write to commit-log. A good insight into all the optimizations that are often involved in a single write process on NoSQL databases is provided through the Cassandra Architecture. Let’s have a look at what that entails below:

 

The Cassandra write path

 

What happens when disk speed is no longer the bottleneck?

When the speed of disk is no longer the bottleneck, as in the case of flash devices, then a lot of the optimization turns into overhead. In other words, with flash devices it will be faster and simpler to access the flash device directly for every write or read operation.

Flash is not just a fast disk

Most of the existing use cases use flash devices as faster disk. Using flash as a fast disk was a short path to bypass the disk performance overhead without the need to change much of the software and applications. Having said that, this approach inherits many unnecessary disk driver overhead. So to exploit the full speed potential of flash devices, it would be best to access the flash devices directly from the application and treat flash devices as a key/value store rather than a disk drive.

Why can’t we simply optimize the existing databases?

When we reach a point in which we need to change much of the existing assumptions and architecture of the existing databases to take advantage of new technology and devices such as flash, that’s a clear sign that local optimization isn’t going to cut it. This calls for new disruption.

The next big thing in big data

Given the background above, I believe that in the same way that demand for big data led to the birth of the current generation of data-management systems, the drive for fast data will also lead to new kinds of data management systems. Unlike the current set of databases, I believe that the next generation databases will be written natively to flash and will use direct access to flash rather than the regular disk-based access. In addition to that, those databases will include high performance event streaming capabilities as part of their core API to allow processing of the data as it comes in, and thus allowing real-time data processing.

 

Fast data in the cloud

Many of the existing databases weren’t designed to run in cloud as first-class citizens and quite often require fairly complex setup to run well in a cloud environment.

As cloud infrastructure matures, we now have more options to run big data workload on the cloud. The next generation databases need to be designed to run as a service from the get-go.

To avoid the latency that is often associated with such a setup, the next generation databases will need to run as close as possible to the application. Assuming that many of the applications will run in one cloud or another, it means that those databases will need to have built-in support for different cloud environments. In addition, they will need to leverage dynamic code shipping to pass code with the data, and in this way allow complex processing with minimum network hops.

In-memory databases and data grid are the closest candidates to drive next-gen flash databases

RAM and flash-based devices have much more in common than flash and hard drive. In both RAM and flash, access time is fairly low and wasn’t really the bottleneck; in-memory databases use direct access to the RAM device through key value interface to store and index data.

Those factors makes in-memory based databases more likely to fit into the next generation flash databases.

The combination of in-memory databases and data grid on top of flash devices also allows the system to overcome some of the key capacity and cost per GB limitations of an in-memory based solution. The integration of the two will allow an increase in the capacity per single node to the limit of the underlying flash device rather than to the limit of the size of RAM.

The overall architecture of the integrated solution looks as follows (Source: Gartner):

 

 

As can be seen in the diagram above, the IMC (in-memory computing) layer acts as the front end to the flash device and handles the transactional data access and stream processing, while the flash device is used as an extension of the RAM device from the application perspective.

There are basically two modes of integration between the RAM and flash device.

LRU Mode – In this mode, we use the RAM as a caching layer to the flash device. The RAM device holds the “Hot,” i.e. the most recently used, and the flash device holds the entire set.

Pros: Optimized for maximum capacity.

Cons: Limited query to simple key/value access

Fast Index Mode – In this mode, all the indices in RAM and the data in flash holds the entire set.

Pros: Supports complex querying, including range queries and aggregated functions.

Cons: Capacity is limited to the size of indexes that can be held in RAM.

In both cases, the access to flash is done directly using key/value interface and not through disk drive interface. As both RAM and flash drives are fairly fast, it is simpler to write data to the flash drive synchronously and in this way avoid potential complexity due to inconsistency.

It is also quite common to sync the data from the IMC and flash devices into an external data store that runs on a traditional hard drive. This process is done asynchronously and in batches to minimize the performance overhead.

Direct flash access in real numbers

To put some real numbers behind these statements, I wanted to refer to one of the recent benchmarks that was done using an in-memory data grid based on GigaSpaces XAP and direct flash access using a key/value software API that allows direct access to various flash devices. The benchmark was conducted on several devices as well as on private and public cloud-based services on AWS. The benchmark shows a ten times increase in managed data without compromising on performance.

 

 

 

For full details of the benchmark, see the following white paper:

XAP MemoryXtend -Massive Application Storage Capacity for Real-Time Applications

 

 

 

Notes from Big Data Business Challenges Panel Discussion

Most of the Big Data topics that I covered in the past were focused mostly on the technical aspects of running Big Data.

In this post, I want to take a different route and discuss the business challenges based on a panel discussion that I had the pleasure of moderating.

 

 

Is Big Data a Big Hype?

One of the first questions that came up in the discussion is whether Big Data is over-hyped due to over-use in marketing content. Many of those who claim to provide either a Big Data solution or implement Big Data are just using the term to re-brand their product or offering.

 There was consensus across the panel that while Big Data is probably over hyped like any big trend, it does represent a real change in both technology and business. Let me explain:

In a pre-Big Data world, Big Data lived in expensive and closed data warehouses, where many of the solutions were often tied to specific, high end hardware. In addition, the term “warehouse” in itself meant that most of the data had little exposure and impact on the business.  Today, Big Data is everywhere and can be done at a fraction of the cost of the traditional data warehouses. With the addition of cloud the barrier to running Big Data has become significantly lower and now every startup can run Big Data almost at the scale of Google.

Having said that, the actual adoption of Big Data tends to vary quite significantly per industry with telecommunications (67%), consumer goods (57%) and financial services (52%) industries as the leaders in recognizing that Big Data can greatly improve their understanding of customer needs according to a recent Hitachi Data Systems Corporation (HDS) study  .The same study indicates that more than 60% of the firms in the financial services and consumer goods industries haven't started any Big Data programs. Healthcare and life science are lagging further behind; 72% of them haven't started any Big Data programs. This means that the perspective of whether Big Data is real or just marketing hype tends to vary as well depending on the industry.

 

How do you make sense out of your Big Data? Do we need a new role for Chief Data Officer?

Many of the questions in the panel had to do with how to make sense of the data.

One of the common strategies to make sense of the data is to assign a data scientist or Chief  Data Officer, expecting them to deal with the data and offer useful insights. Another common strategy is to put all the data in something like Hadoop with the hope that having the data in hand will provide a variety of interesting insights that will change the way we operate.

Putting the data in Hadoop or assigning a specialized data analyst are often just the beginning of a process.

Eyal Veitzman from Wix described how they deal with the challenge. To really make sense of your data, you first need to change the organizational structure in which the business, developers and data analysts will be part of the same group. In this way you can turn insights into action and more importantly have a faster feedback loop that will allow you to iterate quickly.  

 

What is the business value behind Big Data?

The business value behind Big Data comes in different shapes and forms depending on where you apply it in your system. Insightera uses Big Data to capture user interaction and increase the conversion of users using personalized responses. Wix uses Big Data to optimize their user conversion and satisfaction.  Nice Systems uses Big Data to improve their call center user satisfaction. PayPal is using Big Data in all sort of shapes and forms to both detect fraud as well as to optimize their operations. Check Point uses Big Data to collect data from all their gateways and in that way keep their gateways protected from new virus attacks

So how much can we gain from applying Big Data practices in our organization?

According to the Hitachi Data Systems Corporation (HDS) study, Big Data can help companies' revenue by 25%. This applies mostly to companies that use Big Data to optimize their internal operations and products, like most of the companies in the panel. That is obviously more true for companies that sell Big Data insights as part of their business, as in the case of Insightera.

Is there a good visualization tool for Big Data?

Big Data visualization is a tough challenge and there is no simple answer or single framework that fits them all. Using infographic style can help to visualize data in a way that makes it easier to capture lots of data into a single, readable report using a variety of visualization dimensions, such as fonts, colors, graphics, etc. There are many open source frameworks, such as Jaspersoft and many SaaS-based frameworks, aimed to provide specialized reporting and visualization tools. A useful reference on that regard can be found here here: The 36 best tools for data visualization.

Having said that, the important thing is to decide what and how to measure. Quite often, the number of Key Performance Indicators (KPIs) must be limited to a fairly small number.

The second important thing is to realize that as humans, we are not designed to capture and process vast amounts of information and therefore, visualization tools provide only a small aid to a much bigger challenge.

In summary, there is a dangerous slippery slope with Big Data visualization; fairly quickly we can flood our users with many graphs and numbers to a point where what we actually get is a dimension return and more complexity and confusion than clarity. Those who had experience with analytics tools such as Google Analytics are probably familiar with that experience. On the other hand, narrowing down the key performance indicators into a small number of KPIs makes the visualization challenge simple.

 

Moving from Reports to Actions

The key in capturing most of the value of Big Data is to turn Big Data into actions. In the example of Insightera, actions means presenting a user with the most relevant information on a website. In the case of Nice Systems, actions would refer to routing the caller in a call center to the right agent based on the caller’s profile and history of behaviour. In the case of GigaSpaces, this would mean constantly adjusting our provisioning system to detect failures and auto-heal a production environment when something goes wrong.

To get to that level we need to apply automation practices on how we collect, analyze and execute actions out of our data. We also need to be able to process data in real time to reduce the time to action.

Final words: How do make our data instantly actionable?

It is quite clear that the end game of any Big Data initiative is to figure out a way to make it instantly actionable. One of the big challenges on that regard is that quite often we need to compromise between scale and speed. In fact, many of the Big Data systems (Hadoop included) use batch processing to be able to process large amounts information, meaning that the actual insights, and thus actions, from that data will be available only when the batch cycle has been completed.

To make Big Data instantly actionable we need to figure out a way in which we can process vast amounts of data in real time and still keep the cost down.

During the panel discussion, PayPal described their use of In-Memory based architecture to handle this challenge. Still, pure In-Memory-based solutions tend to be fairly expensive and limited in capacity comparing to disk based alternatives. So the next big thing IMO is to devise a way in which we can get In-Memory speed at a cost that is closer to Disk. This is where new advancements in SSD have the potential to disrupt the entire way in which we manage and process Big Data, by providing high capacity durable RAM based out of SSD that can reach terabytes capacity per single machine. The fact that it can be durable means that we could rely completely on In-Memory databases to manage and store larger chunks of our Big Data with no need to offload or reload the data to and from external disks. I also expect that this will lead to the rise of a new generation of In-Memory databases that will be designed primarily for this new kind of durable RAM devices.

 

About the Panel

The Big Data Business challenges panel  was led by the CICC and in collaboration with IGTCloud, focusing primarily on the business challenges side of Big Data. The event was held simultaneously in California and Israel and was hosted by eBay. The panel included a range of industry experts in Big Data from a wide range of companies and industries and included Allen Kamer, Chief Commercial Officer, Optum Analytic (Co-founder of Humedica that was acquired by Optum Health), Doron Simon, Head of Cloud, Business Development, Nice Systems, Mickey Alon, Co-Founder & CEO, Insightera (Acquired by Marketo), Dr. Nachum Shacham, Principal Data Scientist, PayPal, Eyal Veitzman, Director of Business Operations, Wix and Ron Davidson, Head of Security Services, Check Point.

I had the special honor to organize and moderate the panel together with Ido Sarig, General Manager, IoT Solutions Group while Wind River moderated the panel in California.

 

Useful References

• Big data can help companies' revenue by 25%

Telecommunications (67%), consumer goods (57%) and financial services (52%) industries are leaders in recognizing that big data can greatly improve their understanding of customer needs. However, more than 60% of the firms in the financial services and consumer goods industries haven't started any big data programs. Healthcare and life science are lagging further behind; 72% of them haven't started any big data programs. http://timesofindia.indiatimes....­

• Fast Data hits the Big Data fast lane

The usual factors: the same data explosion that created the urgency for Big Data is also generating demand for making the data instantly actionable. Bandwidth, commodity hardware and, of course, declining memory prices, are further forcing the issue: Fast Data is no longer limited to specialized, premium use cases for enterprises with infinite budgets. Not surprisingly, pure in-memory databases are now going mainstream: ..

..Potential use cases for Fast Data could encompass:

• A homeland security agency monitoring the borders requiring the ability to parse, decipher, and act on complex occurrences in real time to prevent suspicious people from entering the country
• Capital markets trading firms requiring real-time analytics and sophisticated event processing to conduct algorithmic or high-frequency trades
• Entities managing smart infrastructure which must digest torrents of sensory data to make real-time decisions that optimize use of transportation or public utility infrastructure
• B2B consumer products firms monitoring social networks may require real-time response to understand sudden swings in customer sentiment
• The 36 best tools for data visualization

A useful reference for popular visualisation frameworks 

Industry Survey Shows: Organizations Turn to Stream Processing for their Big Data Needs

In a stream processing model, data is processed as it arrives to the Big Data system. With a batch processing model, only once the data is stored does the system run a variety of batch analytics - typically through a map/reduce style of processing.

 Facebook and Twitter are good examples of stream processing models in use today, as I outlined in a previous post entitled Facebook’s vs Twitter’s Approach to Real-Time Analytics.

 According to a recent survey of approximately 250 respondents, there is a trend moving towards stream processing in order to speed up analytics, with a significant increase in the popularity of this model when compared with last year. The number of organizations planning to use stream processing in 2014 has more than doubled (24%) from last year’s amount (10%).

This goes hand in hand with the fact that real-time analytics is also becoming more mainstream, as I pointed out in my 2014 Predictions.

Another interesting data point from the survey is that roughly 43% of the respondents defined real-time as sub-second and 42% defined it as sub-minute. This difference is quite interesting and could lead to different approaches for implementing stream processing.

For example, real-time is defined differently by Facebook and Twitter. In the case of Twitter, they chose Storm as their event processing engine, allowing them to process events at a sub-second resolution. Meanwhile, Facebook defines real-time at a 30 second batch window, choosing a logging-based approach to fit their need.

Based on the survey, it appears that both approaches are valid and can be applied in correlation to the degree of real-timeliness of your analysis.

 

Facebook Log-Centric Stream Processing	Twitter Event-Centric Stream Processing

In-Memory Data Grids become a more popular choice for real-time processing

According to the survey, 64% of the respondents indicated that they plan to combine In-Memory-based solutions for delivering their real-time analytics processing. This is also consistent with last year survey done by Ventana Research.

 I believe that will see an even bigger movement in this direction, as the cost/performance ratio of In-Memory Data Grids and In-Memory Databases would go down significantly with the combination of RAM and Flash devices, which together provide a fairly compelling solution from both performance and cost ratio to that of other disk-based alternatives.

 

Big Data in the Cloud increased to 56%

New developments in cloud infrastructure, such as the support for bare metal in the new OpenStack releases, as well as support, high memory, flash disk, etc, are removing many of the technical barriers for running I/O intensive workloads like Big Data in the cloud.

Indeed, according to the survey, there is a significant shift towards the use of Big Data in the cloud with 55% of organizations either using or planning to run their Big Data in the cloud in 2014.

 

Where do we go from here? How does this effect GigaSpaces’ future roadmap?

There are multiple areas in the GigaSpaces roadmap that aim to address this demand. 

• Real-Time Processing through Storm Integration - In this project, we integrate Storm on top of a Memory backend for both stream processing and data-store.
• Support for Flash Disk - The integration with flash disk is planned for our upcoming XAP release and will include support for SanDisk, Fusion I/O and other flash disk devices.
• Big Data in the Cloud - through Cloudify - We've been continuously extending our Big Data portfolio support and recently added support for Storm and Cognos, adding to our existing support for Hadoop, MongoDB, Cassandra, ElasticSearch, etc.

 

References

• GigaSpaces Annual Big Data & Cloud Survey 2013
• Eight Cloud and Big Data Predictions for 2014
• GigaSpaces and Storm Part 2: XAP Integration
• GigaSpaces And Storm Part 1: Storm Clouds
• Big-Data-Real-Time-Performance - Enjoying both worlds with one Architecture!
• 2012 Big Data Survey Report
• In Memory Computing (Data Grid) for Big Data

Eight Cloud and Big Data Predictions for 2014

 

Prediction has always been a fun exercise. It forces you to take a step back from your day to day activities, look at the market "crystal ball" and figure out what the future looks like. This year I found this exercise to be particularly difficult as the amount of new innovation and conflicting trends that are taking place at the same time can be very confusing.

After spending quite some time reading market analyses from different sources (see references at the end) and compiling them with all the things I've seen and heard throughout the year, I came to the following observation on how 2014 will look for Cloud and Big Data.  I'm happy to exchange thoughts on that regard and learn how you envision 2014 will look.

 

Public and Private Cloud in 2014

1. Amazon continues to distance itself while Google and Microsoft are catching up

Amazon continues to lead in public cloud and distance itself from the rest of the market. This year it reached 5 times the size of other cloud vendors combined, as noted in a recent Gartner report. Google and Microsoft are slowly closing the gap as the closest alternatives, but at a much slower pace than one would expect given the significant investment of both companies in this area.

2. 2014 will be the year of Enterprise Clouds

Traditional enterprise players, such as IBM, HP, Cisco and Red Hat, continue to fight for the remaining share of the market, mostly around enterprise adoption. Yet, enterprises have been slower to embrace and execute on private cloud strategy. Part of the reason for slow adoption is the gap between the solution provided by most of the regular contenders - who are still competing on selling an end to end story - and the reality that most enterprises are looking for Open and Hybrid cloud strategy, especially given that there is no clear winner.

3. OpenStack will be the most popular choice for Enterprise Clouds in 2014

OpenStack is now high on the radar for Enterprise Cloud, mostly threatening VMware's strong leadership in that domain. Most of the main contenders have embraced a strong OpenStack strategy, including Red Hat, Ubuntu, Suse, HP, IBM and Cisco, with Cisco taking a surprising leading position over IBM and HP, according to a recent Forrester survey.

VMware's response of embracing OpenStack is still questionable, as the transition to OpenStack is not only about technical integration, but also involves a big shift in the value chain. More enterprises are less keen on paying high cost for hypervisor licenses, which is to date the main revenue channel in the VMware pie.

4. Native OpenStack alternatives will disrupt many of the existing cloud solutions

The rapid adoption of OpenStack will also disrupt many of the existing cloud solutions that were built in a pre-OpenStack world. New solutions that take a more native-to-OpenStack approach will pop up and replace many of the current solutions. A good example of this is Project Solum, which aims to provide a native alternative to existing PaaS solutions, such as CloudFoundry and OpenShift as I outlined in this recent post. I expect that will see more of these disruptions expanding to other frameworks in 2014.

5. Orchestration & Automation will be the next big thing in 2014

Having said all that, the remaining challenge of enterprises is to break the IT bottleneck. This bottleneck is created by IT-centric decision-making processes, a.k.a "IaaS First Approach," in which IT is focused on building a private cloud infrastructure - a process that takes much longer than anticipated when compared with a more business/application-centric approach. One of the ways to overcome that challenge is to abstract the infrastructure and allow other departments within the organization to take a parallel path towards the cloud, while ensuring future compatibility with new development in the IT-led infrastructure.

DevOps has definitely been a key example for a business-led initiative that determines the speed of innovation and, thus, competitiveness of many organizations. The move to DevOps forces many organizations to go through both cultural and technology changes in making the biz and application more closely aligned, not just in goals, but in processes and tools as well.

Enterprises with legacy environments and customers take a two-step approach. They first tackle continuous delivery - automating the packages of their software deliverable and keeping tighter control over new production rollout. Once enterprises feel comfortable with their process and environment, they will automate the entire deployment process into production.

Configuration management, orchestration and workflow automation become key enablers in enterprise transition to cloud, and will gain much attention in 2014 and 2015. Again, Amazon has already recognized that need, introducing into the space a new offering called OpsWorks. It is only a matter of time until will see similar offerings embedded with other cloud providers in both public and private clouds.

As the focus moves to orchestration with greater focus on standardization of deployments and packages, DSL and API become equally important. Existing standards, such as TOSCA and CAMP, will undergo massive modifications to make them simpler to use and fit the cloud environment.

Big Data Predictions

There are many advancements happening within the Big Data/NoSQL domain that I'm not going to touch on. I will focus on two main areas that are close to my work.

6. 2014 will see a major increase of Big Data in the Cloud

Cloud infrastructure is closing the gap with the existing data center, as we see the emergence of support for Bare Metal, High CPU, High Memory and Flash-Disk. Many cloud providers are already offering Big Data as Service, such as Elastic Map Reduce and Redshift, and are continuously expanding their offering on that regard.

This advancement in cloud infrastructure removes almost all of the technical barriers for running I/O intensive workloads, such as Big Data analytics, on the cloud. I expect that in 2014, running Big Data analytics on the cloud will become the first choice for any new project, while the use of Big Data in non-cloud environments will be minimized to only niche use cases with extreme regulations and security constraints.

7. In-Memory Data backed by flash-disk will become a popular choice for Real-Time Big Data analytics
Flash disk pricing changes the economics behind the cost/performance ratio of disk-based solutions, making it possible to achieve the performance of an In-Memory-based solution at a price closer to a disk-based solution. So far, attempts have been made to use flash disks as a fast disk alternative to magnetic drives. This path inherits many of the limitations of a disk-based drive and, therefore, doesn't capture the full potential of flash disk which, like RAM, can provide highly parallel access to data.

At the same time, memory-based solutions like In-Memory Database or In-Memory Data Grid cover a small niche in the Big Data ecosystem, mostly due to the high cost/performance ratio.

I believe that the combination of flash drive, In-Memory data-grids and databases at the front-end change that dynamic and will make memory-based solutions backed by flash disk much more attractive to a bigger niche, specifically as it relates to real-time analytics of Big Data.

In some cases, the combination of memory-based solutions is also integrated with existing Big Data frameworks and, thus, provides seamless performance acceleration. A good example is GigaSpaces' integration with Storm and GridGain's integration with Hadoop.

8. Real-time analytics turns mainstream

The most interesting indication that real-time analytics is becoming mainstream is Amazon's support for real-time analytics, with many of the existing analytics solutions providing real-time analytics capabilities as built-in parts of their reports. Another good example of that is Google Analytics Real Time View.

New disruptive forces in Cloud worth watching in 2014
Disruptive technologies change the market landscape in ways that are difficult to anticipate by their very nature. Rather than predicting their impact, I felt it would be simpler to list them.

• Networking - The networking segment of IT is experiencing a major disruption as of late with the move to Software Defined Network, OpenStack Neutron project and Network Function Virtualization. These developments will change networking from not only a technology perspective, but they are also driving a completely different business model which will be based on utilization or a subscription-based model.

• Linux Container - Linux containers are gaining interest both as light-weight software packaging as well as VMS's. The most commonly used use case for Linux containers has as an underlying container for PaaS. With the introduction of new projects, such as Docker - which makes the Linux container easy to use, we will see wider and more pervasive use of containers as high performance virtualization, as packaging tools in continuous deployment scenarios, etc.

• Bare Metal Cloud - Bare metal cloud has been a small niche in the cloud space, mostly due to the fact that it is usually offered in a static configuration setup. Bare metal clouds provide the same degree of elasticity to bare metal devices. With OpenStack, for example, you can spawn a new bare metal device just as you would provision any other VM. The development would reduce one of the last remaining barriers for bringing mission critical applications to the cloud.

• OpenStack as an Innovation Accelerator - Many items on the list of disruptive technologies that I listed above are not that new, and to a certain degree, have existed for years, like in the case of Linux Container. Often, a disruptive force needs to gain certain critical mass before it can break into massive adoption. OpenStack creates an ecosystem that provides a platform for many users to integrate new technologies in a way that could be consumed by end users immediately and that plays a major role in the acceleration of adoption of many of those disruptive technologies.

Where do we go from here?

There are many individual technologies and trends that have a disruptive force behind them. Having said that, I think that it is the intersection between those technologies that holds the most promising potential. Here are few examples:

• Big Data and the Cloud Application Orchestration - Big Data analytics for operational information could serve as the new "brain" behind a new class of orchestration engine that would combine artificial intelligence decision-making based on trends and historical analysis and handle complex failure and scaling scenarios automatically.

• Putting Network and Applications together - Putting network and applications together holds a lot of promise in the way we scale applications across regions and multiple sites, as well as how we control application SLA's in a shared environment. For example, giving priority to customer-facing services versus batch analytics or optimizing the network routing based on the locality of the data, etc.

Real-Time Big Data with Storm, Cassandra, and XAP

Real-time Big Data is becoming more and more common in Big Data systems.

One of the most common frameworks used for running real-time Big Data system is Twitter Storm backed by a NoSQL database, as shown in the diagram below:

 

In this architecture, Storm is used for processing data as it comes in and the NoSQL data store is used as the output for that processing as well as for reference data storage.

Challenges

This architecture presents a couple of challenges:

• Performance - Storm runs in-memory, and is therefore set to process large volumes of data at in-memory speed. However, a typical storm architecture needs to interface with a data source for its input and a data source for its output. In this context, the overall performance is now determined not by Storm itself but by the data input and output sources. Quite often these interfaces rely on file-based message queues for streaming, and NoSQL for data storage. These interfaces are at least an order of magnitude slower than Storm itself, and therefore become the limiting factor.
• Complexity - Storm itself consists of several moving parts, such as a coordinator  (ZooKeeper), state manager (Nimbus), and processing nodes (Supervisor). The NoSQL data store also comes with its own cluster management. In addition, a typical Big Data system comes with more components such as the application front end, a reporting database, and more. This makes the process of managing the deployment, configuration, fail-over and scaling of such a system quite complex.

Meeting the Performance Challenge

Given that Storm itself runs in memory, it only makes sense that in order for Storm to run at maximum capacity the streaming and data store interfaces should be implemented in-memory as well, as shown below:

As you can see in the above architecture, we added two interfaces that rely on a built-in Storm plug-in - one for the data inputs and the second for data output. In both cases, the underlying implementation is memory-based, and thus removing the impedance that the previous architecture included.

You can read the full details on how this integration works, including a code example, in DeWayne Filppi's post on integration of in-memory computing with Cassandra.

The second optimization that can be added is to the NoSQL data store.

As most analytics applications tend to be read-mostly, we can speed up access to the NoSQL data store using an in-memory local cache. This architecture speeds up read performance between 10 to 1000 times as outlined by Shay Hassidim in his post on real-time Big Data performance.

Meeting the Complexity Challenge

To meet the complexity challenge, we will use a DevOps automation approach using Cloudify in conjunction with Chef or Puppet.

With this approach, we wrap every service with a deployment recipe that abstracts the underlying details on how to manage Storm, Cassandra, and our in-memory data store. Cloudify uses these recipes to automate the deployment of the entire stack. In this way, you only need to interact with Cloudify for the deployment, configuration, and also scaling and fail-over of your stack rather than separately manage each individual component. In addition, we use Cloudify to abstract the underlying infrastructure. This enables us to use the same deployment recipe across different environments. One for testing, another for production, etc. We can also use the same approach to deploy our apps based on the type of workload. We can use a public cloud for running sporadic workload, and thus leverage the elasticity of the cloud and also enable us to create and scale the environment as well as rip it off completely when done. At the same time, we can choose bare-metal machines for I/O intensive workloads, and so on.

You can read more on how you can set Storm using Cloudify on DeWayne's post on Storm and the cloud.

Cloudify comes with a rich set of built-in recipes for other Big Data services as well, making the integration process an out of the box experience. The main ones are listed below:

Final Notes - Optimizing without Code Change

Many real-time Big Data systems are now based on Twitter Storm and a NoSQL data store such as Cassandra.

In this post I tried to outline how we can optimize this architecture by addressing two areas: performance, and management.

The good news is that all this is possible to achieve seamlessly, without any code change. Let me explain:

In the case of Storm we used built-in plug-ins - Spout for streaming and Trident as the data store interface. In this way we can simply plug our memory-based plug-ins under these two integration points. All our existing Storm business logic would work pretty much the same.

We use the same plug-in approach to integrate our in-memory data store with our NoSQL data. This integration makes the data flow between our real time streaming to our NoSQL storage fairly transparently. In addition, it allows us to plug in different NoSQL data stores such as MongoDB, CouchBase, etc., giving us another degree of flexibility.

The same applies to our management layer. Existing Storm and NoSQL deployments are wrapped into plugged in recipes, and don't require any specific code changes.

Not Mutually Exclusive

We don't have to implement all of the optimizations to gain the benefit of this architecture. Each of the optimization points can be plugged in  independently of the others. For  example, if your main pain point is complexity, you can start by only by adding the DevOps automation first. At the same time if your main pain point is performance, you can use the memory-based plug-ins to speed up processing.

The other advantage of the combined architecture that I haven't discussed in this post is that it provides more flexibility in the degrees of consistency and availability in which you set your system, as I outlined in one of my recent posts on the subject In Memory Computing (Data Grid) for Big Data.

References:

• Big Data Real-Time Performance
• GigaSpaces and Storm Part 2: XAP Integration
• GigaSpaces And Storm Part 1: Storm Clouds
• In Memory Computing (Data Grid) for Big Data
• Check out how GigaSpaces XAP and Cassandra integration works

In Memory Computing (Data Grid) for Big Data

The Drive for In Memory Computing?

Memory-based databases and caching product has been available for over a decade. However, so far they have been used in a fairly small niche in the data management solution market.

There have been multiple advances in the industry in both hardware and software architecture, which makes memory-based computing more relevant today than in the past, as outlined in the diagram below.

In a nutshell, the availability of new classes of hardware with the support of 64bit CPU can now support 2TB on a single device. In addition, the advances in software architecture and solutions toward distributed architecture and cloud make it easier to utilize these new hardware capabilities.

In-Memory Computing

In many ways, In-Memory Computing is a close relative of In-Memory Databases. As with many databases, it was designed to enable all the data management aspects that are often expected from traditional databases, such as queries and transactions, with the difference that the data is managed on RAM devices and not disks and thus comes with potentially x1000 better performance and latency according to various benchmarks.

The main differences between the traditional in-memory databases and in-memory-based-computing are that In-Memory-Computing is:

1. Designed for distributed and elastic environments

2. Designed for In-Memory data processing

Executing the code where the data is:

The fact that we can store our data in the same address space as our application is the biggest gain.

Unlike disk and even flash disk devices, we can access our data by reference and thus perform complex data manipulation without any serialisation/de-seralization overhead. With the new class of dynamic languages such as Java, JavaScript, JRuby, and Scala, it is also significantly easier to pass complex logic over the wire and execute it on a remote device.

In-Memory Computing  relies heavily on that capability, and exposes a new class of complex data processing capabilities that fits well with the distributed nature of the data through real-time map/reduce and stream-based processing as a core element of its architecture.

The Big Data Context

According to our recent survey more than 70 percent of the responders said their business requires real-time processing of big data -- either in large volumes, at high velocity, or both.

Interestingly enough another survey by Ventana Research indicated that one of the biggest technical challenges in Big Data is the lack of real-time capabilities (67%). The report also indicated that many of the organizations are planning to use in-memory databases (40%) as part of their Big Data stack. This places the In-Memory Database as a second choice, before specialised DBMS (33%) and Hadoop (32%).  One of the conclusions this survey leads to is that organizations see Data Warehouse Appliances and In-Memory Databases as one of their first choices to deal with the lack of real-time capabilities.

 

No One-Size-Fits-All Solution

While In-Memory Databases fit well in the planned Big Data stack, it is clear that  theres no one-size-fits-all solution. The Big Data stack is going to be based on a blend of various technologies, each covering different aspects of the challenges of Big Data, from batch to real-time, from vertical to horizontal solutions, etc.

The question is: How do we integrate them all, without adding even more complexity to an already complex system?

In this post I will focus specifically on one of the approaches the we used for combining In-Memory Computing together with other Big Data solutions, such as Hadoop and Cassandra.

Putting In-Memory Computing Togther with a NoSQL DB

One of the main motivations to integrate in-memory-based solutions with a NoSQL DB is to reduce the cost per GB of data.

Putting our entire data purely in-memory can be too costly especially for data that were not going to access frequently.

There are various approaches to doing this -- the approach we found most useful is a two-tier approach.

With the two-tier approach the In-Memory Computing systems run separately from the NoSQL database, which acts as the long-term storage.

The Challenge

The main challenge with this approach is the complexity that is associated with synchronising two separate data systems. Specifically, how to ensure that data that is written into the front end In Memory Computing engine gets populated into the NoSQL database reliably, and vice versa.

The Solution

To deal with this challenge we used a similar approach to the one that we used before with RDBMS. Have an implicit plug-in that gets called whenever new data is written and populate it into the underlying database. The plug-in also deals with pre-loading of the data when the system starts. In the RDBMS world we used frameworks like Hibernate to deal with the implicit mapping of the data between the in-memory front end and the underlying database.

Working with Dynamic Data Structure (a.k.a Document Model)

When we tried to apply the same approach with NoSQL databases we could no longer rely on Hibernate as the default framework for mapping the data between the two data systems, as NoSQL databases like Cassandra tend to be fundamentally different from traditional RDBMSs. The main difference is the use of dynamic data structures, a.k.a the Document Model.

To deal with dynamic data structure we added the following hooks:

• Introducing new documents and objects: Users can choose to write or load data in various forms -- Document for non-structured data or Objects or POJOs for structured or semi-structured data.

• Introducing and loading new meta data: To map the data to and from the NoSQL database we also added the ability to introduce new meta data and load the meta data of the object before the actual data is loaded.

• Introducing new indexes: In NoSQL databases you cannot effectively access data that is not indexed. For that purpose we included the ability to introduce indexes on the fly.

You can see how this works specifically with Cassandra in this post -- From The Memory Grid to Cassandra

The Benefit -- Best of both worlds

The main benefit of this two-tier approach is that it allows us to take the best of the memory and file-based approaches without adding too much complexity. The two tiers behave and work as one data system from an end-user perspective. The bits and pieces of how the two system get synchronized is carved out of the system.

Furthermore, the two-tier approach opens up a new degree of flexibility in how we design our Big Data system.

New Degree of Flexibility

If we look at the entire data flow from the point in which a user interacts with our system (this is where we expect low latency and high degree of consistency to our analytics systems; where we record those actions and analyze them, latency is less of an issue and we can also relax some of the consistency constraints), we can see that each stage in our data processing has different  onsistency, latency and performance requirements.

With the two-tier approach there is more flexibility in dealing with those different requirements and still keep everything working as if it were one big system from a usability perspective. Here are a few examples of how this setup can work:

• Consistent data flow from Real-Time to Batch: The integration enables us to handle real-time data processing at in-memory speed and deal with more long-term data processing through the underlying database.
• Performance & latency: The In-Memory Computing system can handle the event processing before the data gets into the database. Or, another approach is to keep the last day (or days) in-memory and the rest of the data in the NoSQL database.
• Mixed consistency model: The In-Memory Computing system is often built for extreme consistency where NoSQL databases often work best with eventual consistency. Usually, the consistency requirements are more relevant at the front end of the system, and becomes less relevant as the data gets older. The combined approach enables us to set our front end for extreme consistency and back end for eventual consistency.
• Deterministic behavior: In many cases, we must ensure that a given set of data can be served under constant performance. Many databases use an LRU based cache to optimise the data access. The limitation of this approach is that the speed at which we can access our data becomes non-deterministic as we often do not control which data is served through the database cache; thus, in some cases we will get a fast response time if we hit the cache and in other cases the same operation can be 10 times longer if we miss the cache. By splitting our in-memory data from our file-based storage we get more explicit control over which data is served at in-memory speed and which data is not, thus ensuring consistent access to that data.
• Faster ETL: By front-ending our Big Data storage with In Memory Computing we can also speed up the time it takes to pre-process and load data to our long-term data system. In this context, we can push the filtering, validation, compression and other aspect of our data processing into memory before it goes into our long-term databases.

Final Words

Big Data systems are complex beast and it is clear that the one-size-fits-all approach doesn't work.

On the other hand, having too many data systems increases the complexity of managing our Big Data system almost exponentially; and our ability to ensure consistent behaviour, data integrity and reliable synchronization across the varous systems becomes an almost un-manageable task if done manually.

Adding real-time capabilities to our Big Data system is a classic area where the kind of integration described in this article is needed. The integration between In-Memory and File-Based approaches as two separate tiers also introduces additioanl areas of flexibility in how we can handle often contradictory requirements such as consistency, scale, latency, and cost. Instead of trying to come up with a least common denominator, we can optimize each tier to the area it fits best.

References:

• From The Memory Grid to Cassandra
• GigaSpaces Big Data Survey
• The Challenges of Big Data Survey (Ventana Research)
• In Memory Computing (Wikipedia)
• http://www.ventanaresearch.com/uploadedFiles/Content/Landing_Pages/Ventana_Research_Big_Data_Benchmark_Research_Presentation.pdf

 

Making Hadoop Run Faster

One of the challenges in processing data is that the speed at which we can input data is quite often much faster than the speed at which we can process it. This problem becomes even more pronounced in the context of Big Data, where the volume of data keeps on growing, along with a corresponding need for more insights, and thus the need for more complex processing also increases.

Batch Processing to the Rescue

Hadoop was designed to deal with this challenge in the following ways:

1. Use a distributed file system: This enables us to spread the load and grow our system as needed.

2. Optimize for write speed: To enable fast writes the Hadoop architecture was designed so that writes are first logged, and then processed. This enables fairly fast write speeds.

3. Use batch processing (Map/Reduce) to balance the speed for the data feeds with the processing speed.

Batch Processing Challenges

The challenge with batch-processing is that it assumes that the feeds come in bursts. If our data feeds come in on a continuous basis, the entire assumption and architecture behind batch processing starts to break down.

If we increase the batch window, the result is higher latency between the time the data comes in until the time we actually get it into our reports and insights. Moreover, the number of hours is finite -- in many systems the batch window is done on a daily basis. Often, the assumption is that most of the processing can be done during off-peak hours. But as the volume gets bigger, the time it takes to process the data gets longer, until it reaches the limit of the hours in a day and then we face dealing with a continuously growing backlog. In addition, if we experience a failure during the processing we might not have enough time to re-process.

Speed Things Up Through Stream-Based Processing

The concept of stream-based processing is fairly simple. Instead of logging the data first and then processing it, we can process it as it comes in. 

A good analogy to explain the difference is a manufacturing pipeline. Think about a car manufacturing pipeline: Compare the process of first putting all the parts together and then assembling them piece by piece, versus a process in which you package each unit at the manufacturer and only send the pre-packaged parts to the assembly line. Which method is faster?

Data processing is just like any pipeline. Putting stream-based processing at the front is analogous to pre-packaging our parts before  they get to the assembly line, which is in our case is the Hadoop batch processing system.

As in manufacturing, even if we pre-package the parts at the manufacturer we still need an assembly line to put all the parts together. In the same way, stream-based processing is not meant to replace our Hadoop system, but rather to reduce the amount of work that the system needs to deal with, and to make the work that does go into the Hadoop process easier, and thus faster, to process.

In-memory stream processing can make a good stream processing system, as Curt Monash’s points out on his research traditional databases will eventually end up in RAM. An example of how this can work in the context of real-time analytics for Big Data is provided in this case study, where we demonstrate the processing of Twitter feeds using stream-based processing that then feeds a Big Data database for the serving providing the historical agregated view as described in the diagram below.

Faster Processing the Google Way: Using Stream-Based Processing Instead of Map/Reduce

Due to a lack of alternatives at the time, in many Big Data systems today Map/Reduce is used in areas where it wasn't a very good fit in the first place.  A good example is using Map/Reduce for maintaining a global search index. With Map/Reduce, we basically rebuild the index, where it would actually make more sense to update it with changes as they come in.

Google moved large part of its index processing from Map/Reduce into a more real-time processing model, as noted in this recent post:

So, how does Google manage to make its search results increasingly real-time? By displacing GMR in favor of an incremental processing engine called Percolator. By dealing only with new, modified, or deleted documents and using secondary indices to efficiently catalog and query the resulting output, Google was able to dramatically decrease the time to value. As the authors of the Percolator paper write, ”[C]onverting the indexing system to an incremental system … reduced the average document processing latency by a factor of 100.” This means that new content on the Web could be indexed 100 times faster than possible using the MapReduce system!

..Some datasets simply never stop growing ..it is why trigger-based processing is now available in HBase, and it is a primary reason that Twitter Storm is gaining momentum for real-time processing of stream data.

Final Notes

We can make our Hadoop system run faster by pre-processing some of the work before it gets into our Hadoop system. We can also move the types of workload for which batch processing isn't a good fit out of the Hadoop Map/Reduce system and use Stream Processing, as Google did.

Interestingly enough, I recently found out that Twitter Storm came up with an option to integrate an in-memory data store into Storm through the Trident-State project. The combination of the two makes lots of sense and something were currently looking at right now so stay tuned.

 

References

• Build Your Own Twitter Like Real Time Analytics - A Step By Step Guide (Blog Post)
• Build Your Own Twitter Like Real Time Analytics - A Step By Step Guide (Online Tutorial)
• Real Time analytics for Big Data: Facebook's New Realtime Analytics System
• Beyond hadoop: fast queries from big data
• Traditional databases will eventually wind up in RAM
• Why the days are numbered for Hadoop as we know it

Related articles

Elephants and Eyeballs: Real-Time Big Data in the Real World

Cassandra on ACID

GigaOm: 'Hadoop's days are numbered...' Are they?

Hadoop Officially a Big Deal for Big Data

Real Time Analytics for Big Data: An Alternative Approach

Lately, we've been talking to various clients about realtime analytics, and with convenient timing Todd Hoff wrote up how Facebook's realtime analytics system was designed and implemented (See previous review on that regard here).

They had some assumptions in design that centered around the reliability of in-memory systems and database neutrality that affected what they did: for memory, that transactional memory was unreliable, and for the database, that HBase was the only targeted data store.

What if those assumptions are changed? We can see reliable transactional memory in the field, as a requirement for any in-memory data grid, and certainly there are more databases than HBase; given database and platform neutrality, and reliable transactional memory, how could you build a realtime analytics system?

Joseph Ottinger and I discussed this, and this is what we came up with.

A Summary of History

To understand what a new design might look like, it’s often useful to consider a previous design. This is a very short summary of Facebook’s realtime analytics system.

First, it’s based on a system of key/value pairs, where the key might be a URL and the value is a counter. Thus, there’s a requirement for atomic, transactional updates to a very simple piece of data. The difficulties come from scale, not from the focus of the system.

The process flow is fairly simple:

• A user creates an event by performing some action on the website. This generates an AJAX request, sent to a service.
• Scribe is used to write the events into logs, stored on HDFS.
• PTail is used to consolidate the HDFS logs.
• Puma takes the consolidated logs from PTail and stores them into HBase in groupings that represent roughly 1.5 seconds’ worth of events.
• HBase serves as the long-term repository for analytics data.

There are some questions around how PTail and Puma serve as scaling agents, and some of the notes around their use are still limited in scale – for example, one of the concerns is that an in-memory hash table will fill up, which sounds like fairly serious limitation to have to keep in mind.

A Potential for Improvement

There are lots of areas in which you can see potential improvements, if the assumptions are changed. As a contrast to Facebook's working system:

• We can simplify the design. If memory can be seen as transactional - and it can - we can use them without transforming them as they proceed along our analytics workflow. This makes our design and implementation much simpler to implement and test, and performance improves as well.
• We can strengthen the design. With a polling semantic, such systems are brittle, relying on systems that pull data in order to generate realtime analytics data. We should be able to reduce the fragility of the system, even while making it faster.
• We can strengthen the implementation. With batching subsystems, there are limits shouldn’t exist. For example, one concern in Facebook's implementation is the use of an in-memory hash table that stores intermediate data; the in-memory aspect isn’t a concern until you realize that the batch sizes are chosen partially to make sure that this hash table doesn’t overflow available space.
• We can allow deployments to change databases based on their requirements. There's nothing wrong with HBase, but it's got specific characteristics that aren't appropriate for all enterprises. We can design a system which you’d be able to deploy on various and flexible platforms, and we can migrate the underlying long-term data store to a different database if needed.
• We can consolidate the analytics system so that management is easier and unified. While there are system management standards like SNMP that allow management events to be presented  in the same way no matter the source, having so many different pieces means that managing the system requires an encompassing understanding, which makes maintenance and scaling more difficult.

What we want to do, then, is create a general model for an application that can accomplish the same goals as Facebook’s realtime analytics system, while leveraging the capabilities that in-memory data grids offer where available, potentially offering improvement in the areas of scalability, manageability, latency, platform neutrality, and simplicity, all while increasing ease of data access.

That sounds like quite a tall order, but it’s doable.

The key is to remember that at heart, realtime analytics represent an events system. Facebook’s entire architecture is designed to funnel events through various channels, such that they can safely and sequentially manage event updates.

Therefore, they receive a massive set of events that “look like” marbles, which they line up in single file; they then sort the marbles by color, you might say, and for each color they create a bundle of sticks; the sticks are lit on fire, and when the heat goes up past a certain temperature, steam is generated, which turns a turbine.

It’s a real-life Rube Goldberg machine, which is admirable in that it works, but much of it is still unnecessary if the assumptions about memory ("unreliable") and database ("HBase is the only target that counts") are changed. Looking at the analogy from the previous paragraph, there’s no need to change a marble into anything. The marble is enough.

A Plan for Implementation

Our design for implementation is built around putting data and messaging together. A data grid is a perfect mechanism for this, as long as it provides some basic features: transactional operations, push and pull semantics, and data partitioning.

A data grid does provide those basic features, or else it's not really much of a data grid; it'd be more of a cache otherwise.

With a data grid, then, the events come in as individual messages. When the user chooses an operation on the web site, an asynchronous operation would write the event, just as Facebook does today. However, instead of filtering and batching the events into various forms, the events are dispatched to waiting processes that perform many transactional updates in parallel.

There’s a danger that those updates might be slower than the generated events, if each event is processed sequentially. That said, this isn’t as much a problem as one might think; if data partitioning is used, then event handlers can receive partitioned events, which localizes updates and speeds them up dramatically.

In fact, you can still use batching to process events as a group; since the events would be partitioned coming in, the batch process would still be updating local data very quickly, which would be faster than individual event processing, even while retaining simplicity.

With this design, there is no overflow condition, because a system that’s designed to scale in and out as most data grids are will repartition to maintain even usage. If a data grid can’t provide this feature intrinsically, of course some management will be necessary, but finding data grids with this feature isn’t very difficult.

One other advantage of data grids is in write-through support. With write-through, updates to the data grid are written asynchronously to a backend data store – which could be HBase (as used by Facebook), Cassandra, a relational database such as MySQL, or any other data medium you choose for long-term storage, should you need that.

The memory system and the database - the external data store - work together. The in-memory solution is ideal for the realtime aspects, the events that affect now. The external data storage solution is designed to handle long-term data, for which speed is not as much of an issue.

A Discussion of Strengths

The key concept here is that event handling is the lever that can move the realtime analytics mountain. By providing a simple, scalable publisher/subscriber model, you simplify design; by using a platform that supports data partitioning, transactional updates, and write through capabilities, you gain scalability.

The data grid’s flexible query API means that events can literally react when data is available.

For a call center, for example, you want to immediately identify signals that show that the caller should be handled differently; imagine an ecommerce site that was able to determine immediately if a user was losing interest, and thus could respond appropriately, before the customer moves on.

With external processes and a long funnel for data, immediate-response capabilities are very difficult to implement, not just because of latency but because the data transformations tend to homogenize the data, instead of allowing rich expressions and flexible event types.

The data grid also has much richer support in terms of client applications. Instead of applications going through an API that focuses on a specific phase of the data’s life (for example, an API focused on HBase), you can focus on a generic API that can capture events at any point in their lifecycle, and from anywhere. An external monitoring process, then, can have the same immediate, partition-aware access to data that the integrated message-handling system does; adding features and analysis is just a matter of connecting a client to the data grid.

Here we have a quick demo that shows much of this in motion. We have a market analysis application, deployed into GigaSpaces XAP via our new Cloud deployment too, Cloudify; it uses an event-driven system to display realtime data, with a write-through to Cassandra on the back-end. The design is very simple, and demonstrates the principles we've discussed here - and can scale up and down depending on demand.

Final words

As real-time-analytics gets into mainstream application design, it becomes important to draw a blueprint on how to build this sort of system without re-inventing the wheel.

Todd Hoff (HighScalability) and Alex Himel (Facebook) provided a fairly detailed description on their solution and even more importantly they even shared the rationales that made them do things in certain ways.

One main difference in assumptions that lead to the different implementation strategies are in reliable memory for event processing, and in the use of passive data storage.

Another difference is that we had to to think of the solution as an easily cloneable solution and therefore a lot of attention was put on the simplicity of the runtime, packaging and management of the solution.

Yet another difference is that we couldn’t decide on a specific database as there isn’t a "one size fits all" solution – for certain customers, SQL would still be preferred choice and the fact that we can buffer the write to the database gives them more headroom while still allowing them to scale on writes.

I hope that this would lead to constructive dialogue on the various tradeoffs which will serve the entire industry...

References

• Video: Realtime Analytics for Big Data: A Facebook Case Study