Nati Shalom's Blog

Achieving 100% Uptime and 139% Increase in Holiday Online Sales: Lessons Learned

 

Background

You don’t need to be an expert to realize that a failure of an eCommerce site during Black Friday or Cyber Monday is a disastrous event, leading to huge loss in revenue and reputation for the retailer. As the share of eCommerce accounts increases to more than 8% of total US retail sales this year, the impact of failure becomes more significant - not just to the site itself, but on the the overall economy. A study on the subject, compiled by Joyent and New Relic, showed that 86% of companies experienced one or more episodes of downtime last holiday season. At the same time, 58% of customers will not use a company’s site again after experiencing site errors.

 

Another study by Radware measured not just the impact of downtime on eCommerce sites, but also the impact of slowness - an even more common and less measured metric.  According to this study a one-second delay correlates to:

• 2.1% decrease in cart size

• 3.5-7% decrease in conversions

• 9-11% decrease in page views

• 8% decrease in bounce rate

• 16% decrease in customer satisfaction

• a 2.2 slowdown equals a 7.7% conversion rate hit.

Meanwhile, KISSmetrics illustrated how page loads longer than three seconds lead to a 40% bounce rate.

Obviously there is enough business incentive here to invest in handling both the downtime and latency issues. Meanwhile taking a look at a typical retailer traffic (source: Akamai) during this season, we notice that the traffic spikes increase by at least 500%:

 

In this post I will share our specific experience and lessons-learned from the 2014 holiday season which turned out to be very successful.  I believe that the results below speak for themselves.

 

2014 Results

 

 

How We Achieved These Results

• Taking a preemptive approach - rather than reacting after a failure occurred- prevented failure in the first place.

• Common Causes: Most failures are the result of misconfiguration or capacity planning guesswork.

• Planning: Proactive tuning with continuous system monitoring produces predictable improvements at scale, eliminating risks from incorrect provisioning.

• Knowledge & Experience: eCommerce applications are complex and built from many subsystems. In many cases, an eCommerce organization does not have the expert skill-set in each of the subsystems. Having an expert in the room helps to bridge this gap and builds the capabilities of business operations.

• Fast Feedback: When product-related issues are identified, we were able to provide the fastest path to protect the business and address concerns in a timely fashion.

To give you a bit more insight on this process I’ve added a section to this post called Stories from the War Room which illustrates a real-life incident and the action that was taken by our on-site engineer to resolve it.

• Using In-Memory Computing to buffer peak load access to shared data resources

• Typical eCommerce systems have shared data resources for managing inventory, orders, and catalog information.

• Putting the shared data resources in-memory provides faster and more efficient (parallel) access to this shared data.

• Data is mirrored back into the database in batches. In this way, peak load transactions are buffered so that database traffic does not crash the database back-end.

• The In-Memory Compute grid acts as a system of record. Failure in the underlying database can be saved without affecting the online users while the database is restored to a working state.

• Using a combination of In-Memory & SSD allows very large In-Memory data sets to be stored at a reasonable cost, while still ensuring fast recovery during failure.

• Self-Healing Systems recover from failure in real time

• Failures are inevitable: Keeping a backup copy in-memory enables zero-downtime systems to service user traffic without interruption, even if something does go wrong.

• Systems provisioned for failure handle failure by design.

• Automatic failover and provisioning eliminates the need to overprovision (costly) resources in case of failure. Traditionally, it’s common for retailers to provision resources for holiday season that are 5 times the capacity of non-holiday traffic infrastructure.

Two Examples

In one case, a Top 100 online retailer used XAP to provide access to its catalog and inventory data and achieved its first zero-downtime holiday season in several years. As a result, this retailer delivered a vastly improved customer experience from previous years (achieving an 18% improvement in customer satisfaction ratings) and generated a 139% increase over 2013 holiday sales.

In another case, a Top 30 US online retailer logged a record-setting peak sales day of $44 million. This was especially notable because that same day the retailer experienced system performance issues caused by an automated hardware failover condition. Fortunately, the retailer’s XAP implementation began automatically relocating application components to standby resources, keeping apps running despite the complications. As a result, consumers continued to shop—and buy—with minimal disruption.

Stories from the War Room

I’ve picked two issues that we identified as our engineers were working on-site with one of our top eCommerce customers. I thought that these two cases provides useful insight on how a preemptive support strategy and a short feedback loop works:

Issue #1: Sudden slow client response time

The quote below was taken from the direct on-site report:

 

GC Spikes is one of the common issues that we encounter for managing in-memory data clusters. As GC tends to compete with the same CPU resources that serves the user transaction, this often leads to overall slowness of the system. Fairly quickly, this slowness can pile up into a huge backlog which can break the system in unexpected areas.

The resolution was to split the cluster into more data containers (GSC in XAP terminology) as this will allow better spread of the load across the entire cluster. In addition, overall capacity (memory and CPU) that was allocated for the cluster was increased to meet the increasing capacity demand.

The diagram below provides a view of one of the clusters at the time the issue occurred.

 

As can be seen, around 23:00 the system started to hit its high CPU mark as a result of GC spikes. The system was gradually rebalanced after couple of hours without facing any downtime.  The preemptive action that was taken to handle this incident prevented it from piling up and causing a complete system failure.

 

Issue #2: Connection Issues

 

In this incident, the increase of concurrent client activities during peak load resulted in a large number of network connections that were opened at the same time. One of the nodes in the cluster was misconfigured with a low limit on the number of concurrent connections that could be opened simultaneously.  The resolution was to kill that faulty node and leverage the self-healing capability of XAP to force an immediate re-route of clients into the backup node while relocating the faulty node into another machine.

  

Final Notes

Peak load performance often tends to stretch any system behavior in areas that are least expected and thus are often hard to handle. Quite often, peak loads lead to unexpected downtime.

There are many cases in which this sort of peak load performance is known in advance, as is the case with Black Friday and Cyber Monday. Still, many eCommerce sites continue to experience downtime or slowness during such events that lead to huge loss of revenue and reputation.

As a software vendor, we have often found ourselves involved in the early architecture discussion phases, which usually take place as a result of failure in the previous year. Despite the fact that we are brought in to solve these peak load performance problems, we are still called during a fire drill when those failures occur. Often, the result of the failure was misconfiguration or a problem on another system that manifested itself as an issue in our product. Obviously the experience of handling fire drills is never be pleasant, neither for us nor for our customer and that is something that we wanted to avoid as we approached the 2014 holiday season.

This year we decided, together with our customers, to take a more preemptive approach by putting an engineer on-site to escort the customer team during the event itself.  This resulted in huge success, leading to 100% up time. Both teams learned much from the experience; the customer learned even better how to operate our product and what to look for to ensure that the system is running properly. We learned much about how the customer is using our product and were able to shorten the feedback loop between the customer and our product and engineering team.

With those lessons in hand I feel that both we and our customers are much more equipped to handle 2015.  I can’t wait to write about the lessons learned from Black Friday 2015.

 

 

 

References:

Avoid Blackout on Black Friday and Cyber Monday [Infographic]

The Real Cost of Slow Time vs Downtime [SLIDES]

How Loading Time Affects Your Bottom Line

Our Open Source Cloud (Re) Architecture with OpenStack + TOSCA

Cloudify OpenStack Native approach - The Story behind the Scenes

In the previous post I outlined what I mean by native-approach to the OpenStack cloud. In a nutshell, a native approach represents out of the box tighter integration with OpenStack, all without being limited only to OpenStack.

In this post I want to share some of the stories behind the design of Cloudify 3.0 and the approach that we have taken to make Cloudify *native* to OpenStack.  I would first start by outlining our motivation behind this approach.

OpenStack-only or Not-only-OpenStack?

It is our core belief that for the foreseeable future supporting hybrid cloud is going to be important for many of the OpenStack users for the following reasons:

1. Transitioning to OpenStack is a long journey - Many enterprises have VMware-based stacks as their core infrastructure. Where many of these organizations have often started their journey toward OpenStack by creating a dev environment, and then gradually moving more workloads into their OpenStack environment as they get more comfortable with it. This process usually spans over months and possibly even years. It became apparent to us, that during this time these customers would still be using VSphere and VCloud in parallel to OpenStack. For such users, having a common platform to manage their app deployment across two environments can smoothen the transition and also reduce the risk of getting stacked if one of the environments doesn’t meet their needs.
2. The public cloud market is going to be dominated by non-OpenStack clouds - while OpenStack is on a trajectory to dominate the private cloud market, the public cloud market looks quite different with AWS as a strong and long-standing leader and other major clouds emerging such as GCE and Azure - hoping to close in. Many organizations wouldn’t want to host their entire data center on a private cloud, and would therefore need to take a hybrid cloud approach. In order to support hybrid cloud, we had to include an abstraction layer that will allow us to integrate with clouds other than OpenStack.

At the same time we wanted to have a more intimate integration with OpenStack in order to leverage the fact that it is open source and allows for a deeper integration with its core services. Through that we were able to enable smooth and simple integration of Cloudify into OpenStack.

Extending or Rewriting?

Cloudify 2.x already took a step toward deeper integration with OpenStack and we were one of the first to announce our support for OpenStack. Recently, we also added support for OpenStack Neutron.

We realized that if we want to make Cloudify fit natively into the OpenStack infrastructure it is not enough to have integration with the OpenStack components.  We also need to ensure that the design behind Cloudify will be consistent with the way other projects and services of OpenStack are implemented. This led us to the conclusion that we wouldn’t be able to be a first class citizen with OpenStack unless we underwent a major redesign of Cloudify.

Going Python?

Our first step in this direction was to move from Java to Python. Moving to Python wasn’t as painful as you would think for us, because many of our developers already knew the language. This decision quickly proved itself, as it became much simpler not just to develop the new version of Cloudify faster, but it also enabled us to join other OpenStack projects as well as reuse some of the same components and frameworks that are already used by OpenStack.

Integrating with Core OpenStack Services

Many of the products that declare support for OpenStack mostly integrate with the compute API (Nova), and quite often come with their own security, messaging, configuration, and other such considerations. We felt that this is not enough, and doesn’t fit in with the way other services are built within OpenStack. Hence, we decided to deepen our integration with other services such as Keystone, Neutron, Heat and later with Ceilometer, Mistral and others.

Going TOSCA?

Cloudify 2.x was designed around a proprietary Groovy-based domain specific language (DSL) for defining application topology and configuration. It became apparent that the DSL had become a central piece of Cloudify architecture. As a result, maintaining a proprietary DSL would only serve as a barrier going forward. TOSCA (Topology Orchestration Specification for Cloud Applications led by the Oasis Foundation) defines a standard templating language. TOSCA was similar and richer in concept to our previous 2.x specification. Because of this, using TOSCA seemed like a natural evolution.

At the time when we decided to use TOSCA, the specification was provided in a fairly complex XML. Our first step for adopting TOSCA was to come with a simplified YAML based version which better fits into the spirit of OpenStack. After the OpenStack summit in Hong Kong, when we first introduced the idea, it became apparent that members from the Oasis group were already thinking along this line.We decided to join forces and join the Oasis organization. We started working with the team to incorporate TOSCA as an official part of OpenStack. During the Atlanta summit, it was decided to integrate the TOSCA project into the Heat project and now there is an official definition of the TOSCA 2.0 specification based on YAML. Our goal is to use the TOSCA 2.0 specification as the official templating language for Cloudify instead of our current TOSCA like DSL.

Integrating with OpenStack Heat

OpenStack is growing, and with this the services that are included as part of the core OpenStack project continue to grow up the stack. One of these services is OpenStack Heat, which started as the equivalent of Amazon Cloud Formation, and is becoming more of a general purpose infrastructure orchestration these days.

We realized that since Heat is closely integrated with OpenStack it will follow the OpenStack release cycles and API, and therefore would provide a useful tool for setting up the OpenStack infrastructure. The approach that we have taken with Heat is an “overlay approach”. This means that users can use Heat as they do today to set up their infrastructure. Then, Cloudify integrates with Heat in a way that will allow it to discover any resource that was provisioned through Heat, and ultimately add the monitoring, logging and software stack on top of that environment. 

Integrating with Neutron - Network Orchestration

Networking becomes a core service in any cloud deployment. Networking refers to an element such as security-group, private IP, floating IP as well as routers, DNS, vLans, load balancers, and such.

Cloudify 2.7 included basic integration with Neutron that allows users to attach floating IPs and set the availability zone configuration.

With Cloduify 3.0, we included support for all the networking elements and can now create vLans, security groups and more, as part of the application deployment.

Design for Scale

Application orchestration often requires intimate and continuous integration with the application so that it can detect failure in sub-seconds and take corrective actions as needed. This often imposes a more complex scalability challenge.

Cloudify 2.x uses a hierarchy of managers, where each manager controls 100+ nodes as a scaling architecture. One of the things that we have experienced over the last year is that there is a growing class of services such as  Big Data and NFV. With these services, a single service can be comprised out of more than 100s and potentially even 1000s of instances. This is why the manager of managers approach didn’t fit well to these use cases.

To enable scaling of 1000s of nodes per manager, we decided to use a message broker approach based on AMQP as well as to separate the provisioning, logging and real-time monitoring tasks into separate services that can scale independently. This allows us to also control the level of intimacy between the Cloudify manager and the application services. As an example, we can tune Cloudify to handle the provisioning and logging only, and use the real-time information that has already been gathered by the infrastructure. This is especially important in the context of OpenStack, as we expect that a large part of the metrics can be gathered through the infrastructure itself.

Integrating your DevOps toolchain of choice

One thing that has become clear through the various OpenStack statistics based on their annual survey, is that many of the OpenStack users are using a toolchain that is comprised of tools available through OpenStack infrastructure, as well as tools that are not specific to OpenStack such as Docker, Chef, Puppet, Ansible, JClouds and others.

Integrating all these tools as part of a deployment system becomes a fairly complex task that can takes weeks and even months.

We realized that if we want to provide a full lifecycle management of the application, it wouldn’t be right to force a particular stack or toolchain on a user, but rather provide an open pluggable architecture that could enable you to easily integrate your choice of tools into the same deployment.

With Cloudify 2.x we included support for Chef, Puppet and also had a cloud driver that allows us to integrate with various cloud infrastructures, as well as with tools like JClouds.

With Cloudify 3.x we created a more generic plugin architecture that allows users to plug in almost any element of their application, starting from the cloud plugin to configuration management, orchestration, monitoring etc.

To make this simple we included a Bash plugin that provides a simple framework for integration with external tools, just by pointing the plugin to the relevant Bash script.

Final Notes

In this post I thought sharing the story behind the scenes that we went through to make Cloudify OpenStack native, and the rationale and decision process behind them, would be relevant to other users that are considering their OpenStack strategy.  Especially for those still deciding how deep they should integrate with OpenStack.

The good news is that, it is quite possible to integrate natively with OpenStack without limiting yourself only to OpenStack. Having said that, the initial investment required to get there is quite big. I hope that this lesson from our own experience will save others time.

The Seven Most Popular APIs in Big Data – Part 2

Many APIs, Same Data In “The Seven Most Popular APIs in Big Data—Part One,”

I described the various forms of popular data management APIs out there and how they’re often used. As I noted toward the end of the post, currently each of the APIs is often tied with a specific data model and data store and is used by different APIs that serve other use cases. One of the main reasons behind this is that each API represents a specific optimization that requires a fairly different data structure. Common to all of the various techniques however is that they were written under the assumption that disk is a bottleneck. That led to various point optimizations, architecture, and algorithms that are used by each of the APIs to bypass the disk bottleneck.

Memory Based Data Management Is Ready for Flash

Here's the good news: this need not always be the case. Flash devices have become so advanced that they are able to completely remove the disk from causing this bottleneck, which then makes it possible for us to explore API and data structures that will serve them in a completely different way.
I would argue that there's no longer a need for point optimizations that are not compatible with one another and we can instead use a common data structure to serve each of the APIs.

Same Data, Many APIs

So unlike the majority of existing data management solutions which have been designed with the assumption that disk will be a bottleneck, In Memory Data Grids are based on RAM and therefore designed with the assumption that the data device is fast. Best of all, moving memory-based solutions into flash doesn't require any significant change to your current system. To illustrate this, let's look at our solution, XAP, and what it's able to do:

This example shows how one can write the same data as an object, read it as a document, query it through SQL via JDBC connection, navigate through its subfields as in object graphs, and so on. XAP even allows for the same data store to run both transactional data and analytics, vastly reducing complex data transformation.

Utopia vs. Reality

The ideal system would be one where we could rely on just one common data store that's been outfitted with a set of light-weight services that could expose different APIs and semantics to access data, as illustrated in the diagram below:
In reality however, the maturity and cost of a flash-based solution isn't yet ready for such a transformation. A more realistic scenario is for a flash-based device to be used as a data bus or hub that can act as a front-end for various databases and for more high performance and latency-sensitive use cases. In order to reduce transformation complexity, however, flash will need to include built-in synchronization with those databases and streaming technologies so that the area of complexity is handled implicitly.
Memory Based Data Management Systems Are More Suitable to Serve as High Speed Data Buses

Many of the existing databases were designed with the assumption that they serve as the main data store; therefore, moving from MySQL to Mongo for example often means a complete re-write. Memory Data Grids on the other hand were designed as an extension to the database that takes/holds the part of the data with high contention and serves it in-memory to speed up the access time and scalability. This approach is considered more complementary as it only affects part of the application that needs high speed access, where the remaining of the application continues to work with the same underlying database as if nothing had changed. As a result of that, most of the existing Data-Grid solutions already include fairly rich data synchronization plugins—hence why I would argue that they are more suitable to serve as a Bus or a Hub.

Example: Using High Speed Data Bus with Storm

I’ll leave you with one example of where this framework is effective: “Storm.” This stream processing framework that's used by Twitter includes a few plug-in external feeds (Spouts) while also still maintaining the state of processing (Trident State). The following illustration demonstrates how this integration can be used to handle real time processing of web page analytics by using Storm and XAP as the high speed data bus.

The Seven Most Popular APIs in Big Data – Part 1

Part One: An Overview of the Current Situation

It was only a few years ago when nearly everyone relied on SQL exclusively to tackle Big Data needs, but as the demand for speed and space increases, so have our options. Now there are a number of new data systems that are mostly based around NoSQL, with each of them having been developed to best serve specific areas.
In this post, we'll be taking a look at seven APIs in particular and explore how these systems can be optimized for maximum speed and memory capabilities.

The Seven Most Popular APIs in Big Data

1. SQL: The term “SQL” may suggest that this API is no longer relevant in this new data world, but most of NoSQL implementations support a major subset of SQL. This system is able to provide a rich set of query and data management and is often the least common denominator of many data management systems.

SELECT EmployeeID, FirstName, LastName, HireDate, City FROM Employees
WHERE City = 'London'

2. Document: The document API allows users to write a different structure of fields to the same logical table without any need for schema evolution (which is why document API is often known as “schema less”). It's one of the most popular for web applications that uses the JASON data model.

db.inventory. insert( { _id: 10, type: "misc", item: "card", qty: 15 } )

3. Object Graph: This navigation API is most suitable for hierarchical data structures (i.e. social graphs).

firstNode = graphDb.createNode( );
firstNode.setProperty( "message""Hello, ");
secondNode = graphDb.createNode( );
secondNode = .setProperty( "message", "World! ");

relationship = firstNode.createRelationshipTo (secondNode, RelTypes.KNOWS );

relationship.setProperty ("message" , "brave Neo4j" )

4. Tuple API: Tuples are one of the most common APIs for messaging and stream processing use cases. This API represents a simple data structure that is able to map into a flat data object. It’s often based on using the same tuple structure that was used to write the data instance as the query language, meaning that the tuple acts as a “mask” that indicates which instance type and matching fields are to be selected.

     JavaSpace space = getSpace( );
AttrEntry e = new AttrEntry ( );
e.name = "Duke";
e.value = new GIFImage ("dukeWave.gif");
space.write (e, null, 60 * 60 *1000);
// lease is ignored -- one hour will be enough

5. Key/Value: Key/Value represents the simplest form of data structures. As the name suggests, it consists of a single index per data object. This API is the most popular for caching and is often used as the underlying data structure of more advanced data management solutions.

Memcached Client c=new Memcached Client(new InetSocketAddress ("127.0.01", 11211));
c.set ("someKey",3600, someObject);
Object myObject = c.get( "someKey");
c.delete ("someKey")

6. Stream Based: This event processing model is the most suitable for handling any scenarios where continuous updates will be necessary. It's a popular API for real-time analytics scenarios, which explains why it's becoming increasingly popular for Big Data systems that rely heavily on incremental updates, but which don't require the locking of a large set of data.

TopologyBuilder builder = new TopologyBuilder(); 

builder.setSpout("spout", new RandomSentenceSpout (), 5); 

builder.setBolt ("split", new SplitSentence(), 8).shuffleGrouping("spout");

builder.setBolt ("count", new WordCount(), 12).fieldsGrouping("split", new Fields("word"))

7. Map/Reduce: This API is used to perform aggregation on distributed data. The Map/Reduce model is able to break up aggregation operations into two or more phases. Map executes the aggregation in each data node, and Reduce takes all of the sub aggregations from each node and then reduces them into one consolidated result. Operations such as calculating max, average, and mean is an example of the Map/Reduce Model.

CREATE TABLE input (line STRING);
LOAD DATA LOCAL INPATH 'input.tsv' OVERWRITE INTO TABLE input;

-- temporary table to hold words...
CREATE TABLE words (word STRING);

and file splitter.py'

INSERT OVERWRITE TABLE words
   SELECT TRANSFORM(text)
    USING 'python splitter.py
    AS word
   FROM input;

SELECT word, count (*) AS count FROM words GROUP BY word;

Which System Is Best for Me?  

There is no “one-size fits all” approach when it comes to data management systems. Because most of today's data management systems have an API which is tied to a data model in which the data is stored, we can't write data in one API and read it with another.
This means that if you want to use that same data for a different purpose, one would need to maintain copies of that data to match each use case API and data store. As such, a typical application would need to include a combination of various data management solutions with complex data flows between them, as illustrated below:

 But does it really have to be that complex?

Stay tuned for “The Seven Most Popular APIs in Big Data—Part Two” to find out!

 

Using SSD As A Foundation For New Generations Of Flash Databases

“You just can't have it all” is a phrase that most of us are accustomed to hearing and that many still believe to be true when discussing the speed, scale and cost of processing data. To reach high speed data processing, it is necessary to utilize more memory resources which increases cost. This occurs because price increases as memory, on average, tends to be more expensive than commodity disk drive. The idea of data systems being unable to reliably provide you with both memory and fast access—not to mention at the right cost—has long been debated, though the idea of such limitations was cemented by computer scientist, Eric Brewer, who introduced us to the CAP theorem.

The CAP Theorem and Limitations for Distributed Computer Systems

Through this theorem Brewer stated that it was impossible for any distributed computer system to be able to provide users with these three following guarantees simultaneously:

• Consistency (Every node will be able to view the same data at the same time)

• Availability (Every request will receive a response)
• Partition Tolerance (The system will continue to operate even if the system faces any arbitrary failures)

 We can evaluate different approaches for data management solution by the above three properties and the tradeoffs that each one of them faces when utilized over the other. For example, what is the tradeoff of putting more emphasis on consistency? The trade off will most likely be less availability or partition tolerance.

 Traditional RDBS solutions offered consistency over partition tolerance and cost. In-Memory caching solution such as memcache offered a different set of tradeoffs, for example speed over some degree of consistency (between the in-memory data and the data held in disk).In Memory Data Grid or In Memory Computing, as it is called today, extended the use of memory as the system of record and thus enabled higher degree of consistency by reducing the dependency on the underlying disk databases. The new generation of NoSQL databases took a different approach and offered speed and scale at low cost over consistency by utilizing distributed commodity storage and relying heavily on asynchronous data flow to speed up the data processing.

 Quite often there is a direct correlation between those tradeoffs and cost - for example high degree of consistency often relies on synchronous data-flow and replication operations which often come at a cost of speed. To overcome the speed limit we often need to use memory or other flavors of high speed storage which often translate to high cost. On the other hand if we can compromise on consistency, we can offer speed and scale using commodity resources (  without relying on high cost resources).

As technology advances and demands for cost effective and scalable, reliable data increase, new solutions are coming to light, with one of the most favored being a combination between two items we are already familiar with: RAM and flash.

 

Why SSD Is a Viable Storage Solution

Many of the underlying assumptions in the previously mentioned alternatives that  was built under the assumption that disk is the bottleneck. If disk is the bottleneck, we needed layers of optimizations to minimize the access to disk. SSD provides a high speed storage device, which gets rid of the idea that disk is the bottleneck. This is a fundamental change in one of core assumptions behind the design of many databases today, let me explain..

 For example if the disk is slow it makes sense to put various filters to minimize the access to disk. As disk gets faster all of those filters becomes an overhead. In many cases it would be faster writing directly to disk without those additional layers. In similar way the use of asynchronous operations need to change as well. When the disk is slow it makes sense to use asynchronous operations to deffer write operations. However, when the disk is no longer the bottleneck it would be faster and vastly simpler to access the disk directly. This also allows avoiding many of the consistency tradeoffs that i mentioned earlier that are often a result of those asynchronous operations.

 

Putting SSD and RAM Together using off Heap Storage

 RAM storage can be as big as necessary and it is incredibly quick—but it costs about ten times more than a flash disk. By putting SSD and RAM together we can optimize the cost/performance ratio. The main challenge in doing so, remain around consistency-i.e. how do we keep the data in-memory and flash in sync so that from an end user perspective this integration would look completely seamless?

One method of such integration is referred to as off-heap storage.  

Off Heap storage is often implemented as a plug-in that is used by the memory data-store to offload its data from the actual RAM into Flash and visa versa.

Traditionally Off Heap was implemented with shared memory which is basically a block storage that provides external access to the same RAM device. By doing so, we bypass the management overhead of managing data in RAM through JAVA. The limitation of that approach is that the capacity is still limited by the size of RAM. It also forces an external data management and garbage collection layer to manage this external heap.

With SSD, users can now have both the memory and the SSD device synchronized and regularly kept up-to-date, meaning storage is scalable and consistent.

Some of the SSD drivers provides a key,value interface which makes the integration vastly simpler as it takes care of its own data  management.  Having said that, most of the In-Memory implementation that uses SSD as off-heap storage provides fairly limited functionality in terms of query and transaction support. This is due to the fact that they( What is they?)  integrate with SSD through a disk storage interface.  

 

Using SSD as a Foundation for Flash DataBase

 To really make the best use of SSD and maximize its potential we need to think of SSD as foundation for database. This requires a more tight and native integration with SSD in order to overcome some of the limitations that exist with many of the current In Memory Data Grid and SSD implementations.

 The specific set of features that are needed for this sort of integration includes: 

• Portable and native Key/Value interface that works against any flash device.
• Using Flash as Durable Storage - Flash can be used as a persistent data-store and not just as extension to RAM. As such we can leverage the durability of flash to speed up the load time of data from the underlying flash device incase of a planned or un-planned recovery process.
• Support for batch operations - batch operations are a common way to speed up the access to flash by minimizing the number of cross boundaries calls between the RAM device and the underlying flash devices.
• Transactional - To support transactional access we need to extend the batch operations to fail, succeed as a single atomic operation.
• Query index support - to speed up the query and access time, the data needs to be indexed in a way that fetching a particular object by its IDcould point directly to its physical location on disk.

 

To make this a practical reality GigaSpaces partnered with SanDisk who through fusion I/O acquisition own the majority of the SSD market. As part of this partnership, we implemented a new version of our In Memory Data Grid, XAP MemoryXtend, which is now integrated with SanDisk ZetaScale interface. ZetaScale implements all of the above features as a general purpose flash disk interface. Through this integration we are now able to utilize the XAP RAM based storage for handling complex queries and analytics as well as transactional support. We can also leverage the XAP cluster support for integrating multiple flash devices that are mounted on various machines and make them work as one big transactional database ( as illustrated in the diagram below).

The XAP MemoryXtend cluster consists of a number of partitions each plugged into a local SSD drive.  Each partition can have at least one backup running on a separate machine to ensure  availability. The XAP client is a smart proxy that abstracts the underlying cluster and exposes all of the physical partitions through a single data-grid interface. The proxy routes the write or query requests to a particular partition in cases where we’re looking for a specific data item. In the case of agregated queries it will invoke a parallel query against all nodes and consolidate the results into a single result-set.

 

Flash DataBase as Service

As Flash devices are now being supported by cloud providers such as Amazon, it only make sense to leverage this capability and offer this sort of Flash based database on-demand.

This is where the XAP Deployment and Orechestration comes handy as it automates the process of deployment and management of XAP clusters across a variety of cloud infrastructures.

For more information, check out The next Big Disruption in Big Data.

The next big thing in big data: fast data

The big data movement was pretty much driven by the demand for scale in velocity, volume, and variety of data. Those three vectors led to the emergence of a new generation of distributed data management platforms such as Hadoop for batch processing and NoSQL databases for interactive data access. Both were inspired by their respective predecessors Google (Hadoop, BigTable) and Amazon (Dynamo DB).

As we move to fast data, there’s more emphasis on processing big data at speed. Getting speed without compromising on scale pushes the limits in the way most of the existing big data solutions work and drives new models and technologies for breaking the current speed boundaries. New advancements on the hardware infrastructure with new flash drives offer great potential for breaking the current speed limit which is bounded mostly by the performance of hard drive devices.

Why using existing RDBMS, NoSQL on top of flash drive is not enough

Many of the existing databases — including more modern solutions such NoSQL and NewSQL — were designed to utilize the standard hard drive devices (HDD). These databases were designed with the assumption that disk access is slow and therefore they use many algorithms such as Bloom filters to save access to disk in case the data doesn’t exist. Another common algorithm is to use asynchronous write to commit-log. A good insight into all the optimizations that are often involved in a single write process on NoSQL databases is provided through the Cassandra Architecture. Let’s have a look at what that entails below:

 

The Cassandra write path

 

What happens when disk speed is no longer the bottleneck?

When the speed of disk is no longer the bottleneck, as in the case of flash devices, then a lot of the optimization turns into overhead. In other words, with flash devices it will be faster and simpler to access the flash device directly for every write or read operation.

Flash is not just a fast disk

Most of the existing use cases use flash devices as faster disk. Using flash as a fast disk was a short path to bypass the disk performance overhead without the need to change much of the software and applications. Having said that, this approach inherits many unnecessary disk driver overhead. So to exploit the full speed potential of flash devices, it would be best to access the flash devices directly from the application and treat flash devices as a key/value store rather than a disk drive.

Why can’t we simply optimize the existing databases?

When we reach a point in which we need to change much of the existing assumptions and architecture of the existing databases to take advantage of new technology and devices such as flash, that’s a clear sign that local optimization isn’t going to cut it. This calls for new disruption.

The next big thing in big data

Given the background above, I believe that in the same way that demand for big data led to the birth of the current generation of data-management systems, the drive for fast data will also lead to new kinds of data management systems. Unlike the current set of databases, I believe that the next generation databases will be written natively to flash and will use direct access to flash rather than the regular disk-based access. In addition to that, those databases will include high performance event streaming capabilities as part of their core API to allow processing of the data as it comes in, and thus allowing real-time data processing.

 

Fast data in the cloud

Many of the existing databases weren’t designed to run in cloud as first-class citizens and quite often require fairly complex setup to run well in a cloud environment.

As cloud infrastructure matures, we now have more options to run big data workload on the cloud. The next generation databases need to be designed to run as a service from the get-go.

To avoid the latency that is often associated with such a setup, the next generation databases will need to run as close as possible to the application. Assuming that many of the applications will run in one cloud or another, it means that those databases will need to have built-in support for different cloud environments. In addition, they will need to leverage dynamic code shipping to pass code with the data, and in this way allow complex processing with minimum network hops.

In-memory databases and data grid are the closest candidates to drive next-gen flash databases

RAM and flash-based devices have much more in common than flash and hard drive. In both RAM and flash, access time is fairly low and wasn’t really the bottleneck; in-memory databases use direct access to the RAM device through key value interface to store and index data.

Those factors makes in-memory based databases more likely to fit into the next generation flash databases.

The combination of in-memory databases and data grid on top of flash devices also allows the system to overcome some of the key capacity and cost per GB limitations of an in-memory based solution. The integration of the two will allow an increase in the capacity per single node to the limit of the underlying flash device rather than to the limit of the size of RAM.

The overall architecture of the integrated solution looks as follows (Source: Gartner):

 

 

As can be seen in the diagram above, the IMC (in-memory computing) layer acts as the front end to the flash device and handles the transactional data access and stream processing, while the flash device is used as an extension of the RAM device from the application perspective.

There are basically two modes of integration between the RAM and flash device.

LRU Mode – In this mode, we use the RAM as a caching layer to the flash device. The RAM device holds the “Hot,” i.e. the most recently used, and the flash device holds the entire set.

Pros: Optimized for maximum capacity.

Cons: Limited query to simple key/value access

Fast Index Mode – In this mode, all the indices in RAM and the data in flash holds the entire set.

Pros: Supports complex querying, including range queries and aggregated functions.

Cons: Capacity is limited to the size of indexes that can be held in RAM.

In both cases, the access to flash is done directly using key/value interface and not through disk drive interface. As both RAM and flash drives are fairly fast, it is simpler to write data to the flash drive synchronously and in this way avoid potential complexity due to inconsistency.

It is also quite common to sync the data from the IMC and flash devices into an external data store that runs on a traditional hard drive. This process is done asynchronously and in batches to minimize the performance overhead.

Direct flash access in real numbers

To put some real numbers behind these statements, I wanted to refer to one of the recent benchmarks that was done using an in-memory data grid based on GigaSpaces XAP and direct flash access using a key/value software API that allows direct access to various flash devices. The benchmark was conducted on several devices as well as on private and public cloud-based services on AWS. The benchmark shows a ten times increase in managed data without compromising on performance.

 

 

 

For full details of the benchmark, see the following white paper:

XAP MemoryXtend -Massive Application Storage Capacity for Real-Time Applications

 

 

 

Notes from Big Data Business Challenges Panel Discussion

Most of the Big Data topics that I covered in the past were focused mostly on the technical aspects of running Big Data.

In this post, I want to take a different route and discuss the business challenges based on a panel discussion that I had the pleasure of moderating.

 

 

Is Big Data a Big Hype?

One of the first questions that came up in the discussion is whether Big Data is over-hyped due to over-use in marketing content. Many of those who claim to provide either a Big Data solution or implement Big Data are just using the term to re-brand their product or offering.

 There was consensus across the panel that while Big Data is probably over hyped like any big trend, it does represent a real change in both technology and business. Let me explain:

In a pre-Big Data world, Big Data lived in expensive and closed data warehouses, where many of the solutions were often tied to specific, high end hardware. In addition, the term “warehouse” in itself meant that most of the data had little exposure and impact on the business.  Today, Big Data is everywhere and can be done at a fraction of the cost of the traditional data warehouses. With the addition of cloud the barrier to running Big Data has become significantly lower and now every startup can run Big Data almost at the scale of Google.

Having said that, the actual adoption of Big Data tends to vary quite significantly per industry with telecommunications (67%), consumer goods (57%) and financial services (52%) industries as the leaders in recognizing that Big Data can greatly improve their understanding of customer needs according to a recent Hitachi Data Systems Corporation (HDS) study  .The same study indicates that more than 60% of the firms in the financial services and consumer goods industries haven't started any Big Data programs. Healthcare and life science are lagging further behind; 72% of them haven't started any Big Data programs. This means that the perspective of whether Big Data is real or just marketing hype tends to vary as well depending on the industry.

 

How do you make sense out of your Big Data? Do we need a new role for Chief Data Officer?

Many of the questions in the panel had to do with how to make sense of the data.

One of the common strategies to make sense of the data is to assign a data scientist or Chief  Data Officer, expecting them to deal with the data and offer useful insights. Another common strategy is to put all the data in something like Hadoop with the hope that having the data in hand will provide a variety of interesting insights that will change the way we operate.

Putting the data in Hadoop or assigning a specialized data analyst are often just the beginning of a process.

Eyal Veitzman from Wix described how they deal with the challenge. To really make sense of your data, you first need to change the organizational structure in which the business, developers and data analysts will be part of the same group. In this way you can turn insights into action and more importantly have a faster feedback loop that will allow you to iterate quickly.  

 

What is the business value behind Big Data?

The business value behind Big Data comes in different shapes and forms depending on where you apply it in your system. Insightera uses Big Data to capture user interaction and increase the conversion of users using personalized responses. Wix uses Big Data to optimize their user conversion and satisfaction.  Nice Systems uses Big Data to improve their call center user satisfaction. PayPal is using Big Data in all sort of shapes and forms to both detect fraud as well as to optimize their operations. Check Point uses Big Data to collect data from all their gateways and in that way keep their gateways protected from new virus attacks

So how much can we gain from applying Big Data practices in our organization?

According to the Hitachi Data Systems Corporation (HDS) study, Big Data can help companies' revenue by 25%. This applies mostly to companies that use Big Data to optimize their internal operations and products, like most of the companies in the panel. That is obviously more true for companies that sell Big Data insights as part of their business, as in the case of Insightera.

Is there a good visualization tool for Big Data?

Big Data visualization is a tough challenge and there is no simple answer or single framework that fits them all. Using infographic style can help to visualize data in a way that makes it easier to capture lots of data into a single, readable report using a variety of visualization dimensions, such as fonts, colors, graphics, etc. There are many open source frameworks, such as Jaspersoft and many SaaS-based frameworks, aimed to provide specialized reporting and visualization tools. A useful reference on that regard can be found here here: The 36 best tools for data visualization.

Having said that, the important thing is to decide what and how to measure. Quite often, the number of Key Performance Indicators (KPIs) must be limited to a fairly small number.

The second important thing is to realize that as humans, we are not designed to capture and process vast amounts of information and therefore, visualization tools provide only a small aid to a much bigger challenge.

In summary, there is a dangerous slippery slope with Big Data visualization; fairly quickly we can flood our users with many graphs and numbers to a point where what we actually get is a dimension return and more complexity and confusion than clarity. Those who had experience with analytics tools such as Google Analytics are probably familiar with that experience. On the other hand, narrowing down the key performance indicators into a small number of KPIs makes the visualization challenge simple.

 

Moving from Reports to Actions

The key in capturing most of the value of Big Data is to turn Big Data into actions. In the example of Insightera, actions means presenting a user with the most relevant information on a website. In the case of Nice Systems, actions would refer to routing the caller in a call center to the right agent based on the caller’s profile and history of behaviour. In the case of GigaSpaces, this would mean constantly adjusting our provisioning system to detect failures and auto-heal a production environment when something goes wrong.

To get to that level we need to apply automation practices on how we collect, analyze and execute actions out of our data. We also need to be able to process data in real time to reduce the time to action.

Final words: How do make our data instantly actionable?

It is quite clear that the end game of any Big Data initiative is to figure out a way to make it instantly actionable. One of the big challenges on that regard is that quite often we need to compromise between scale and speed. In fact, many of the Big Data systems (Hadoop included) use batch processing to be able to process large amounts information, meaning that the actual insights, and thus actions, from that data will be available only when the batch cycle has been completed.

To make Big Data instantly actionable we need to figure out a way in which we can process vast amounts of data in real time and still keep the cost down.

During the panel discussion, PayPal described their use of In-Memory based architecture to handle this challenge. Still, pure In-Memory-based solutions tend to be fairly expensive and limited in capacity comparing to disk based alternatives. So the next big thing IMO is to devise a way in which we can get In-Memory speed at a cost that is closer to Disk. This is where new advancements in SSD have the potential to disrupt the entire way in which we manage and process Big Data, by providing high capacity durable RAM based out of SSD that can reach terabytes capacity per single machine. The fact that it can be durable means that we could rely completely on In-Memory databases to manage and store larger chunks of our Big Data with no need to offload or reload the data to and from external disks. I also expect that this will lead to the rise of a new generation of In-Memory databases that will be designed primarily for this new kind of durable RAM devices.

 

About the Panel

The Big Data Business challenges panel  was led by the CICC and in collaboration with IGTCloud, focusing primarily on the business challenges side of Big Data. The event was held simultaneously in California and Israel and was hosted by eBay. The panel included a range of industry experts in Big Data from a wide range of companies and industries and included Allen Kamer, Chief Commercial Officer, Optum Analytic (Co-founder of Humedica that was acquired by Optum Health), Doron Simon, Head of Cloud, Business Development, Nice Systems, Mickey Alon, Co-Founder & CEO, Insightera (Acquired by Marketo), Dr. Nachum Shacham, Principal Data Scientist, PayPal, Eyal Veitzman, Director of Business Operations, Wix and Ron Davidson, Head of Security Services, Check Point.

I had the special honor to organize and moderate the panel together with Ido Sarig, General Manager, IoT Solutions Group while Wind River moderated the panel in California.

 

Useful References

• Big data can help companies' revenue by 25%

Telecommunications (67%), consumer goods (57%) and financial services (52%) industries are leaders in recognizing that big data can greatly improve their understanding of customer needs. However, more than 60% of the firms in the financial services and consumer goods industries haven't started any big data programs. Healthcare and life science are lagging further behind; 72% of them haven't started any big data programs. http://timesofindia.indiatimes....­

• Fast Data hits the Big Data fast lane

The usual factors: the same data explosion that created the urgency for Big Data is also generating demand for making the data instantly actionable. Bandwidth, commodity hardware and, of course, declining memory prices, are further forcing the issue: Fast Data is no longer limited to specialized, premium use cases for enterprises with infinite budgets. Not surprisingly, pure in-memory databases are now going mainstream: ..

..Potential use cases for Fast Data could encompass:

• A homeland security agency monitoring the borders requiring the ability to parse, decipher, and act on complex occurrences in real time to prevent suspicious people from entering the country
• Capital markets trading firms requiring real-time analytics and sophisticated event processing to conduct algorithmic or high-frequency trades
• Entities managing smart infrastructure which must digest torrents of sensory data to make real-time decisions that optimize use of transportation or public utility infrastructure
• B2B consumer products firms monitoring social networks may require real-time response to understand sudden swings in customer sentiment
• The 36 best tools for data visualization

A useful reference for popular visualisation frameworks 

Industry Survey Shows: Organizations Turn to Stream Processing for their Big Data Needs

In a stream processing model, data is processed as it arrives to the Big Data system. With a batch processing model, only once the data is stored does the system run a variety of batch analytics - typically through a map/reduce style of processing.

 Facebook and Twitter are good examples of stream processing models in use today, as I outlined in a previous post entitled Facebook’s vs Twitter’s Approach to Real-Time Analytics.

 According to a recent survey of approximately 250 respondents, there is a trend moving towards stream processing in order to speed up analytics, with a significant increase in the popularity of this model when compared with last year. The number of organizations planning to use stream processing in 2014 has more than doubled (24%) from last year’s amount (10%).

This goes hand in hand with the fact that real-time analytics is also becoming more mainstream, as I pointed out in my 2014 Predictions.

Another interesting data point from the survey is that roughly 43% of the respondents defined real-time as sub-second and 42% defined it as sub-minute. This difference is quite interesting and could lead to different approaches for implementing stream processing.

For example, real-time is defined differently by Facebook and Twitter. In the case of Twitter, they chose Storm as their event processing engine, allowing them to process events at a sub-second resolution. Meanwhile, Facebook defines real-time at a 30 second batch window, choosing a logging-based approach to fit their need.

Based on the survey, it appears that both approaches are valid and can be applied in correlation to the degree of real-timeliness of your analysis.

 

Facebook Log-Centric Stream Processing	Twitter Event-Centric Stream Processing

In-Memory Data Grids become a more popular choice for real-time processing

According to the survey, 64% of the respondents indicated that they plan to combine In-Memory-based solutions for delivering their real-time analytics processing. This is also consistent with last year survey done by Ventana Research.

 I believe that will see an even bigger movement in this direction, as the cost/performance ratio of In-Memory Data Grids and In-Memory Databases would go down significantly with the combination of RAM and Flash devices, which together provide a fairly compelling solution from both performance and cost ratio to that of other disk-based alternatives.

 

Big Data in the Cloud increased to 56%

New developments in cloud infrastructure, such as the support for bare metal in the new OpenStack releases, as well as support, high memory, flash disk, etc, are removing many of the technical barriers for running I/O intensive workloads like Big Data in the cloud.

Indeed, according to the survey, there is a significant shift towards the use of Big Data in the cloud with 55% of organizations either using or planning to run their Big Data in the cloud in 2014.

 

Where do we go from here? How does this effect GigaSpaces’ future roadmap?

There are multiple areas in the GigaSpaces roadmap that aim to address this demand. 

• Real-Time Processing through Storm Integration - In this project, we integrate Storm on top of a Memory backend for both stream processing and data-store.
• Support for Flash Disk - The integration with flash disk is planned for our upcoming XAP release and will include support for SanDisk, Fusion I/O and other flash disk devices.
• Big Data in the Cloud - through Cloudify - We've been continuously extending our Big Data portfolio support and recently added support for Storm and Cognos, adding to our existing support for Hadoop, MongoDB, Cassandra, ElasticSearch, etc.

 

References

• GigaSpaces Annual Big Data & Cloud Survey 2013
• Eight Cloud and Big Data Predictions for 2014
• GigaSpaces and Storm Part 2: XAP Integration
• GigaSpaces And Storm Part 1: Storm Clouds
• Big-Data-Real-Time-Performance - Enjoying both worlds with one Architecture!
• 2012 Big Data Survey Report
• In Memory Computing (Data Grid) for Big Data

An Application-Centric Approach to DevOps

 

Matt Jaynes, Founder of DevOps University, wrote a great post on Hacker News, What is Continuous Deployment, in which he points out one of challenges that many people gloss over when they embark on their DevOps journey. In a nutshell, Matt was saying that, "You should get your house in order before aspiring to do automation or CD." In other words, if your testing is not fully automated or is taking too long to process, then having continuous deployment may not be right for you.

The thing that gets me so interested in this entire DevOps movement is that it forces you to think holistically about software development, business cycles and organizational culture. Some refer to this transition as the industrialization of IT. Indeed, there are many parallels between the shift from making cars by hand into a fully automated manufacturing pipeline and the DevOps movement.

With regards to Jaynes’ post, even if you can build a car faster, lack of proper QA may lead to shipping defective parts, and you will still be shipping slowly. In both options, your fast pipeline would end up producing diminishing return, as you may end up paying more for the warehouse or for handling "recall" or fixing defects, let alone the potential loss of reputation or even the entire business.

In short, you need to get your house in order before launching continuous deployment right into production.

Another area of complexity in building continuous delivery and deployment is that it often involves fairly complex processes for handling updates. I noticed that in many cases, people need to write a lot of custom code to implement these processes. That custom code can be fairly hard to maintain and track.

In this post, I would like to share a different approach to handling continuous delivery based on the experience of a Cloudify user - PaddyPower.

Paddy Power are a betting and entertainment company headquartered in Dublin Ireland.  Innovation has been the key to their success and from day one they sought to differentiate themselves from the chasing pack.  Like many organisations, they faced the need to speed up their development and deployment cycle. They went through a complete cycle of moving into agile development as well as transforming their infrastructure into a cloud-based infrastructure based on CloudStack.

They use Cloudify and Chef to run their DevOps automation and continuous delivery.

The way they approached DevOps in general, and continuous delivery in particular, is quite interesting because they took an application-centric approach to DevOps, as opposed to an infrastructure-centric approach as used in some other cases. Let me explain...

What is an Application-Centric Approach to DevOps?

In many DevOps discussions we often tend to jump into low level discussions about tools, and fairly quickly we start to lose sight of what we are actually trying to achieve.

Quite naturally, many start with one of the first steps in automation, which is configuration management - i.e. our ability to consistently configure, setup and update machines with the right stack. In this model, the world looks like a bunch of machines at the core onto which we then tie our software parts. I would refer to that approach as an infrastructure-centric approach.

An application-centric approach is looking at the same problem, but from a different perspective in which we start with a consistent definition of our application stack and blue-print (Load-Balancer, Web Container, Database, etc) and SLAs (failure detection, scaling rules, upgrade processes, etc). Configuration management comes as a natural complementary step for handling the installation and setup of our environment.

In this model, there is a greater emphasis on the orchestration of our application and its associated processes. A good example of such a process is continuous delivery. The execution of a continuous delivery process involves many steps that go far beyond a one-time execution of a given script. We often need to determine in which environment the script is going to be executed, which pieces of our stack are going to be included in each particular deployment, which configuration change is going to be included as well as code updates, and if there will be a database update- then which part will be affected by the change. Another example is continuous availability, automating the recovery process of our application in the event of failure. Such processes can become quite complex as we start to handle multiple availability zones, regions, as well as multiple environments for dev, test and prod.

With an infrastructure-centric approach, we often need to implement those processes into the code or scripts. Quite often, those scripts become very hard to maintain.

With an application-centric approach, we use a set of rules and events to handle the automation of those processes in a more generic fashion. With this approach, many of the policies and processes are handled more implicitly. For example, we can have a policy that if we fail during an update, the process will roll-back, or we can have a policy for creating an exact clone of our system for QAT, and only when it passes the full test may we start rolling it out, gradually moving traffic into those new processes.

Another difference between an application vs infrastructure-centric approach is in the level of abstraction that we expose to our developers. With an infrastructure-centric approach, each developer is often exposed to every part of our system. A continuous delivery model, where developers continuously interact with the system exposing the low level details of the infrastructure to every developer, can lead to complexity that is often unnecessary when all that is needed is to push code to an existing deployment. It can also lead to high vulnerability as a simple mistake can very easily bring an entire system down.

I thought that the following slide by John Turner, Software Development Manager at PaddyPower, does a good job of explaining how an application-driven approach through PaaS abstraction helps to address this challenge:

In summary, PaaS fits nicely into PaddyPower’s DevOps and continuous delivery strategy in the following ways:

• PaaS strives to abstract provisioning and deployment, thus reducing complexity.

• PaaS constraints applications to use features provided by the platform.

• PaaS is development focused.

In the case of Cloudify, the integration with Chef provides a great foundation for more open PaaS, one that fits nicely into DevOps environment. Chef cookbooks and recipes can be incorporated as part of the Cloudify definition of the application stack. It also fits well with the configuration and operational management of the platform.

Join me in our upcoming podcast with Ben Kepes of Clouderati and Diversity.Net, John Turner of PaddyPower and Andy Hawkins from Opscode as we discuss PaddyPower’s journey to app-centric DevOps. Watch it live via Google+ Hangout on Air | October 30th, 6:00 PM GMT, 10:00 AM PST.

What are the Specific Features in Cloudify that Makes it DevOps Oriented?

I will use Matt Jaynes’ definition from his book Taste Test:

• Directive & Directives Script -  Cloudify currently uses Groovy-based recipes with extended DSL as its directives. The Cloudify recipe allows you to combine Chef or Puppet directives as part of the Cloudify recipie. Our future release will include support for Yaml/TOSCA directives and may also include support for Ansible.

• Master Node & Children Nodes - Cloudify automatically bootstraps Master and Child nodes. It has a cloud plug-in that will allow you to automate this entire process and avoid any manual host/IP configuration. The Cloudify integration with Chef takes care of provisioning of Chef Master and Child nodes as well.

• Remote Execution - Cloudify uses custom-commands as a way to route calls to a group of individual sets of service instances. The integration with Chef includes built-in custom commands for updating Chef Coookbooks as well as for executing Chef-Knife and other Chef commands as described here.

In addition, Cloudify provides built-in services to query the state of the cluster through the ServiceContext API and share states across services in the cluster through the Attribute Store API.

Executing a Typical DevOps Task with Cloudify

1. Cloning a service or a complete stack - One of the features of Cloudify is the abstraction of its recipe (blue print) from the underlying infrastructure through a Cloud Driver. A recipe can point to a type of compute or storage node. The actual binding to the physical compute or storage resources takes place at runtime during the deployment process, based on the target environment. That means that you can easily clone a complete or only part of a given application stack across DEV, QA, PROD in a consistent way that will ensure that you use the exact setup between environments. The CloudDriver abstraction allows you to also adapt the size of the resource in each stage without touching the actual recipe - so for example a SMALL-LINUX machine in QA may point to a machine with smaller size and SLA than a SMALL_LINUX in production.

2. Updating a service with a new version - Custom-commands allow you to push updates to a particular instance or group of instances. Quite often, custom command calls will be triggered through a build system, such as Jenkins. Each command has access to the state of the cluster through the Attribute Store and ServiceContext and can, therefore, coordinate the update processes with the state of other services.

3. Orchestration flow - You can use the Cloudify Rest-API or Command Line to write custom flows. Certain flows, such as fail-over and auto-scaling, are built-in and in those cases you will only need to provide custom code to trigger for the flow.

Looking Into the Future

One of the things that we are working on right now is enhancing our orchestration capabilities, allowing users to choose more built-in flows or define their own custom flows that will be fully managed by the Cloudify orchestration engine.

We hope you join us for our live podcast on October 30th at 6:00 PM GMT / 10:00 AM PST on Google+ Hangout on Air.

References

• DevOps PaaS and everything in between.

• Its the apps stupid

• Application Centric Devops

• GigaSpaces Cloudify & VMware Cloud Foundry the new PaaS Jailbreaker

• Putting DevOps and PaaS together with Cloudify

 

Facebook’s vs Twitter’s Approach to Real-Time Analytics

Last year, Twitter and Facebook have released new versions of their real-time analytics systems.

 In both cases, the motivation was relatively similar — they wanted to provide their customers with better insights on the performance and effectiveness of their marketing activities. Facebook’s measurement includes “likes” and “comments” to monitor interactions. For Twitter, the measurement is based on the effectiveness of a given tweet – typically called “Reach” – basically a measure of the number of followers that were exposed to the tweet. Beyond the initial exposure, you often want to measure the number of clicks on that tweet, which indicate the number of users who saw the tweet and also looked into its content.

Facebook’s vs Twitter’s Approach to real-time Analytics 

Facebook Real-Time Analytics Architecture – Logging-Centric Approach:

  Relies on Apache Hadoop framework for real-time and batch (map/reduce) processing. Using the same underlying system simplifies the maintenance of that system.

• Limited real-time processing — the logging-centric approach basically delegates most of the heavy lifting to the backend system. Performing even a fairly simple correlation beyond simple counting isn’t a trivial task.

• real-time is often measured in tens of seconds. In many analytics system, this order of magnitude is often more than enough to express a real-time view of what is going on in your system.

• It is suitable for simple processing. Because of the logging nature of the Facebook architecture, most of the heavy lifting of processing cannot be done in real-time and is often pushed into the backend system.

• Low parallelization — Hadoop systems do not give you ways to ensure ordering and consistency based on the data. Because of that, Facebook came up with their Puma service that collects and inputs data into a centralized service, thus making it easier to processes events in order.

• Facebook collects user click streams from your Facebook wall through an Ajax listener which, then, sends those events back into the Facebook data centers. The info is stored on Hadoop File System via Scribe and collected by PTail.

• Puma aggregates logs in-memory, batching them in windows of 1.5 seconds and stores the information in Hbase. 

• The Facebook approach puts a huge limit as to the volume of events that the system can handle and have significant implications over the utilization of the overall system.

 

Twitter Real-Time Analytics Architecture – Event-Driven Approach:

• Unlike Facebook, Twitter uses Hadoop for batch processing and Storm for real-time processing. Storm was designed to perform fairly complex aggregation of the data that comes through the stream as it flows into the system, before it is sent back to the batch system for further analysis.

• Real-time can be measured in milliseconds. While having second or millisecond latency is not crucial to the end user — it does have a significant effect on the overall processing time and the level of analysis that we can produce and push through the system. As many of those analyses involve thousands of operations to get to the actual result.

• It is suitable for complex processing. With Storm, it is possible to perform a large range of complex aggregation while the data flows through the system. This has a significant impact on the complexity of the processing. A good example is calculating trending words. With the event-driven approach, we can assume that we have the current state and just make the change to that state to update the list of trending words. In contrast, a batch system will have to read the entire set of words, re-calculate, and re-order the words for every update. This is why those operations are often done in long batches.

• Extremely parallel – Asynchronous events are, by definition, easier to parallelize. Storm was designed for extreme parallelization. Ultimately, it determines the speed level of utilization that we can get per machine in our system. Looking at the bigger picture, this quite substantially adds to the cost of our system and to our ability to perform complex analyses.

Final Words

 Quite often, we get caught in the technical details of these discussions and lose sight of what this all really means.

If all you are looking for is to collect data streams and simply update counters, then both approaches would work. The main difference between the two is felt in the level and complexity of processing that you would like to process in real-time. If you want to continuously update a different form of sorted lists or indexes, you’ll find that doing so in an event-driven approach, as is the case of Twitter, can be exponentially faster and more efficient than the logging-centric approach. To put some numbers behind that, Twitter reported that calculating the reach without Storm took 2 hours whereas Storm could do the same in less than a second.

Such a difference in speed and utilization have a direct correlation with the business bottom line, as it determines the level and depth of intelligence that it can run against its data. It also determines the cost of running the analytics systems and, in some cases, the availability of those systems. When the processing is slower there would be larger number of scenarios that could saturate the system.

References:

• Slides: real-time Big Data with Storm, Cassandra and In Memory Computing
• Video: Real-time Analytics for Big Data: A Facebook Case Study
• Video: C* Summit 2013: Real-Time Big Data with Storm, Cassandra, and In Memory Computing
• Nathan Marz (Twitter) on Storm