Nati Shalom's Blog

Memory is the New Disk for the Enterprise

Back in June 2008 i wrote a post in response to an InfoQ article:  RAM is the new disk... where i quoted Tim Bray and others comparing file system performance to memory:

Memory is the new disk! With disk speeds growing very slowly and memory chip capacities growing exponentially, in-memory software architectures offer the prospect of orders-of-magnitude improvements in the performance of all kinds of data-intensive applications. Small (1U, 2U) rack-mounted servers with a terabyte or more or memory will be available soon, and will change how we think about the balance between memory and disk in server architectures....

… Disk will become the new tape, and will be used in the same way, as a sequential storage medium (streaming from disk is reasonably fast) rather than as a random-access medium (very slow). Tons of opportunities there to develop new products that can offer 10x-100x performance improvements over the existing ones.

It didn't take long for this vision to become a reality through the work that we're doing these days with Cisco. By integrating GigaSpaces XAP with the Cisco UCS machine we are demonstrating our ability to easily load hundreds of gigabytes into a single box, and to scale linearly with growing capacity without any performance degradation. This is a great example of how middleware that was built for memory from the ground up, combined with hardware that was equipped to provide terabytes of memory in a single box, can be game changing.

This exciting combination makes it possible to manage 15-20x the amount of data in-memory, per partition. This, in turn, makes it possible to store the entire application data set in‑memory, and gain not only 10x the performance but also great simplicity, because the application no longer needs to deal with a *miss* ratio in the cache; and, at the same time, there are no consistency issues because all the data resides in-memory.

The table below was taken from a recent Stanford research paper -- The Case for RAMClouds. It provides estimates of the total storage capacity needed for one-year’s-worth of customer data for a hypothetical online retailer and a hypothetical airline. In each case the total requirements are no more than a few terabytes, which would fit in a modest-sized RAMCloud. The last line provides the estimated cost for purchasing the necessary RAMCloud servers :

As the table illustrates, storing the entire annual data of an online reservations system or an online retailer can be mapped completely into memory. The combination with UCS makes the deployment of such a system simple to manage and economic. The online retailer's annual data can fit into 4 UCS chassis, and the entire annual data set of the airline reservation system can fit into 2 UCS chassis!

The Next Big Thing -– In-Memory Elastic Database

In one of my recent talks, Elastic Data on the Cloud: Hype or Reality? I also discussed the trend in the context of the cloud database. I spoke about how the addition of elastic middleware  -- which enables allocation of data service on-demand, sharing that data with other tenants of the application, and provides built-in dynamic scaling as needed -- will make this sort of deployment significantly simpler and more economical.

  

What’s Next

In my last CTO Note I described our current progress with our 7.1 XAP release and our plans toward the future 8.x release:

Here is what we’ve done to take XAP even closer to this vision, in the recent 7.0 release and the upcoming 7.1 release (slated for April 14, 2010):

• Fast network –- XAP 7.1 will make dynamic scaling simple by enabling you to set up distributed system with a single API call. This not only provides a simpler API for developers, but also eliminates manual work for sizing, provisioning, and configuration at the deployment stage, and reduces the maintenance effort by automatically adjusting cluster configuration in response to changing loads, machine failure, and other events.
• Large memory -– XAP can manage terabytes of data in-memory. It can even manage up to 384 GB in a single VM, through its certification for use with Cisco UCS, coming up in version 7.1.
• Multi-core -– as of version 7.1, XAP comes with new concurrent transaction management for better utilization of multi-core processors. This comes on the heels of major improvements in multi-core utilization in version 7.0, which resulted in a 300% performance improvement on the Intel Nehalem processor. New benchmarks will shortly be released showcasing the performance of XAP 7.1 on Cisco UCS, which is based on the latest multi-core technology.
• Virtualization and multi-tenancy –- XAP 7.1 is the first middleware platform with built-in multi-tenancy, making it extremely simple to share applications on the same resources with full isolation and enterprise-grade security. This dramatically cuts the cost per user for software vendors that use XAP for SaaS-enablement, because it enables them to cram more users onto the same hardware resources. Full VMware integration is also available, enabling multi-tenancy throughout the entire virtualized application stack.

Our 8.x release will include more exciting stuff in the same direction, with JPA support and continuation of our work to make our elastic middleware simpler. We will also be releasing a tighter packaging with Cisco UCS that will make large data deployments fully optimized from the application down to the hardware.

References:

• InfoQ Article - RAM is the new disk...

• Why Existing Databases (RAC) are So Breakable!

• The Common Principles Behind the NOSQL Alternatives

• The Case for RAMClouds: Scalable High-Performance Storage Entirely in DRAM By Department of Computer Science Stanford University

• Are Cloud Based Memory Architectures the Next Big Thing?, highscalability.com

• The Missing Piece in the Virtualization Stack (Part 2)

The Common Principles Behind the NOSQL Alternatives

A few weeks ago, I wrote a post describing the drive behind the demand for a new form of database alternatives, often referred to as NOSQL. A few weeks ago during my Qcon presentation, I went through the patterns of building a scalable twitter application, and obviously one of the interesting challenges that we discussed is the database scalability challenge. To answer that question I tried to draw the common pattern behind the various NOSQL alternatives, and show how they address the database scalability challenge. In this post I'll try to outline these common principles. 

The Common Principles Behind the NOSQL Alternatives

Assume that Failure is Inevitable

Unlike the current approach where we try to prevent failure from happening through expensive HW, NOSQL alternatives were built with the assumption that disks, machines, and networks fail. We need to assume that we can’t prevent these failures, and instead, design our system to cope with these failures even under extreme scenarios.  Amazon S3 is a good example in that regard. You can find a more detailed description in my recent post Why Existing Databases (RAC) are So Breakable!.  There I outlined some of the lessons on how to architect for failures, from Jason McHugh's presentation. (Jason is a senior engineer at Amazon who works on S3.)

Partition the Data

By partitioning the data, we minimize the impact of a failure, and we distribute the load for both write and read operations. If only one node fails, the data belonging to that node is impacted, but not the entire data store. 

Keep Multiple Replicas of the Same Data

Most of the NOSQL implementations rely on hot-backup copies of the data, to ensure continuous high availability.  Some of the implementations  provide you with a way to control it at the API level, i.e. when you store an object, you can specify how many copies of that data you want to maintain at the granularity of an object level. With GigaSpaces, we are also able to fork a a new replica to an alternate node immediately, and even start a new machine if it is required. This enables us to avoid the need to keep many replicas per node, which reduces the total amount of storage and therefore cost associated with it.

You can also control whether the replication should be synchronous or asynchronous, or a combination of the two. This determines the level of consistency, reliability and performance of your cluster. With synchronous replication, you get guaranteed consistency and availability at the cost of performance (a write operation followed by a read operation is guaranteed to return the same version of the data, even in the case of a failure). The most common configuration with GigaSpaces, is synchronous replication to the backup, and asynchronous to the backend storage.

Dynamic Scaling

In order to handle the continuous growth of data, most NOSQL alternatives provide a way of growing your data cluster, without bringing the cluster down or forcing a complete re-partitioning. One of the known algorithms that is used to deal with this, is called consistent hashing. There are various algorithms implementing consistent hashing. 

One algorithm notifies the neighbors of a certain partition, that a node joined or failed. Only those neighbor nodes are impacted by that change, not the entire cluster. There is a protocol to handle the transitioning period while the re-distribution of the data between the existing cluster and the new node takes place. 

Another (and significantly simpler) algorithm uses logical partitions. With logical partitions, the number of partitions is fixed, but the distribution of partitions between machines is dynamic. So for example, if you start with two machines and 1000 logical partitions, you have 500 logical partitions per machine. When you add a third machine, you have 333 partitions per machine. Since logical partitions are lightweight (they are basically a hash table in-memory), it is fairly easy to distribute them. 

The advantage of the second approach is that it is fairly predictable and consistent, whereas with the consistent hashing approach, the distribution between partitions may not be even, and the transition period when a new node joins the network can take longer. A user may also get an exception if the data that he is looking for is under transition. The downside of the logical partitions approach, is that the scalability is limited to the number of logical partitions.

For more details in that regard, I recommend reading Ricky Ho's  post entitled NOSQL Patterns.

Query Support

This is an area where there is a fairly substantial difference between the various implementations.The common denominator is a key/value matching, as in a hash table. Some implementations provide more advanced query support, such as the document-oriented approach, where data is stored as blobs, with an associated list of key/value attributes. In this model you get a schema-less storage that makes it easy to add/remove attributes from your documents, without going through schema evolution etc. With GigaSpaces we support a large portion of SQL. If the SQL query doesn’t point to a specific key, the query is mapped to a parallel query to all nodes, and aggregated at the client side. All this happens behind the scene and doesn’t involve user code.

Use Map/Reduce to Handle Aggregation

Map/Reduce is a model that is often used to perform complex analytics, that are often associated with Hadoop. Having said that, it is important to note that map/reduce is often referred to as a pattern for parallel aggregated queries. Most of the NOSQL alternatives do not provide built-in support for map/reduce, and require an external framework to handle these kind of queries. With GigaSpaces, we support map/reduce implicitly as part of our SQL query support, as well as explicitly through an API that is called executors. With this model, you can send the code to where the data is, and execute the complex query directly on that node.

For more details in that regard, I recommend reading Ricky Ho's post entitled Query Processing for NOSQL DB.

Disk-Based vs. In-Memory Implementation

NOSQL alternatives are available as a file-based approach, or as an in-memory-based approach. Some provide a hybrid model that combines memory and disk for overflow. The main difference between the two approaches comes down mostly to cost/GB of data and read/write performance.

An analysis done recently by Stanford University, called “The Case for RAMCloud” provides an interesting comparison between the disk and memory-based approaches, in terms of cost performance. In general, it shows that cost is also a function of performance. For low performance, the cost of the disk is significantly lower the RAM-based approach, and with higher performance requirements, the RAM becomes significantly cheaper.

The most obvious drawbacks of RAMClouds are high cost per bit and high energy usage per bit. For both of these metrics RAMCloud storage will be 50-100x worse than a pure disk-based system and 5-10x worse than a storage system based on flash memory (see [1] for sample configurations and metrics). A RAMCloud system will also require more floor space in a datacenter than a system based on disk or flash memory. Thus, if an application needs to store a large amount of data inexpensively and has a relatively low access rate, RAMCloud is not the best solution.
However, RAMClouds become much more attractive for applications with high throughput requirements. When measured in terms of cost per operation or energy per operation, RAMClouds are 100-1000x more efficient than disk-based systems and 5-10x more efficient than systems based on flash memory. Thus for systems with high throughput requirements a RAM-Cloud can provide not just high performance but also energy efficiency. It may also be possible to reduce RAMCloud energy usage by taking advantage of the low-power mode offered by DRAM chips, particularly during periods of low activity.  In addition to these disadvantages, some of RAM-Cloud's advantages will be lost for applications that require data replication across datacenters. In such environments the latency of updates will be dominated by speed-of-light delays between datacenters, so RAM-Clouds will have little or no latency advantage. In addition, cross-datacenter replication makes it harder for RAMClouds to achieve stronger consistency as described in Section 4.5. However, RAMClouds can still offer exceptionally low latency for reads even with cross-datacenter replication.

Is it Just Hype?

One of the most common questions that I get this days is: “Is all this NOSQL just hype?”, or “Is it going to replace current databases?"

My answer to these questions is that the NOSQL alternatives didn’t really start today. Many of the known NOSQL alternatives have existed for more than a decade, with lots of successful references and deployments. I believe that there are several reasons why this model has become more popular today. This first is related to the fact that what used to be a niche problem that only a few fairly high-end organizations faced, became much more common with the introduction of social networking and cloud computing. Secondly, there was the realization that many of the current approaches could not scale to meet demand. Furthermore, cost pressure also forced many organizations to look at more cost-effective alternatives, and with that came research that showed that distributed storage based on commodity hardware can be even more reliable then many of the existing high end databases. (You can read more on that here.) All of this led to a demand for a cost effective “scale-first database”. I quote James Hamilton, Vice President and Distinguished Engineer on the AWS team, from one of his articles One Size Does Not Fit All:  

“Scale-first applications are those that absolutely must scale without bound and being able to do this without restriction is much more important than more features. These applications are exemplified by very high scale web sites such as Facebook, MySpace, Gmail, Yahoo, and Amazon.com. Some of these sites actually do make use of relational databases but many do not. The common theme across all of these services is that scale is more important than features and none of them could possibly run on a single RDBMS”

So to sum up – I think that what we are seeing is more of a realization that existing SQL database alternatives are probably not going away any time soon, but at the same time they can’t solve all the problems of the world. Interestingly enough the term NOSQL has now been changed to Not Only SQL, to represent that line of thought.

References

• Recorded Dzone Webinar on NOSQL Alternatives
• NoSQL Alternatives: Dzone Inteview   by Nitin Bharti
• No to SQL? Anti-database movement gains steam – My Take
• NoSQL And Elastic Caching Platforms Are Kissing Cousins by Mike Gaultieri Forrester
• Why Existing Databases (RAC) are So Breakable!
• NoSQL Ecosystem By Jonathan Ellis,
• NoSQL: Distributed and Scalable Non-Relational Database Systems by Jeremy Zawodny
• NOSQL Patterns by Ricky Ho
• Query Processing for NOSQL DB by Ricky Ho
• One Size Does Not Fit All  by James Hamilton
• NOSQL debrief by Johan Oskarsson,
• The Case for RAMClouds: Scalable High-Performance Storage Entirely in DRAM by Department of Computer Science Stanford University
• GigaSpaces Map/Reduce API

Why Existing Databases (RAC) are So Breakable!

One of the core assumption behind many of today’s databases is that disks are reliable. In other words, your data is “safe” if it is stored on a disk, and indeed most database solutions rely heavily on that assumption. Is it a valid assumption?

Disks are not reliable!

Recent analysis of disk failure patterns (see references below) break the disk reliability assumption completely. Interestingly enough, even though these studies were published in 2007, most people I speak to aren’t aware of them at all. In fact, I still get comments from people who are basing their reliability solution on disk-based persistency, insisting on the argument that whether or not you are reliable amounts to whether or not your data is synched to a disk.

Below are some of the points I gathered from the research, which should surprise most of us:

• Actual disk failure/year is 3% (vs. estimates of 0.5 - 0.9%) – this is a 600% difference on reported vs. actual disk failure. The main reason is that manufacturers can’t tests their disks for years, so their tests are an extrapolation of statistical behavior which proves to be completely inaccurate.
• There is NO correlation between failure rate and disk type – whether it is SCSI, SATA, or fiber channel. Most data centers are based on the assumption that investing in high-end disks and storage devices will increase their reliability – well, it turns out that high-end disks exhibit more or less the same failure patterns as regular disks! John Mitchell had an interesting comment on this matter during our Qcon talk, when someone pointed to RAID disks as their solution for reliability. John said that since RAID is based on an exact H/W replica that lives in the same box, there is a very high likelihood that if a particular disk fails, its replica will fail in the same way. This is because they all have the exact same model, handle the exact same load and share the same power/temperature. 
• There is NO correlation between high disk temperature and failure rates – I must admit that this was a big surprise to me, but it turned out that there is no correlation between disk temperature and disk failure. In fact, most failures happen when disks were at low or average temperature.

Why are existing database clustering solutions so breakable?

Most existing database clustering solutions rely on a shared disk storage to maintain their cluster state, as can be seen in the diagram below.

 

Typical database clustering architecture based on shared storage

The core, implicit assumption behind Oracle RAC, IBM DB2, and many other database clustering solutions, is that failure can be avoided by purchasing high-end disk storage and using expensive hardware (fiber optics, etc). As can be seen from the research I mentioned earlier, this core assumption doesn’t correlate with the failure statistics. Hence I argue that the database clustering model is inherently breakable.

Failure is inevitable – cope with it

Jason McHugh, a senior engineer at Amazon who works on S3, gave a very interesting talk at the Qcon conference (Amazon S3: Architecting For Resieliency In The Face Of Failures), where he outlined the service failure handling architecture of Amazon’s Simple Storage. Jason’s main point was that rather than trying to prevent failure from happening, you should assume that failure is inevitable and instead, design your system to cope with it.

Failure can happen in various shapes and forms:

• Expect drives to fail
• Expect network connections to fail (independent of the redundancy in networking)
• Expect a single machine to go out

Human error example: Failure to use caution while pushing a rack of servers

How to design your application for resilience in the face of failure

Jason outlined some of the lessons from his experience at Amazon:

• Expect and tolerate failures – build your infrastructure to cope with catastrophic failure scenarios: "Distributed systems will experience failures, and we should design systems which tolerate rather than avoid failures. It ends up being less important to engineer components for low failure rates (MTBF) than it is to engineer components with rapid recovery rates (MTTR)." (from Architecture Summit 2008 - Write up)
• Code for large-scale failures – design your application to cope with the fact that services can come and go.
• Expect and handle data and message corruption – don’t rely only on TCP to ensure the consistency of your data. Add validation as part of your application code to make sure that messages are not corrupted.
• Code for elasticity – load/demand may grow unexpectedly. Elasticity is essential to dealing with unexpected demand without bringing the system to its knees. It is also important that when you exceed your capacity, the system won’t crash but instead report denial of service.
• Monitor, extrapolate and react – monitoring at the right level enables you to detect failures before they happen and take the appropriate corrective actions.
• Code for frequent single machine failures – rack failure can cause an immediate disappearance of multiple machines at the same time.
• Game days – simulate failures and learn how to continuously improve your operational procedure for handling failures and architecture.

Memory can be more reliable then disk

Many people assumes that memory is an unreliable data storage.  That assumption holds true if your data “lives” on a single machine; in this case if the machine fails or crashes your application crashes. But what if you distribute the data across a cluster of nodes and maintain more than one copy of the data over the network? In this case, if a node crashes the data is not gone; it lives elsewhere and can be continuously served from one of its replicas.

Under these conditions, I can argue that under certain conditions, memory can be more reliable than disk. One of the reason is the hardware itself – unlike disk drives that rely on mechanical moving parts, RAM is just a silicon chip. As such the chances for failure can be significantly smaller. Unfortunately, I don’t have the data to back up this argument, but I think it’s very likely that this is the case. In an article published on InfoQ titled RAM is the new disk, Tim Bray brings an interesting observation on this matter:

Memory is the new disk! With disk speeds growing very slowly and memory chip capacities growing exponentially, in-memory software architectures offer the prospect of orders-of-magnitude improvements in the performance of all kinds of data-intensive applications. Small (1U, 2U) rack-mounted servers with a terabyte or more or memory will be available soon, and will change how we think about the balance between memory and disk in server architectures. Disk will become the new tape, and will be used in the same way, as a sequential storage medium (streaming from disk is reasonably fast) rather than as a random-access medium (very slow). Tons of opportunities there to develop new products that can offer 10x-100x performance improvements over the existing ones.

As a matter of fact we (GigaSpaces/Cisco) are working these very days on benchmarking  Cisco UCS, where we are evaluating the possibility to manage x100GB and potentially even Terabytes of data in a single node. The results look extremely promising, so the future is not that far ahead.

References

• KHB: Real-world disk failure rates: surprises, surprises, and more surprises by Valerie Aurora (Henson)
• Disk failures in the real world: What does an MTTF of 1,000,000 hours mean to you? By Bianca Schroeder and Garth A. Gibson
• The Case for RAMClouds: Scalable High-Performance Storage Entirely in DRAM By Department of Computer Science Stanford University
  
• Amazon S3: Architecting For Resieliency In The Face Of Failures By Jason McHugh
• Failure Trends in a Large Disk Drive Population - Google, Inc., February 2007
• InfoQ Article - RAM is the new disk...
• Are Cloud Based Memory Architectures the Next Big Thing?, highscalability.com