Nati Shalom's Blog

A interesting note on Google Megastore & CAP,..

I was reading James Hamilton's coverage on Google Megastore: The Data Engine Behind GAE (a must read!), where he summarizes the main architecture assumption behind Google Megastore.

The thing that caught my eyes was the following line:

Support for consistency unusual for a NoSQL database but driven by (what I believe to be) the correct belief that inconsistent updates make many applications difficult to write (see I Love Eventual Consistency but …)

This was later further elaborated upon in one of James' comments below the posting:

The cap theorem is undeniably true but, like many engineering issues, it requires judgment. Consistency can be maintained over remarkably a large scope of data. The availability that CAP forces to be given up certainly exists but can be made quite small. Overall, I would say that CAP encourage many to give up hope way too quickly and build an eventually consistent system when better choices are available with more work. The paper under discussion here shows that a high-scale store can be fully consistent over substantial volumes of data and they wisely decided to not make the entire store globally consistent. Interpreting and applying CAP is a judgement call and we see one approach here. SimpleDB chose to do both (http://perspectives.mvdirona.com/2010/02/24/ILoveEventualConsistencyBut.aspx) which is along the lines you recommend in your conclusion: “as scalable data sources mature they will increasingly and perhaps ubiquitously allow user-selectable CAP choices in much the same way that SQL stores almost always allow user-selectable transaction consistency”. I think you are right.

Interestingly enough James also points to another interesting part of his previous post on the subject (I love eventual consistency but...) where he describes how Amazon deals with consistency tradeoffs with SimpleDB:

What I’ve learned over the years is that strong consistency, if done well, can scale to very high levels. The trick is implementing it well. The naïve approach to achieve full consistency is to route all updates through a single master server but clearly this won’t scale. Instead divide the update space into a large number of partitions, each with its own master. That scales but there is still a tension between the number of partitions and the cost of maintaining many partitions and avoiding hot spots. The obvious way to avoid hot spots is to use a large number of partitions but this increases partition management overhead. The right answer is to be able to dynamically repartition to maintain a sufficient number of partitions and to be able to adapt to load increases on any single server by further spreading the update load.

I found those comments particularly interesting as many people refer to Google and Amazon as the poster children for justifying eventual consistency.

It is interesting to see that the reality is that even Google and Amazon - which I would consider the extreme cases for big data - realized the limitation behind eventual consistency and came up with models that can deal with scaling without forcing a compromise on consistency as I also noted in one of my recent NoCAP series here, here and here.

Speaking of Score Card..

They say that timing is everything in life.. this time was no exception.

Yesterday I published my score card for 2nd generation PaaS and how it differs from Application Servers.

A few hours later, I received an email with a presentation from one of our new customers that is going to present at our GigaSpaces Roadshow.

This slide caught my attention:

 

* Note that GS stands for GigaSpaces.

The customer is a large ISV (1900 employee, $3B in revenue...). Their new product is a real-time BI system that needs to process events in real time from multiple data sources -– social media included (most other alternatives do that in batch mode) and produce actionable reports on customer activities. They had fairly challenging scaling, latency, and cost requirements:

• Massive Scale, 100M Customers, a billion events a day
• Business real time < 100ms
• Low cost 0.05 customer/year

Now it is not often that a company the size of GigaSpaces competes with giants like IBM and Oracle on one of their core IPs and leaves them so far behind.

How was this possible?

Well, our advantage is in our disadvantage –- size. Let me explain.

Removing the Silos

GigaSpaces as a case study

As a small company, we have to be extremely efficient at what we do.  We adopted Lean methodology and Scrum across the board, and we stick to that model quite religiously.  Every feature goes through millions of automated tests. We manage to build GA-quality releases every two weeks.

On the product side, we have designed our product in a way that follows the principles that I laid out in my previous post, i.e. -- without Silos. In GigaSpaces everything is mapped to data. Tables, Queues, even method invocations are mapped to the same underlying data structure representation. 

We maintain ONE cluster throughout the entire stack to manage this scalability and high availability of the data.  ALL of our APIs are simple facades on top of our shared data cluster (a.k.a space). This means that when we optimize performance, fix bugs, improve scalability, and add new features -- it applies immediately through the entire stack, including the .Net and C++ parts of our product.

The fact that all of our APIs share the same data opened new opportunities for optimization that weren’t possible before. For example, we can avoid a lot of the synchronization overhead that is associated with passing a message and then updating it. We can also mesh the driftnet APIs  without unnecessary copies.

As an example, I can choose to use the key/value store API for low latency transaction processing and the SQL API for analytics over the same data that was produced through the key/value APIs.

Because of our size, we couldn’t afford the high maintenance and support that often comes with the type of deployments that we deal with. As a result, we were forced to develop lots of built-in automation and troubleshooting tools as an integrated part of our product, and later we integrated it with the development framework and application in such a way that every application is made operationally ready from its inception.

Now, the nice thing in all this is that both we and our customers benefit from that efficiency. We gain more customers without increasing our costs. Our customers get a better quality product, short fix cycles, and more features faster.

The latest 8.0 release is the best evidence of this, IMO. We managed to deliver 3 times more features than what we normally accomplished in previous releases, and all that with only a minor increase in our development team.

 *A Little Promotion*

If you're in London on the 2nd of February , Paris on 3rd, San Francisco on 7th, or NYC on the 9th, I’d like to welcome you to join our Roadshow and hear more first-hand stories from these customers (Dow Jones, Sears, Amdocs, and many more top-tier organizations). See you there!

You can register and read the full details here.