Nati Shalom's Blog

Bringing Data Awareness to the Grid (and Amazon EC2)

For those who aren't familiar with the concept of Grid Computing, a brief primer:

Grid is a well known paradigm designed to reduce the time it takes to perform a batch job through distributed parallelization.

How does it work?

Most grid solutions are built out of three main components:

1. Compute Agents - a lightweight process that makes a compute resource grid-aware; i.e., available to the grid.
2. Resource Manager - collects and monitors information about availability of compute resources, as well as maintaining the liveliness of those resources.
3. Scheduler - this is where the heavyweight work is done. The scheduler is responsible for matching between the task specification and the available resources.

Different solutions in the market (such as DataSynapse, GridGain, Grid Systems, and Platform Computing) refer to those three by different names, but in general the principles are pretty much the same.

Using a Compute Grid allows us to break a batch job into smaller units of work, a.k.a  tasks. At this point we can submit the job to the scheduler, specifying the requirements and specifications (Service Level Agreements) required for running the job. Requirements can be specified as a type of OS/HW pre-requisites, etc. The scheduler then takes those requirements and executes the job on the available compute resources that match those requirements. Compute farms can be as large as 10Ks of CPUs.

This approach was used with great success in the past few years, and today most financial institutions have a compute backbone as a core piece of their IT infrastructure.

Utilizing Compute Grids for a Wider Range of Applications

It is now time to take advantage of the success of Compute Grids and apply it to a wider range of applications within the organization, including to those that are considered very high-end, such as front-office applications in capital markets (trading, etc.). A key for enabling this type of applications on the Grid is solving the data bottleneck. This is where In-Memory Data Grids come to the rescue.

Victoria Livschitz from Grid Dynamics gave a presentation in Spain last week During an HP event in which she discussed the various options for integrating the two complementing technologies. Her scenarios ranged from a relatively trivial scenario in which the two solutions “live” together, side by side, in the same environment with very little integration between them; and ended with a complete data-aware scheduling, in which the compute grid becomes data-aware and is using this awareness to optimize the way jobs are routed and executed. Livschitz describes a case study for creating inter-day P&L (Profit and Loss) reporting for a trading application using the GigaSpaces In-Memory Data Grid and DataSynapse Compute Grid. In that scenario, Grid Dynamics managed to reduce the application start-up time from 40 min to 15 min and access time from couple of minutes to seconds, by putting the database behind an In-Memory Data Grid. This enabled the move from post trade reporting (over night) to inter-day reporting. View Livschitz' presentation (pdf).

Compute Grids for the Masses

Due to the large amount of required resources, Grids were so far mostly used by large organizations, such as major financial or manufacturing organizations. However, the introduction of Amazon Elastic Compute Cloud (Amazon EC2) brings the grid to the masses, and with the recent upgrades made to this platform, it is becoming applicable to both low-end and high-end applications. In fact, at GigaSpaces, we're developing an integration with Amazon EC2 that would make it data-aware and therefore will meet the tough requirements of those high-end applications. As Geva Perry wrote in his blog post Scaling Stateful Applications on Amazon EC2:

...we've made available a public AMI (Amazon Machine Image) for the GigaSpaces XAP platform. We're still working on improvements and optimizations, but it is ready for people to start playing around with.

• You can find the GigaSpaces AMI here.
• General description in the Amazon Web Services Solution Catalog here.
• Detailed paper with set up instructions and code examples here (PDF).

We're already working with a couple of beta customers on EC2, including some folks who are building a Web 2.0 social networking type sites.

Stay tuned for updates on this topic!

Why most large-scale Web sites are not written in Java [Personal View]

In my previous post, I asked the question: Why most large-scale Web sites are not written in Java. That question triggered very active discussions in the comments to my post, as well as on TheServerSide, and more recently, on Artima.

The responses to this question varied from those who took it as an offense against Java, and replied that Java is used in large-scale web applications (and even gave specific names and references); and those who said that LAMP is probably a better choice because it is much simpler. I picked Mike O'Keefe response on Artima, which IMO provides a good answer to this question:

1. Some of the biggest sites are using LAMP since that's what was proven, reliable, and had market share and skilled programmers, when they were written. Take Lands End, one of the first e-commerce sites, written in perl, C, running on Oracle DB. Or Slashdot - perl. Still is, I think. Certainly Slashdot scales, given the "slashdot effect", when Slashdotters hit a linked site, which invariably falls over.

2. All of the sites in Shalom's list are what I would call "content management", not heavy on the business logic. I think Shalom understands this as web vs. financial. But the point here is, if you are serving up mostly static pages, without too much business logic, certainly those are different requirements than a "financials" type app. Take Flickr- photos, or YouTube - video, Digg - text messages, etc.

3. In addition to point #1, it also is important to consider the skills of the person writing the app. If the folks who wrote e-bay, as legend has it, over a weekend with a few perl scripts, or Facebook (PHP), in a similarly short time I think, followed the usual procedure required for your basic business app, they'd still be hashing out the requirements ;) Additionally, while certainly there are tools to simplify this (Rational Application Developer, NetBeans, Eclipse, AppFuse), but I can guarantee the effort, including steep learning curve required to hack out a simple app in J2EE (or Spring, or Struts, etc), on top of learning Java, is considerably more than hacking together a few perl scripts (or, nowadays, Ruby/Rails). To put it another way, Bill Gates scraped together 50K and bought the rights to Q-DOS. It didn't matter that even then were much better OSs out there, like CP/M. Marketshare's the key. Time enough to rewrite and build on more solid ground like NT. What's important is to take the time to do that, this will be key for Facebook.

It was quite amusing to read that some thought that I was anti-Java myself☺ (Thanks to Guy Nirpaz, my colleague at GigaSpaces, for clearing that up). Some felt that the question itself was misleading. Well, I think that they are probably right in saying so. Finding an accurate way to put such question is very challenging.
Others felt that this was some kind of a marketing spin.

What seemed to be common to a lot of the responses is that the Java community tends to be fairly protective and defensive at first to new ideas, and many in the community view them as a threat. By doing so, we are missing a great opportunity to learn from approaches taken in the LAMP world, which address similar problems faced by the JEE community. Clearly there must be interesting lessons we can learn from them.

We simply can't ignore the fact that the sites that I was referring too were able to deal with very large-scale requirements without Java, and be very successful.

Final Words

The final word on what should be the *right* application stack hasn't been spoken yet, and we are probably going to see more of that debate in the future. The interesting question that remains open is whether the Java/JEE world and LAMP will converge, or continue a parallel evolution.

There are new emerging frameworks that try to bridge between the two worlds: Run PHP Web Applications on the Java Platform  is a good example, and Caucho and Spring On Rails. There are plenty of other emerging frameworks that I haven't listed that try to address the same challenge. Is that going to lead to convergence? - I certainly hope so. IMO there is a good chance that such convergence will happen simply because the challenges that even a relatively simple site needs to deal with become more complex when you add scalability to the list of requirements. This makes the simplicity argument in the LAMP stack significantly weaker,and the value of Java more relevant. It is now up to the Java community to embrace the LAMP world and not treat it as a threat.

Why most large-scale Web sites are not written in Java

During the past few weeks I've had discussions with my colleague Geva Perry trying to answer the question Why most large-scale Web sites are not written in Java?

There is a lot of information in the blogosphere describing the architecture of many popular sites, such as Google, Amazon, eBay, LinkedIn, TypePad, WikiPedia and others.

The folks at Pingdom compiled some of this information, based on information from High-Scalability:

Looking at these architectures some observations come to mind: Most of these sites are using LAMP as the core runtime stack. Some have gone so far as to develop their own file system (Google, GFS). Some are using caching to solve the database bottleneck (memcached and the like). Many of them were forced to develop these solutions themselves, as at the time there was no ready-made alternative that could meet their requirements.

The application stack of these Web applications is very different from the stack that mission-critical applications in the financial world are built with. In the financial world, Java -- and to a lesser degree J2EE -- is used extensively. In recent years scalability requirements in capital markets led to a rapid shift in the middleware stack, introducing Compute Grid solutions for virtualization of CPU resources, enabling parallelization of batch applications. Data Grids were also introduced, enabling the virtualization of memory resources. Spring is becoming the common development framework in this world. At GigaSpaces, we're seeing more and more cases where Spring acts as a complete alternative to J2EE.

If we examine both worlds, we can see that both are facing similar challenges related to scalability. Not surprisingly, both ended up introducing similar solutions for addressing the scalability challenges:

On the Data Tier we see the following:

1. Adding a caching layer to take advantage of memory resources availability and reduce I/O overhead
2. Moving from a database-centric approach to partitioning, aka shards  

On the Business Logic Tier:

3. Adding parallelization semantics to the application tier (e.g., MapReduce)
4. Moving to scale-out application models to achieve linear scalability
5. Moving away from the classic two-phase commit and XA for transaction processing  (See: Lessons from Pat Helland: Life Beyond Distributed Transactions)

While there are many similar challenges, and to a certain degree, similar architectures, it seems that both worlds (Web and Financial) took different routes as it relates to the application stack.

Over at the High-Scalability site, someone posted the question: Why doesn't anyone use j2ee?
The answer given in that post can be summarized as follows:

1. LAMP provides a cost-effective solution (most of it relies on *free* open source stack).
2. Java is still used, but not as the primary language, i.e., it is used as one component either in the back-end or the front-end (e.g., servlets).

I have my own thoughts on this matter, but I'll be very interested to see if anyone has any reasonable explanation for it, before I jump in.

Thoughts?

UPDATE (October 11, 2007): This post generated a very active debate in several places, including TheServerSide, and more recently, on Artima. In this post I respond and give some additional thoughts.