Nati Shalom's Blog

Need scalability? Don't forget pricing

In most discussions about scalability, we often approach the topic as a pure technical/architecture challenge, and ignore cost issues. The problem is that when we truly scale our application, and want to benefit from economies of scale, we're going to end up with scale limitations, not because of technical issues, but because of the pricing  and licensing models.

Scalable pricing

Scalable pricing means a pricing scheme that provides the benefits of economies of scale. Below are pricing models commonly used for software products and how they fit in the new dynamically-scalable world.

• Free - while this certainly sounds like the best option (and may very well be) the customer needs to be aware of the following:

- The free license of a software product typically does not include support: not an option for most mission critical applications.
- When you do pay extra for support, you will typically be charged just like any other run-time license on a per CPU basis.
- Make sure that the company behind the product has a sustainable business model, otherwise there is a good chance that it will either die when its funding dries up or change its license model to monetize its user base. That's fine, but all it means is that it's not really a free offering in the long run, and you don't know what the pricing model will be exactly.
- In terms of total cost of ownership (TCO), free products are not necessarily the cheapest option. TCO is dependent on many factors, for example, dependency on other products (and their license costs), the need for integration and maintenance, etc. See my post, Economies of non scale, for more on the topic.

• Subscription model - With a subscription model you pay a fixed periodical fee, typically on an annual basis for infrastructure software, and on a monthly basis for SaaS. Subscription pricing is suitable for on-demand scalability as it provides the flexibility to grow or reduce cost based on the annual use of the product.
• Pay per use - this model is even more flexible then subscription model as it gives you higher granularity. Pay per use is provides in various forms where the usage can be a measure of CPU utilization or bandwidth utilization. Amazon for example charge per machine utilization for its EC2 services and data-utilization for its data services.
• Perpetual license - This model is used to buy licenses in advance and pay for support separately (normally 15-20% on top of the per CPU license). This is the most commonly used model with commercial software products, however, due to the large initial investment required by this model, it doesn't fit well with on-demand environment.
• Enterprise unlimited license - This model enables you to pay premium price in advance (based on potential future usage) and gives you the freedom to use the software without any limit. This model fits to environment where you anticipate that over a fairly short period of time the usage of the product will become wide and therefore the pay-per use or any of the other models mentioned above will become more expensive.

Which model to choose?

Each of the models has pros and cons and therefore the answer depends on your situation. Also, over time, as the situation changes, you will probably realize you need a different license model, and so it becomes equally important that the product you choose will give you the freedom to move from one model to another in the future.

GigaSpaces scalable pricing

With GigaSpaces we continuously look into ways to make our software license cost fit the on-demand world. For example, we launched a free Start-Up program that provides a totally FREE version of GigaSpaces for startups (hundreds of start-ups have already signed up for this program since we launched it last year). We also provide a Pay-Per-Use model for those running on Amazon EC2.

We felt that even though this is a fairly flexible pricing, we could do better. As of our 6.6 release, we added the option to buy our software at a yearly subscription price, and we also launched a new package called XAP Standard Edition, which is sold at a very low price of $9,500k per package (not CPU) where the package includes two servers, 4 GigaSpaces nodes and up to 50 clients or remote servers.

These changes were designed to address the needs of developers looking to start running their applications at a relatively low scale, who need the full functionality of the product, but cannot afford the full XAP price. Another principle that we kept when we designed this package is that moving from Standard to Premium edition wouldn't require any change in your architecture or code - which means that you could always scale to the premium edition just by changing the license key.
More details about the new pricing model is available here

Other references:
GigaSpaces and the Economics of Cloud Computing

Economies of Non-Scale

GigaSpaces as Alternative to GoogleAppEngine for the Enterprise

I Just came across an interesting post by Josh Heitzman who writes about his negative experience with Google App Engine, which led him to examine a list of Alternatives to Google App Engine. He points out GigaSpaces XAP as one of them:

One particularly interesting EC2 third party provider is GigaSpaces with their XAP platform that provides in memory transactions backed up to a database. The in memory transactions appear to scale linearly across machines thus providing a distributed in-memory datastore that gets backed up to persistent storage. A lot of the docs reference Java, but the page returned by the aforementioned link states “…deploy applications that use Java, .Net, C++, or even scripting languages…” so after a cursory investigation it is not clear what aspects of their platform is only accessible via Java and which aspects are generally accessible. Bears more investigation.

[See my comment to the post relating to the question about .Net and C++ support]

Josh's post raises the question of what is Platform-as-a-Service (PaaS)?

Platform-as-a-Service is a term used to describe a new set of development platforms that are typically accessible through the web. These platforms enable you to develop new applications easily, without the need to install any software or set up a development and deployment environment. A good example of this is Force.com from Scalesforce.com. Other SaaS providers have similar platforms. It seems that the common motivation behind this trend is to enable the SaaS provider a way to expose their internal framework to other partners and users and build an eco-system around their SaaS product. This led to the emergence of dedicated, proprietary platforms. Google App Engine is a similar effort from Google initiative to expose some of their own platform to external users.

These platforms, as Josh experienced, were not designed as a general purpose enterprise platform, and therefore, it is not surprising that they lack many of the elements that you would normally expect from enterprise middleware, such as transaction support, security and standard APIs.

Unlike such Internet based platforms, GigaSpaces XAP was primarily designed as an enterprise middleware platform. It is used in the most demanding mission-critical applications that require extreme scalability and low-latency. During the past year we have extended our middleware platform to the Internet cloud, starting with tight integration with Amazon EC2 and followed with partnerships and integration with leading players in the market, including GoGrid, Joyent, RightScale, CohesiveFTand others. We recently launched our cloud framework in private beta. It enables building enterprise applications on GigaSpaces XAP via the internet. In this way, you can run an application on a hosted GigaSpaces environment, without even downloading the GigaSpaces software.

What makes GigaSpaces XAP an alternative to Google App Engine for the enterprise?

• Support for existing enterprise applications:

One of my previous posts discussed: Google App Engine - what about existing applications?

In this post, I want to reiterate this point, which goes to the heart of one of the main differences between GigaSpaces XAP and most Internet based PaaS, including Google App Engine and Force.com. Many enterprise applications are already built in Java, .Net or C++. To support enterprise applications, you first need support for these core languages. GigaSpaces XAP not only supports these languages, but also enables efficient interoperability among them.

• No vendor lock-in: 

While I would argue that lock-in is unavoidable at some level, and that every platform imposes some lock-in, I'd also argue that it is important to examine the nature of the lock-in, and how easy it is to migrate from one platform to another. With XAP, we invested heavily in making lock-in minimal through abstraction, aspects, support for standard APIs and more. As of 6.6, we users can take existing web applications and deploy them on our platform without touching the application code. You can read more about this in: Can scaling be made seamless?
To make things easy, we published a migration guide that shows step-by-step how you can take existing transactional JEE applications and deploy them on our platform (locally or on the public cloud). We measured the performance and scalability gain you get by running JEE applications on GigaSpaces XAP versus traditional JEE application servers. We ran the exact same application code on both platforms and measured at least 5 times better scalability efficiency and performance increased as outlined here. We also published a new "pet clinic" demo that comes with source code, configuration and documentation, and can be used as a reference guide for running standard JEE application with zero or with minimal code changes. This reference implementation is available here.

• Designed to support both local enterprise clouds and Internet clouds:

Although GigaSpaces can now run entirely on a public cloud, such as EC2, it is clear that many enterprises are not ready to run on public clouds, but would rather run their apps on an on-premise, private cloud. Supporting this requires existing development tools used to develop enterprise applications. XAP enables use of common development frameworks (e.g., Eclipse, Maven, Ant, in Java; and Visual Studio in .Net). You can write, test and debug your application locally and then deploy it on the cloud for testing or for production. You can decide at any point where you want to deploy your application, whether on a public Internet cloud, or on a private cloud in the corporate data center. You can also create a hybrid model that involves both a public and private cloud simulataneously.

•  Enterprise-grade reliability and scalability:

Most Internet-based PaaS impose a radical shift in the way applications are built, and specifically on scalability and reliability. Many of them leave you to deal with failure scenarios on your own, or alternatively, force you to accept the fact that you may lose data if you want to achieve scalability. They also require that you re-write your application if you want to make it scalable.
Many of the assumptions the platforms operate under are not acceptable to enterprise-grade applications. GigaSpaces XAP was designed to meet the most demanding requirements for maintaining both scalability and 24/7 high-availability without losing any data and without compromising scalability or performance.

This is only a partial list of differences, but I think that it makes clear how GigaSpaces is different than most Internet-based PaaS offerings, including Google App Engine.

You can read more about our cloud offering on gigaspaces.com/cloud. If you are interested, try out our new Cloud Framework and see for yourself how easy it is to set up a production-ready cluster with load- balancing and scalability within minutes.

Financial crisis explained

For those of you who are having trouble understanding the current financial crisis just like i do, i would strongly recommend listening to the video below by Marketplace Senior Editor Paddy Hirsch.

A picture is worth a thousand words, or better, A video is worth a million words:

Cloud Summit Event in Sunny Tel Aviv

Cloud summit event is coming up with lots of interesting lectures from Google, Yahoo, Amazon, eBay
You can see the full program here.

In my presentation "Getting ready for the clouds" I'm going to talk about the steps that can be taken to bring existing application to the cloud and use a live demo that shows a real life demo on how easy it deploy a production ready application with load-balancing, dynamic-scaling, caching and data-base in just few minutes. I'm also going to show what self-healing really means by killing one of the machines and seeing the impact of that failure on the application.

Another nice thing about this event is that its going to happen at perfect timing for those who want to run away from the real winter clouds and enjoy the sun and beaches of Tel Aviv.

Making EDA programming simple with JeeWiz

Event Driven Architecture (EDA) is becoming more popular these days, as the drive for loosely coupled and scalable architecture forces us to break our systems into components and integrate them through some sort of workflow.  Having said that, thinking in asynchronous events is not a trivial concept to deal with, seeing as we used to thinking and programming in a synchronous manner.

Space-Based Architecture lends itself very nicely to EDA, because it provides a means to register for events, manage the state of events and trigger different business logic elements based on state changes.
This makes the programming of EDA relatively simple compared with some of the other options, such as messaging and database systems. The following diagram shows how a typical EDA would look like in a Space-Based world - you can read the full description here.

While Space-Based Architecture makes EDA relatively simple compared with alternatives it can be made even simpler using advanced code generation tools that follows the Model Driven Development pattern.

JeeWiz is one of the leading products in that space: 

"The goal of JeeWiz is to automate software development as much as possible. JeeWiz builds all the code, configuration and build jobs that can be derived from high-level models of a system, achieving unprecedented levels of automation."

Matthew Fowler, Founder and CEO, New Technology/Enterprise Ltd. gave a presentation in our latest London Event introducing GigaSystemBuilder using JeeWiz which enables a model-driven development with GigaSpaces. JeeWiz is an Eclipse-based tool that makes it easy to create an entire project fairly easy. The product itself is highly customized. Users can use the same model to build their own templates, and in this way automate a large part of their development. The following diagram taken from Matthew's presentation, shows how a typical development process would look like with JeeWiz.

Matthew's presentation contains more details about the specific integration with GigaSpaces and what the generated code would look like -- I would highly recommend looking into it. The presentation is available online here. I was also happy to see that the GigaSpacesBuilder Eclipse-plugin is now available for download here. It comes with full documentation and an easy guide to get you through the first steps.

Well done Mathew and the JeeWiz team!

Is MapReduce going mainstream?

I'm getting a lot of questions lately about the use of MapReduce: how it compares with other technologies such as Grid, and how the the different solutions that claims support for MapReduce (GigaSpaces included) fit into the puzzle. A good starting point is the intense discussion on the cloud computing mailing list under the topic: "Is Map/Reduce going mainstream?" where I contributed some of my own thoughts on the topic.

To summarize the questions on this topic, I'd state it as follows:

1. How can we reduce the barrier-to-entry for implementing MapRreduce specifically, and parallel processing applications in general?

2. Many of the use cases for MapReduce represent some sort of data analytic application. But can MapReduce be used as a generic parallel processing mechanism? Specifically, is it suited to deal with issues such as data affinity, asynchronous batch processing, etc.?

In this post I'll try to answer these questions, but first, a few clarifications:

What is MapReduce?

Quoting from the Wikipedia definition

MapReduce is a software framework introduced by Google to support parallel computations over large (multiple petabyte^[1]) data sets on clusters of computers.

Why do we need a new model for processing large data sets?

Unlike central data-sources, such as a database, you can't assume that all the data resides in one central place, and therefore, you can't just execute a query and expect to get the result as a synchronous operation. Instead, you need to execute the query on each data-source, gather the results and perform a 2nd-level aggregation of all the results. To speed the time it takes to run this entire process, the query needs to be done in parallel on each data source. The process of mapping a request from the originator to the data source is called "Map"; and the process of aggregating the results into a consolidated result is called "Reduce".

MapReduce implementations

Hadoop is the most well-known MapReduce implementation.

Hadoop is an open source project that implements the exact spec defined by Google in Java. As such, it was designed primarily to enable MapReduce operations on distributed file systems and was not really designed as a general purpose parallel processing mechanism.

The wikipedia entry on MapReuce (http://en.wikipedia.org/wiki/MapReduce) has references to other implementations in other languages, including Greenplum, Skynet, and Disco.

Other forms of MapReduce implementations

Over time, the term MapReduce has expanded in definition to describe a more general purpose pattern for executing parallel aggregation of distributed data-sources, rather than referring to a specific type of implementation. GigaSpaces, GridGain, and to a degree, Terracotta, all took a different approach than Hadoop in their MapReduce implementations. Rather than implementing the exact Google spec in Java, these three aimed to take advantage of the Java language and make the implementation of the MapReduce pattern simpler to the average programmer (I'll get back to that later).

How MapReduce differs from other grid implementations?

Compute Grid

While MapReduce represents one form of parallel processing for aggregating data from distributed data sets, it is not the only one. "Compute Grid" is a term used to define another form of parallel processing, used mostly to compute intensive batch processing. A typical batch processing takes a long-running Job, breaks it into small tasks and enable the execution of those tasks in parallel to reduce the time it takes to execute the job (Compared with the time it would have taken to execute the tasks sequentially). This model is a good fit for executing relatively compute-intensive and stateless jobs. A typical scenario for this would be a Monte Carlo simulation, such as the one used to perform risk analysis reports in the financial industry. This type of analysis is more compute-intensive than data-intensive. Most compute-grid implementations have the following components:

1. Scheduler

2. Job executor

3. Compute agent

The executor submits jobs. The scheduler is responsible for taking the job, splitting it into a set of small tasks (this process requires specific application code) that are sent in parallel based on a certain policy to a set of compute nodes. The agents on each compute node execute those tasks. The results of those tasks are aggregated back to the scheduler.

The scheduler is responsible for monitoring and ensuring the execution of the tasks. The scheduler was designed to support advanced execution policies, such as priority-based execution as well as advanced life-cycle management.

Master/Worker Pattern (simple Compute Grid)

The Master/Worker pattern is a simplified version of parallel batch execution, based on the Tuple Space model. Tuple Spaces emerged from the Linda project at Yale university. JavaSpaces is the main Java implementation of the model. A good description of this model is provided in this article. In a master/worker pattern, tasks are assumed to be evenly distributed across worker machines. In this case there is no need for an intermediate scheduler. Load-balancing is achieved through a polling mechanism. Each worker polls the tasks and executes them when it's ready. If a worker is busy, it simply won't process the tasks, and if it is free it will poll the pending tasks and process them. Consequently, Workers running on a more powerful machine will process more tasks over time. In this way, load balancing is implicit, supporting simple task distribution models. For this reason, master/worker implementations tend to be more useful for simple compute-grid applications.The fact that there is no need for an explicit scheduler makes master/worker more performant and better suited for cases where latency is an important factor.

MapReduce & Compute Grid: Summary

Although both Map/Reduce and Compute Grids provide a parallel processing model for solving a large- scale problems, they are each optimized for addressing a different kind of problem. MapReduce was designed to address shortening the time it takes to process complex data-anlytics scenarios. The results of the processing need to be returned in real-time, as the originator of the task normally blocks until its completion. Compute Grid applications are aimed at speeding-up the time it takes to process complex computetional tasks. the Job is executed as a background process that can often run for a few hours. Users don't typically wait for the results of these tasks, but are either notified or poll for the results. With MapReduce, the application tends to be data-intesive, therefore scalability is driven mostly by the ability to scale the data through paritioning. Executing the tasks close to the data becomes critical in this scenario. Compute Grid applications tend to be stateless, and normally operate on relatively small data-sets (compared with those of MapReduce). Consequently, data affinity is considered an optimization rather than a necessity.

When to use MapReduce, Compute Grids and Master/Worker?

• If you need to agregate data that resides in a distributed file system then I would recommend the use of Hadoop and the like.

• If you need to agregate data that resides in other data sources, such as an in-memory data-grid (IMDG), you should consider GigaSpaces, or a combination of compute grid and data grid products.

• If your application is compute-intensive and relatively stateless in nature â€" you should consider the classic Compute Grid implementations.

• If you're looking for a real-time (or near-real-time) and lightweight compute-intensive application, you should consider Master/Worker implementations

In reality, most compute-intensive application are not purely stateless. To execute the tasks the compute tasks need to process data that is coming from either a database or a file system. In small scale applications, it is common practice to distribute the data with the job itself. In large scale compute-grid applications, however, passing the data with the job can be quite inefficient. In such cases, it is recommended to use a combination of Compute and Data Grid. In this case, the data is stored in a shared data-grid cluster and passed by reference to the compute task. So we see the need for a combination of Compute and Data Grids becoming more common.

Too many options? Feeling confused?

At this point you may be scratching your head wondering whether or not your application falls precisely in any of the above categories.

A quick reality check will reveal that many existing applications consist of a variety of the above scenarios, mixed with traditional client-server models.

In such cases, attempting to use a different product for each scenario in our application is going to make things extremely complex.

How do we make distributed programming like MapReduce simple?

This question has been the driving force for many of our recent development efforts.

To simplify things, we realized that we need to:

1. Grid enable existing programming models -- Use abstraction and virtualization techniques to introduce parallel processing as part of a normal client/server programming model.

2. Reduce the amount of frameworks -- Provide a common model for using both parallel computing models: batch (compute-intensive) and real-time aggregation (data-intensive).
3. Make data-awareness implicit with all APIs -- In reality, most application are stateful to some degree, so we need to make data-awareness implicit within our API and not as an afterthought. External integration solutions tend lead to complexity.

Where does GigaSpaces fit in?

GigaSpaces emerged from the tuple space model, specifically JavaSpaces, and was one of the first implementations of the Master/Worker pattern. At a later stage, we extended our JavaSpaces implementation to a full IMDG (In-Memory Data-Grid). In large scale compute grid applications, the GigaSpaces Data-Grid is often used in conjunction with other Compute-Grid implementations, either commercial or open source. This puts GigaSpaeces in a unique place, providing data-grid and data-aware compute grid capabilities using the same architecture. We also provide built-in integration of our Data-Grid with more advanced Enterprise Compute Grid products, such as those from DataSynapse and Platform Computing.

As of version 6.0, we offered abstraction layers (referred to as the Service Virtualization Framework or SVF) that take advantage of our existing space-based implementation in a way that doesn't require a complete re-write or a steep learning curve for developers who have already written their business logic as SessionBeans, Spring Remoting, RMI, CORBA, SOAP and other common Client/Server programming models. Our aim was to make distributed programming simple to the average programmer. We achieved this goal by following the same principles that I laid out above. For example, we introduced a set of abstractions on top of our space-based implementation. As we support both data distribution and task distribution, we are able to reduce the number of required frameworks and runtime components, as well as avoid the need for external services to ensure data affinity. In addition, we extended our support for aggregated MapReduce queries using a new Executor framework. With this we can support MapReduce and batch processing using the Master/Worker pattern and the *same* consolidated programming model.

The idea behind all this is to make scale-out development simple by making the API as close as possible to prevailing programming models, and by reducing the number of products and components required to scale either data-intensive or compute-intensive applications.

Final notes

The emergence of MapReduce specifically, and Grid computing in general, creates a need for another type of programming model currently missing in most existing mainstream frameworks and products. So far the solution has been to provide different specialized frameworks to to address each need. The fact that we have so many different frameworks (MapReduce included) makes things more complex.

On the Cloud Computing mailing list, Chris K Wensel wrote the following comment:

'thinking' in MapReduce sucks. If you've ever read "How to write parallel programs" by Nicholas Carriero and David Gelernter (http://www.lindaspaces.com/book/), many of their thought experiments and examples are based on a house building analogy. That is, how would you build a house in X model or Y model. These examples work because the models they present are straightforward.......If companies like Greenplum are using MapReduce as an underlying compute model, they must offer up a higher level abstraction that users and developers can reason in.

Indeed Making MapReduce part of mainstream development requires a higher level abstraction. The high level abstraction needs to provide means to use existing programming models on top of MapReduce to shorten the learning curve and transition from existing applications to distributed scale-out applications. Having said that, this is not enough, as we're still going to end up with multiple frameworks for addressing various parallel programming models that are not covered with MapReduce, such as Compute Grids and batch processing. It is therefore critical to map those different models into a coherent and consistent model that would support all various programming semantics, including MapReduce, Master/Worker and batch processing, in addition to the classic Client/Server model, with the ability to smoothly transition among them, without the need to switch or integrate different frameworks for each, and without the need to write our business logic in a completely differently way for each.