Nati Shalom's Blog

Platform as a Service: The Next Generation Application Server?

Platform as a Service is a term that can be fairly confusing for many people. Normally the term is associated with Google App Engine from Google and Force.com from Salesforce.com as the main references for this model. From a technical point of view, it is aimed to provide a similar type of value to the one that is currently provided by many of the application servers i.e. it provides a generic container that can host different applications and shield them from the details of the underlying operating system, network, database implementation. Unlike most of the existing application servers it was designed for massive scaling from day one. Another big difference is in the way it is being consumed. With PaaS you don’t need to install any software and go through all the hoops to setup a cluster environment etc. PaaS is provided as a hosted service that is pre-configured and installed. You get a production ready environment right at the start.

Key Characteristics of a Cloud/SaaS Enabled Application Platform

Last week I came across David Mitchel Smith presentation from Gartner. David provided a good definition to the main PaaS characteristics. A snippet from his presentation covering this area is given below:

What’s interesting is the great emphasize on Multitenancy support. The fact that the platform is going to be shared between multiple applications and potentially even different customers require various levels of tenancy support and isolation to ensure that even though were taking advantage of the fact that we can share resources between applications, each application needs to be able to use the platform as if it is running on its own dedicated resources. Another interesting point is the need for XTP support. Many would view XTP as a niche that is normally referenced in the high end part of the market, so it is fair to ask why would XTP fit into a general PaaS solution?  XTP represents a model for supporting enterprise transaction processing applications in an extremely scalable environment. In our case, scalability is not necessarily driven from the demand of a particular application but from the fact that many applications are going to run on a shared environment.  A PaaS targeted to enterprise applications would need to provide support for this level of scalable transaction processing support as a core service.

Can you run your existing business applications with GAE or Force.com?

No. Unfortunately as in anything in life reality can really spoil the party.

Force.com was designed to make it simple to run business applications that are database centric i.e. CRM, Reporting etc. It provides a rich set of high level services that make building such applications extremely simple. Google provides a more generic application platform and recently announced support for Java which is a big step towards reducing vendor lock-in concerns.  Google seems to be geared for consumer based applications. Force.com offers a more high level platform that is based on its proprietary services. This means that in order to take advantage of their service you will need to go through a complete re-write.

Force.com is based on a database centric architecture. They also seem to be limited in scalability as they partition their database per application. This means that if your application needs to scale more than what a single database can provide, you can find yourself pretty much locked.

Google's recent support for Java makes their offering closer to standard JEE application servers, however their current support impose a lot of limitations due to their sandbox model. These limitations mean that at the end of the day GAE can be applicable only to a small set of relatively simple applications.  Since there is no guarantee or control over the resources that you are going to receive from their underlying infrastructure, it is likely that the application performance will be unpredictable and will therefore be affected by other applications that are sharing the same hardware.

The fact that the platform as a service shields you from the details of the underlying infrastructure is what makes it simple and that is both the advantage and limitation. You can’t control the environment, you can’t choose your operating system, you can’t install your own set of services and you can’t control the performance characteristics of this platform.

This puts a huge adoption barrier for most current enterprises.

IaaS vs PaaS

Infrastructure as a Service provider provides a hosted service model that offers plain machine level access. This model is also known as Server as a Service. The fact that you get access to the bare metal allows you to run almost any application on this hosted environment. Unlike PaaS, IaaS gives you extreme flexibility. You can choose your own operating system, install any package that you want, you can setup your firewall, security etc. Amazon is known to be the leader in this space and also provides  a set of services on top of their infrastructure such as SimpleDB, SQS and MapReduce. Having said all that, this flexibility comes at the cost of complexity. In many cases you will need to install your own software, configure it, tune it etc. before you could make it run effectively on the cloud. Many application developers don’t have the skill-set to do that. This exposes many operational challenges as many organizations are not geared to support this type of environment from their own IT. The high level services provided by Amazon are still proprietary and would require a complete re-write if you plan to use such services.

PaaS for Enterprise applications – doing it right

The ideal solution would be to combine the best of the two worlds i.e. the flexibility of IaaS and simplicity of PaaS, and here is how:

• Build a Generic PaaS on top of AWS – To build a PaaS we don’t need to re-invent the wheel. Unlike Google and Force.com we don’t need to own the infrastructure, we can actually use Amazon infrastructure or even better build our PaaS such that it can be portable between Amazon IaaS and a VMware IaaS. By doing so, the PaaS can provide us the ability to deploy applications in a simple way just as in GAE but would still enable us to control the environment, install our own software and get the full flexibility that IaaS provides.
• JEE as first class citizen – Many of the existing enterprise applications are built in JEE. Making JEE a first class citizen within our PaaS environment will enable you to leverage the existing skill-sets within those organizations. Similar to the standard model that has more than one implementation out there, reducing the lock-in factor significantly.
• Pre-configured for extreme scalability – All the services provided through the PaaS will need to be implemented and pre-configured for extreme scaling and come with a production ready setup to enable dynamic scaling and fault-tolerance.

Next generation application Server?

Yes, In my opinion, PaaS represents the next wave in middleware technology. One that is targeted for virtualized enterprises, one that was designed for scale-out from the get go, one that fits the new way of delivering SaaS applications and one that can be extremely simple to use. The current PaaS players i.e Google and Salesforce and to a lesser degree Microsoft represent one type of player, those that own it all, i.e. the infrastructure and the platform. There is already a new emerging category of players in the PaaS market, Application PaaS players. Application PaaS players would specialize on delivering only the platform and not the hardware and network infrastructure. They act as the bridge that will enable portability between different infrastructure providers including the internal IT (a.k.a private-cloud). The application PaaS players will also be segmented in similar ways to the way application servers are segmented today, i.e there would be the one targeting the low-end consumer market in similar ways to Google App Engine, the ones that will be aimed toward the high end of the market and those who will be specialized in certain languages or development framework, i.e. Ruby/ Java/ .Net etc. In the PaaS type of world, those who would be able to provide a holistic solution that works smoothly across all the application tiers would have an advantage over those who are providing point solutions.

What about my existing applications ?

Most people would categories PaaS as a platform that is delivered through the internet. That statement would make the idea behind PaaS irrelevant for a large part of the existing enterprise applications, as a majority of them are not ready to run their application in a hosted service over the internet. Let’s examine that statement:

Many of the existing IT run farms of application servers in their internal IT. Those application servers are running in an internal data center that is not that different from any other hosted services, only that it is a specialized hosted service tailored for the needs of the specific organization. I would therefore argue that those organizations that are already running such application server farms would find it easier to evolve such server farms to PaaS model than to change their entire IT infrastructure into internal cloud. The reason is that those applications were already written to run in an application container model, therefore a large part of the transition work can be done within the container implementation and outside the application code. Targeting them first would therefore be potentially easier transition than trying to transform all your other applications into a virtualized environment.

I know that there are many people out there that would argue that internal PaaS is not a PaaS because it doesn’t answer the exact definition of PaaS. However in my view the similarities exceed the differences and the value to the enterprise would be almost identical to the one that I would receive by any of the internet based PaaS platform.

Will I lose control?

PaaS will provide the internal IT much better control over the applications that are running on their environment. You can have high visibility as to ways the application consume resources. In the same way, you can control fairly tightly the application security, scalability, resource management and fault-tolerance which can finally be managed in a consistent way across all the applications.

Final words

Platform as a Service represent the next generation application server IMO. GAE and Force.com are most known references in the market for that model. Both tries to offer the complete stack and that’s both their advantage and limitation. There already is a new category of Application PaaS providers that specialize on providing PaaS on top of existing infrastructure providers. There would be generic PaaS providers (similar to GAE) geared for the low end part of the market and those that are geared for the high end of the market. There would also be more vertical PaaS providers (Similar to Force.com)  i.e. those that will provide PaaS for a certain segment of applications or segment of the market such as Online Gaming PaaS, Telco PaaS etc. A good example for such a service is Twilio. As I outlined in my recent post (Google App Engine plus Amazon AWS: Best of both worlds), this is not just a theory but a reality in the making. GigaSpaces' contribution to this new reality is our new cloud framework on Amazon EC2. Geva Perry provides an excellent overview of other good examples that follows that same line his post What's Really Exciting About Cloud Computing.

References:

• Google App Engine plus Amazon AWS: Best of both worlds
• Waiting to see if App Engine or an alternative gets there first
• What's Really Exciting About Cloud Computing.
• JavaSpaces and the Next Big Thing
• Space-Based Architecture for AppEngine
• GigaSpaces as Alternative to GoogleAppEngine for the Enterprise
• GigaSpaces introduces portable PaaS
• Google Brings App Engine's Pros and Cons to Java
• Google App Engine and The Java Web; The Wrong Java?

Google App Engine plus Amazon AWS: Best of both worlds

George Lawton wrote a a good summary of my JavaOne talk in his article titled Google App Engine plus Amazon AWS: Best of both worlds 

Google App Engine (GAE) is focused on making development easy, but limits your options. Amazon Web Services is focused on making development flexible, but complicates the development process. Real enterprise applications require both of these paradigms to achieve success… What we really want is the flexibility and performance of AWS and the simplicity and ease of use of GAE.

This is exactly what we had been working on for the past year, leading us to the launch of our new cloud platform. With this platform we leverage GigaSpaces XAP as the high performance scale-out application server and Amazon as the robust and flexible IaaS. Together they form an alternative Platform as a Service geared for enterprise grade applications. This allows the cloud environment to inherit the extreme performance, latency and scalability of the XAP platform, which in turn enables achieving your performance and scaling target with less machines, implying a lower cost.

Real-life case study: Primatics financial – Risk analysis as a service

Francis de la Cruz and Argyn Kuketayev from Primatics Financial joined me through the presentation. In their part of the session they described their experience in developing a SaaS application for Real Time analytics.

Kuketayev described how Primatics used this approach to create a new automatically scaling cloud version of an existing banking application. Primatics initially developed a mortgage securities application that allows banks to estimate the value of a basket of hundreds of thousands of loans. The value of these loans fluctuates as economic conditions change and some portion of home owners cannot afford to make payments on their loans. Banks normally only need to assess the value of these loans at the end of each month, making them an ideal candidate for cloud services like AWS.

From a scalability perspective the challenge is to be able to provide a highly multi-tenant application that need to serve many firms, many users in that same firm each running many jobs at the same time. Implementing such a model can be fairly complex as you will need to be able to manage the life cycle of each job and each user independently and in isolation from one another.

 

Trying to build such a service directly on Amazon is going to be fairly complex, as you can learn from George’s summary below:

Primatics wrote the first version of EVOLV:Risk as a hosted web application for a regional bank.. The application needed to be fault tolerant so that if one node crashed, they did not have to restart the application over again from the beginning. Kuketayev said that it is not just about the loss of four hours, but the office is trying to close out the month and needs to access data to end the monthly cycle so they can go home.

Using GigaSpaces' toolset they rewrote the entire application infrastructure in about four-months to run on top of AWS. Now they can kick off as many instances as required for different banking customers, and each instance runs significantly faster than before. Kuketayev said that it is important for banks that none of their applications run on the same infrastructure as another bank.

The diagram below shows the specific architecture that Primatics ended up using. Those that are familiar with Space Based Architecture would find it fairly straight forward:

The application is built out of  a set of processing units. Each processing unit contains the compute agents in the form of a polling-container.  The compute agents gets a a reference to a remote Data Grid that is shared by all processing units. Each agent gets the job injected to it by the polling container and gets a reference to the data it required to process the job. Once the job is completed, the result is stored back in the space. The results are flashed out back to a database through a mirror service.

In a case of a failure, other compute agents are able to continue from the exact point of failure and continue the job processing as if nothing happened. This is because the state of the job is kept safe in the data-grid and not in the agent’s memory.

Kuketayev from Primatics nicely summarized thye lesson he learned after going through the experience of trying to build it on his own vs. trying to use GigaSpaces:

Kuketayev said that one of the biggest lessons is that you need to have your infrastructure do the provisioning for you automatically, or otherwise you end up spending a lot of time just turning things on and off. He said they are now using configuration APIs to automate this process, whereas before they were using scripts. This allow for automatically throttling and failover recovery without human intervention.

Kuketayev advised "You need to make sure you use the right tools … You don't want to have to worry about provisioning and reliability. Make sure you have provisioning, failover, monitoring and SLA out of the box."

The full JavaOne presentation is available here:

Alternative to Google App Engine on Amazon

View more OpenOffice presentations from Nati Shalom.

Final words

Fr solution providers the size of Primatics, building a risk analysis application as a service couldn’t be possible without cloud computing. Cloud enabled them to offer their solution as a service without the need to go through major investment of building a data center to support it.

Primatics’ experience is not special. One of the benefits of building Software as a Service is that you have one shared environment for all your customers. At the same time, one of the challenges is that in a shared environment, failure becomes more public and will impact ALL your clients. If the system doesn’t scale well, you’re going to be hit twice as hard as in a standalone application.

Building a robust and scalable SaaS application can be fairly complex. A good cloud infrastructure will get you a first class data center, but it won’t solve your application requirements.What’s interesting with cloud computing is that it forces you to think about the cost and efficiency of your application more than ever before. In the Primatics example, running a simulation of 100 nodes for 3 hours is very likely to fail at some point. A failure during such a simulation will immediately cost you 300 hours, not to mention the fact that you might lose the simulation window for the day and the reputation challenge you’ll will be facing with your customers. In addition, putting the data in-memory and making the application run 3-5 times faster means that you would need 1/5 of the machine power, which saves 80% of the cost of running the application.

I believe that the challenges imposed by cloud computing force us to focus on what we do best and avoid investing in areas which are not core to our business. Because the pay-per-use model significantly lowers the cost barrier, going down the path of writing your own infrastructure, as many have tried to do before, will be much more expensive and risky then ever before.

References:

• Google App Engine plus Amazon AWS: Best of both worlds 
• The Need for Speed - Saving Costs By Improving Application Efficiency
• eBay Subsidiary Uses XAP to Boost Performance - Expects to Meet Same Loads with 95% Less Servers
• The impact of cloud computing on build vs. buy behavior

GigaSpaces Launches a New Version of its Cloud Computing Framework

We have been working for a while on our new Cloud Computing Framework (CCF). We wanted to wait until we had some real customers and partners behind the platform in production before spreading the word on a larger scale. And now, I am very happy to report that we have reached this point sooner than expected and just in time for the JavaOne event.

The new version includes an enhanced integration with the Amazon billing system Devpay designed to enable users to use GigaSpaces as a service on a true pay per use model. All you need is an Amazon account and your done. Unlike other products, that are offered in similar fashion, we decided not to provide just a bundle of our software packed in an Amazon image (AMI) but instead provide a more tight integration with EC2 that will enable our users to use a single GigaSpaces product for all GigaSpaces versions as well as customize the installation to include their own software packages and application code on the fly. The new CCF provides a true end to end experience that will enable our users to deploy and manage their *entire* application from the load-balancer to the database in one single click.  The latest release of CCF includes support for the new GigaSpaces 7.0 release. Users that are looking to try out the new XAP product can now do that for free without the need to download or install anything.  This will enable you to try out any of the built-in examples, test the high availability features and run benchmarks in just one click. The new version includes lots of additional improvements to the user experience as well as new features and bug fixes. To view a detailed list of the new features click here.

Those that will be visiting JavaOne this week will be able to join the hands-on labs session and experience a full deployment of JEE application using CCF. You will also be able to gain insight from one of our customers, Primatics Financial on how they used CCF to build their Risk Management application as a service – See  full details here

In this post I will try to address some of the most common questions that we often receive when introducing this new service:

Who is using GigaSpaces CCF and Why?

The new release is already used by different customers in the following areas:

ISV’s and Service providers using CCF for SaaS enablement:

Many ISV's are looking for ways to move their standalone applications into a SaaS based offering. In doing so they face a few challenges:

1. Scalability – now their application needs to serve not just the users of a certain customer at a certain location but all customers from all locations. This increases the scalability challenge on their product by an order of magnitude.

2. Continuous high availability – one of the challenges that is imposed by having a shared platform for all customers is that the impact of failure can be catastrophic as failure can affect all of your clients at the same time and becomes public fairly quickly.

3. Portability – There are cases where some customers would still require their application be installed in their local data center for security and latency purposes. Today, this often means you will need to develop two seperate branches of your product to serve these two scenarios. With GigaSpaces you can write your application once and deploy it in the cloud or in your customer data center without having to modify your application and without locking your application to a specific cloud provider.

4. Performance – In developing high performance applications such as Trading, Risk Analysis, or SIP server, keeping the latency at sub msec and throughput at a maximum becomes critical. This is where our XAP middleware has a proven record in running numerious production mission critical applications.

The following are a few public references that chose to use XAP:

- Primatics Financial is using GigaSpaces CCF to build a high performance risk management solution

- Orbyte is using GigaSpaces CCF for building state of the art high performance trading applications that can be offered as service.

Other customers include Online Gaming companies that were looking for a simple and cost effective manner to launch new online games and provide a gaming application platform that enables other gaming providers to host their applications easily in a cloud environment. See further details on our online gaming solution here.

Enterprise customers using CCF to improve time to market of new applications and for large scale testing

According to recent research conducted by Forrester, one of the main requirements driving cloud adoption by the enterprise market is that current IT can’t keep up with the business demands of their organization:

• Capacity planning is too difficult – The current method of provisioning applications based on estimated peak loads has become either too costly or impossible. Most systems are non linearly scalable which means that knowing how many resources would be needed to meet certain goal is known only through trial and error.
• Time to market – launching new applications and services often takes months and sometimes year.
• The business wants a simple and cost effective way to prototype – not all applications make it to production but these initiatives still require the business to invest upfront in the purchase and configuraiton of infrastructure to support these prototypes.
• Cost saving – There are sets of applications that have fluctuating demand on resources.  Enabling these applications to “lease” a common set of resources only when under heavy demand can lead to huge cost saving and efficiency.

I discussed the enterprises challenges in my previous talks. The most recent one is the one I gave during the CloudSlam event:  Practical Guide to Developing Enterprise Applications on the Cloud- Online Presentation 

A good example of an enterprise customer who is already using our CCF in production is a Large Telco Service Provider that searched for ways to reduce the time to market for launching new scalable web applications. This customer was able to bring a project that previously took more than 6 months just to finish the development and testing to 4 weeks with only 90$ spent on pre-investment.

Large scale testing, performance benchmarks and demo as a service

Users that are not ready to use the cloud to run their production systems can still receive benefits from the cloud for testing and demonstration purposes. There are a few challenges that we tried to address with CCF to make this type of use simpler:

1. Making cloud testing environments identical to your local testing environment – To effectively test your application on the cloud and deploy a production version of your system locally, your application must be at a point where it behaves fairly the same in both environments.

2. Setting up a large scale environment of hundreds of nodes can be fairly complex

3. Trouble shooting failure in such an environment can be a fairly complex and lengthy process.

If your application is already running on our XAP product then porting it to the cloud and running it locally in your production environment will not require any change to your code or configuration. With CCF setting up a large scale environment of hundreds of nodes with the new CCF takes just a few minutes. We designed the CCF to run not only GigaSpaces but other external services which are essential for tests such as JMmeter, Tomcat servers etc. Users can easily configure their own initialization and command scripts when the machine is launched and define which software packages will be installed on their machines on the fly. We added the ability to capture the logs of multiple machines easily and monitor the system and middleware behavior at runtime using a zero installation environment all through the web.

GigaSpaces and our existing customers and partners have already been using CCF to run live demos and recently GigaSpaces started using CCF for our own internal large scale testing purposes. Using the cloud for demo purposes has been extremely useful to show cases that previously were hard to demonstrate such as fail-over and scalability in a real production environment. You can read more on how you can use CCF for running your own demo on the cloud in a similar way without going through all the challenges we had gone through in one of my recent posts: Demo as a Service . Similarly CCF can be ideal for running large scale Benchmarks.

Partner ecosystem -

We designed our CCF in away that would enable us to run not just GigaSpaces components but almost any application such as JEE or Spring based applications. This enables our Solution provider partners to use CCF to offer their specialized solutions and offerings for both GigaSpaces and their other software partners.

System Integrators are also looking for ways to gain a competitive advantage by offering turn key solutions at a significantly shorter time to market and lower initial investment. I tried to outline below some of the public references on these two categories:

• Solution providers

Real time production monitoring:  Dynatrace offers enhanced integration with CCF for enhanced production monitoring.  Dynatrace users are able to monitor in real time how their application is performing and trace their application bottlenecks in real time – see details on this integration here.

Model Driven Development framework - New Technology/enterprise (NT/e) - GigaSystemBuilder helps Java developers prove applications quickly - in a few days - on a local grid or in the cloud, and then rapidly move on to production development, rather than waste time on finger-trouble or implementation details – see more information here as well as a live demo on their deployment manager solution

• System integrator

In many cases customers are looking for experts that will enable them to gain the benefit of the cloud and bridge some of the skill-set gaps. AAR is one of our leading system integrators in that field who has already delivered several solutions on top of our GigaSpaces CCF for Enterprise customers. For them, using CCF was a tool to gain a competitive edge and provided their customers the ability to launch new applications in a matter of weeks as opposed to months and at a fraction of the setup (development environment, testing environment, staging environment) and hardware costs that are associated with setting up such an environment with traditional projects.

Here is a quote from one of our System Integrators in the UK that describes their experience in using CCF for the Telco Service provider that I mentioned earlier.

“We use GigaSpaces XAP and CCF to deploy a standard JEE web application on EC2. To address security concerns we kept the business logic outside of the cloud on the customer site. We exposed the business logic through a secured web services channel. We use XAP data grid for maintaining the session high availability and for reducing the latency associated with accessing the remote data center. We used the XAP integrated Jetty container for hosting our web application and through that gained the built-in self healing and auto-scaling provided through the CCF. In this way we were able to deal with potential failure or load without human intervention. From a management and monitoring perspective we were able to leverage the integrated ganglia monitoring and integrated visual representation of different metrics, like cpu/memory/network usage of the entire cluster. We used JMX to bind our custom Mbeans to the integrated ganglia UI. Overall GigaSpaces CCF gave us big boost on utilizing ec2”

I must admit that from our end this was probably the smoothest project we had been engaged with. The reason was probably related to the fact that unlike other projects we had full control over the environment. This enabled us to minimize potential miss configuration errors which is one of the major areas for complexity in setting up such a project. Another benefit was the support. When we faced an issue it was very easy to log-in directly to the machine and fix the problem on the spot without the need to go through a complex process of authorization from the IT Security and without the need to create a reproduction package and ship it over.

The customer was able to deploy a new business application quickly without putting all the initial investment upfront. In this way they gained the freedom to decide at any point if the application should continue, be canned or if they should move it entirely to their local IT.

Looking into the future

The future looks very promising as we continue to work on bringing new customers and businesses towards using CCF and the latest XAP 7.0. We are currently working on supporting the Amazon European datacenter and taking advantage of some of the new XAP 7.0 cluster administration API to enable users to provide thier own custom SLA deployment. At this point, I would like to ask for your specific feedback and wish list for our next major CCF release.

Interesting talks, and free drinks in JavaOne

Its been two years since I've last visited the JavaOne conference. This year is going to be particularly interesting as its going to be the first major Java event following the Oracle acquisition of Sun.

I will have a Technical Session on Tuesday titled: Alternative to Google Application Engine for Java™ Technology-Based Applications, where I'm going to outline the difference between the Google and Amazon approach for cloud computing and discuss how we can combine the best of the two approaches. Argyn Kuketayev and Francis de la Cruz from Primatics is going to join me  and present their experience in deploying a risk management application as a service and provide some of the technical details on how they where able to scale-out their application  on the cloud.

Daniel Templeton from Sun Microsystems will have a lab session: PetClinic in the Clouds: Scaling a Classic Enterprise Application In this Hands-on Lab, participants will take a popular Web application (the Spring PetClinic sample application) and modify it so that it can be deployed on the Amazon EC2 cloud computing infrastructure. They will be exposed to using the GigaSpaces platform as a service, in-memory data grid concepts, the OpenSpaces framework, cloud computing concepts, and persistence as a service using Sun's MySQL™ database technology.

We are also co-hosting an event Tuesday June 2 at 8PM with our partner Webtide with whom we've done a great integration for Jetty.

Among those who will attend the party will give a chance to win a free book Savvy Guide for cloud computing by Jim Liddle.

*Note: Space is going to be limited so if you want to ensure your place make sure to register on the online registration site that we set for this event.

The list below include all the sessions and labs that I hope to see – any recommendations on other interesting talks would be appreciated.

Session ID	Session Title	Session Type	Speakers and Company	Date/Time	Venue Room
TS-4605	Enterprise JavaBeans™ 3.1 (EJB™ 3.1) Technology Overview	Technical Session	Kenneth Saks, Sun Microsystems, Inc.; Marina Vatkina, Sun Microsystems, Inc.	Tuesday June 02 10:50 AM - 11:50 AM	Hall E 134
TS-4308	Architecting Robust Applications for Amazon EC2 Chris Richardson,	Technical Session	Chris Richardson Consulting	Tuesday June 02 12:10 PM - 1:10 PM	Esplanade 307-310
TS-4390	Castle in the Clouds: SaaS Enabling JavaServer™ Faces Applications	Technical Session	Lucas Jellema, AMIS	Tuesday June 02 12:10 PM - 1:10 PM	Esplanade 302
TS-3817	Google App Engine: Java™Technology in the Cloud	Technical Session	Toby Reyelts, Google; Max Ross, Google; Don Schwarz, Google	Tuesday June 02 3:20 PM - 4:20 PM	Hall E 135
TS-5454	Alternative to Google Application Engine for Java™ Technology-Based Applications	Technical Session	Nati Shalom, GigaSpaces Argyn Kuketayev Francis de la Cruz Primatics	Tuesday June 02 4:40 PM - 5:40 PM	Esplanade 302
LAB-5564BYOL	PetClinic in the Clouds: Scaling a Classic Enterprise Application	Hands On Lab	Michal Bachorik, Sun Microsystems, Inc.; Shay Hassidim, GigaSpaces; Daniel Templeton, Sun Microsystems, Inc. Wednesday June 03	Wednesday 1:35 PM - 3:15 PM	Hall E 132
TS-5214	Java™ Persistence API 2.0: What's New ?	Technical Session	Linda DeMichiel, Sun Microsystems, Inc.; Anil Gaur, Sun Microsystems, Inc.	Wednesday June 03 2:50 PM - 3:50 PM	Hall E 134
BOF-1304	Meet The App Engine (Java™) Team	BOF	Kevin Gibbs, Google; Toby Reyelts, Google; Max Ross, Google; Don Schwarz, Google	Wednesday June 03 7:45 PM - 8:35 PM	Hall E 135
PAN-5366	Cloud Computing: Show Me the Money	Panel Session	Jeff Barr, Amazon.com; Jeff Collins, Intuit; Chris Fry, Salesforce; Simon Guest, Microsoft; Gregor Hohpe, Google, Inc.; Raghavan Srinivas, Self; Lew Tucker, Sun Microsystems, Inc.	Thursday June 04 9:30 AM - 10:30 AM	Gateway 102-103
BOF-5392	Grails Integration Strategies	BOF	Dave Klein, Contegix	Thursday June 04 6:30 PM - 7:20 PM	Esplanade 307-310
TS-5307	Building Next-Generation Web Applications with the Spring 3.0 Web Stack	Technical Session	Keith Donald, SpringSource; Jeremy Grelle, SpringSource	Friday June 05 12:10 PM - 1:10 PM	Esplanade 307-310
LAB-5960	Storing Data in the Cloud	Hands On Lab	Craig Hubbard, Sun Microsystems, Inc.; Chris Kutler, Sun Microsystems, Inc.; Craig McClanahan, Sun Microsystems, Inc.	Thursday June 04 9:30 AM - 11:10 AM	Hall E 130-131

Tip* - If you want to find your own sessions I would highly recommend using the JavaOne search tool.

Seeyu next week!

GigaSpaces based solution makes it to the finalist of Cisco Developer Contest

I was very pleased to read an email from Leonardo, who was the winner of the OpenSpaces Developer Challenge (a worldwide programming contest using the Gigaspaces application server which was held last year), saying that he is now a finalist in the Cisco developer contest. Here's a bit about him and the application he submitted:

About Leonardo

Leonardo worked for several ISPs in various roles as network administrator and java programmer for IT consulting firms, and finally as software architect in high-performance Java EE based projects. He is passionate about parallel programming, distributed computing and more recently semantic web and its applications on software engineering.

Leonardo was the winner of the OpenSpaces Developer Challenge. He enjoys reading about various technologies in the field of computer science. When he is not developing code, he prefers to spend time with family and friends, walk in the park, or watch a movie.

About the application

Resource Management Platform is a proposal to develop an event based platform that leverages AXP, Services Gateway Initiative (OSGI), Jini and JavaSpaces technologies to enable deployment of IP Multimedia Subsystem (IMS) applications based on Session Initiation Protocol (SIP); more specifically, the Call Section Control Function (CSCF) components. It will have admission control mechanisms to manage Call processing.

This solution improves infrastructure manageability for large scale IMS applications. Such a platform will potentially be useful to enable deployment of high-performance, network-based SaaS (Software as a Service) or Cloud Computing solutions at the network edge by leveraging AXP.

You can find the full details about his project here.

Leonardo's project is interesting, because it shows how you can use Space Based Architecture (SBA) for implementing a scalable Telco application and offer it as SaaS application on the cloud.

Interestingly enough, I got another email the week before from Amin Abbaspour, who presented another case study illustrating how you can build a scalable SMS service using SBA, as shown in this diagram:

 

What the two projects have in common, from an architecture perspective, is that they both represent a highly scalable Event Driven design. The unique thing about Event Driven applications is that they require a combination of messaging, data and service interaction that needs to be tightly orchestrated to meet high performance/low-latency requirements without compromising on consistency, ordering (FIFO) and reliability. This combination of requirements represent one of the hardest challenges in building scalable architectures. Trying to meet this type of challenge in the traditional way by integrating messaging system for event delivery , database or simple caching (like Memcached or TC) for data and a traditional application server for business logic is going to lead to fairly complex architecture. Trying to reach linear scalability and keeping the latency low with so many moving parts is close to impossible. This is what makes SBA such a good fit. The main difference about SBA is that it recognizes there is strong dependency between messaging, data and business logic. The key is to have one shared clustering, high availability and scalability for all three components of the architecture. This makes it possible to reduce the number of moving parts and network hops associated with each business transaction, thereby increasing reliability.

On a personal level, I was very pleased to see that the software we are developing is helping people like Leonardo and Amin to build their own carrier and put themselves in a unique spot in highly competitive market.

Good luck Leonardo and Amin!

References

• Making EDA programming simple with JeeWiz
• Cisco AXP Developer contest
• What's new in GigaSpaces XAP 7.0
• GigaSpaces on the cloud
• OpenSpaces developer portal

Practical Guide to Developing Enterprise Applications on the Cloud- Online Presentation

According to a recent survey, available skill sets is one of the leading decision factors for organizations in determining which application platform to use, while scalability and availability are next. This reveals one of the main obstacles for bringing enterprise applications to the cloud: How do you take something as disruptive as cloud computing, and bring it to an enterprise environment, without forcing a complete re-write?

In my recent talk at the CloudSlam (online) Conference, I tried to summarize the challenges people face when trying to deploy enterprise applications on the cloud. Try to imagine a scenario where you have an existing JEE system, and your system is under load. At this point you would like to add more machines to accommodate that load. What will happen to your application in this case? Will your application be able to take advantage of the additional capacity? How do you know how many machines should be added to meet the load in the first place?

The answers I usually hear to these questions are fairly consistent. If you’re lucky, you’ll only require configuration changes to be at a point where you can utilize the extra resources. Knowing how many machines to add in order to meet a certain load is yet another challenge. Going through the normal capacity planning process could take weeks with the way systems are currently running.

At the same time, if you’re going to deploy our application in a static manner (reserved instances, static IP, …), it would probably make it simpler to deploy your existing application on the cloud, but it probably won’t make much economical sense. So the question I was trying to answer is how we get from the static application deployment most of us are using today, to a point where we can get end-to-end scaling and elasticity of our application from the load-balancer to the database without going through a complete re-write.

In this presentation, I tried to suggest a set of solutions to overcome the various challenges I mentioned earlier and how to apply them in a gradual manner to avoid huge initial investments and high risk. There were also some interesting questions toward the end, centered specifically around one of the hardest problems – getting the data pieces sorted out in a such a dynamic environment. Those who missed this presentation can now view it online:

Note: The voice quality isn’t that great, plus the desktop sharing on Webex didn’t give a clear indication when my desktop was actually shared, so the beginning of the talk is missing the first two or three slides. If you have questions or want to get a copy of the presentation, just shoot me an email.

During the upcoming JavaOne conference there will a hands-on lab session by Daniel Templeton from Sun Microsystems (PetClinic in the Clouds: Scaling a Classic Enterprise Application) where users can go through the steps and deploy a Pet Clinic application on the cloud. 

In addition to that lab, I'm going to have a Technical Session (Alternative to Google Application Engine for Java™ Technology-Based Applications).

For those that are not going to join the JavaOne event and want to get some hands-on experience on how the steps that i outlined in the presentation can be implemented in real application, I’d recommend looking Pet Clinic demo that is available on Amazon cloud using our cloud framework.

I’m happy to say that the number of customers that are already using this approach through our Enterprise Cloud middleware platform is growing quickly and were seeing more and more application that would traditionally considered as not "cloud compatible", being deployed on the cloud. Below are few of the public references that were mentioned just recently on Jim Liddle's blog:

• Telco application running on Amazon EC2 and GigaSpaces
• Using GigaSpaces to Trade in the EC2 Cloud

Designing a Scalable Twitter

Guy Nirpaz, Uri Cohen and Shay Banon came up with an interesting exercise as part of the recent partner training that took place at the GigaSpaces office. In this exercise, the students were asked to come up with a scalable design for Twitter, using Space-Based Architecture.

There are some interesting scalability lessons from this exercise, which are applicable to anyone looking to implement new-style real-time web applications such as the ones used for social networking.

In this post I'll  try to summarize the main patterns to put into place and considerations to make when designing such a scalable architecture.

Background:

For those of you who are not yet familiar with the service, Twitter is sort of a SMS-service meets discussion board.  You can post short messages (up to 140 characters) that can be shared with a group of subscribers that are referred to as "followers". The main difference between twitter and other messaging applications is that both SMS and Instant Messaging (IM) applications were designed primarily for one-on-one communications whereis Twitter was designed primarily for broadcast communications (publish/subscribe, or pub/sub). Another aspect that is special about Twitter is that by default anyone can follow anyone else. In other words, it was designed for open communications, not private, as were IM and SMS.

What are Twitter's scalability challenges?

1. Sending a tweet (a message on Twitter is known as a 'tweet') -– The challenge is how to handle an ever-growing volume of tweets and re-tweets and responses that can lead to a viral "message storm"

2. Reading tweets – The challenge is how to handle a large number of concurrent users that continually “listen” for tweets from users (or topics) they follow.

Designing A Scalable Twitter

Choosing the right scalability patterns

Almost every challenge in software architecture has its roots in one of the existing patterns. So the simplest course is to start by looking for those patterns, and choosing the right patterns to scale the application. Looking at many other scalable architectures, we'll begin with a partitioning pattern as the core design principle. By partitioning our Twitter-like application we'll spread the load across a cluster of servers and scale by simply adding more servers (i.e., partitions).  Another important architectural observation about Twitter is that it doesn’t fit into the classic database-centric design that most web applications do. On the flip side, it doesn’t fit well with a messaging-centric design (pub/sub) either. It is a combination of the two.

A pattern that is suitable for this type of collaborative messaging is known as a blackboard pattern.  In our design, we will use those two design patterns -- partitioning and blackboard -- as the foundation for our scalable Twitter application. With the foundation in place, let’s list the requirements and examine how these patterns can be used to scale the app.

Scalability Requirements

We'll assume a relatively extreme scaling requirement:

• Tweet Volume: 10 billion tweets per day
• Tweet Storage: 100 Gigabytes per day (with 10:1 compression)

Additional assumptions:

• Tweets are limited to 140 characters
• Tweets are immutable, i.e., there are no updates, only inserts
• Twitter limits client applications to 70 requests per hour

Now that we have the foundational patterns and clear requirements, we can design the architecture. We'll start first with the blackboard system.

Using an In-Memory Data Grid (IMDG) as a Blackboard System

The are several approaches to building a blackboard system. To maximize performance and scalability, we'll store the data in memory, thus avoiding disk I/O, which is often the main cause for contention. For years, Java has provided a model for designing blackboard systems known as JavaSpaces. More recently, distributed caching has become popular and can provide similar capabilities to those of JavaSpaces. Let's examine two popular distributed caching approaches for our blackboard system:

1. Simple read-mostly caching using memcached
2. Read/write caching, also known as an In-Memory Data Grid (IMDG)

Choosing between memcached and an IMDG

Memcached enables us to to store the data (tweets) in a distributed memory set and read it in a scalable fashion. Having said that, be aware that memcached is not transactionally-safe and is not designed for reliability (i.e., it doesn’t support fail-over and high availability). That means that if we use memcached or something similar, we will have to use a database as the back-end. Every tweet posted will have to be written to both memcached and the database in a synchronous fashion to ensure that no tweet will be lost. This approach may be good enough for scaling read access, however, for writes and updates it offers limited scalability.

Unlike memcached, which was designed for simple read-mostly caching, In-Memory Data Grids  are designed for handling a read/write scenario, and can therefore act as the system-of-record for both write and read operations. We can still use a database for long-term persistence, but because the IMDG maintains its reliability purely in memory, we can write and update the database asynchronously and avoid hitting the database bottleneck.

Todd Hoff, author of highscalability.com wrote an interesting summary that covers the different products in this space in a recent post:  Are Cloud Based Memory Architectures the Next Big Thing?

Todd provide a clear explanation of how an IMDG works (using GigaSpaces):

 

• A POJO (Plain Old Java Object) is written through a proxy using a hash-based data routing mechanism to be stored in a partition on a Processing Unit. Attributes of the object are used as a key. This is straightforward hash based partitioning like you would use with memcached.
• You are operating through GigaSpace's framework/container so they can automatically handle things like messaging, sending change events, replication, failover, master-worker pattern, map-reduce, transactions, parallel processing, parallel query processing, and write-behind to databases.
• Scaling is accomplished by dividing your objects into more partitions and assigning the partitions to Processing Unit instances which run on nodes-- a scale-out strategy. Objects are kept in RAM and the objects contain both state and behavior. A Service Grid component supports the dynamic creation and termination of Processing Units.

Back to our Twitter app: Given the scalability requirements, we will need to scale both reads and writes, and therefore, an IMDG is a more suitable approach to implementing the blackboard system.

Now let’s examine how the use of an IMDG as the blackboard system enables us to scale both sending and reading tweets. Let's start by designing the partitioned cluster.

Designing a partition architecture

One of the main considerations in designing a partition cluster of any kind is determining the partition key, such as a Customer ID in a CRM application or a Trade ID in a trading application. At first glance, it sounds like a trivial decision, but choosing the right partitioning key requires a deep understanding of the application usage patterns and data model.  In the case of Twitter, we could choose to partition the application by the data-type, the user, the tweet itself or the followers. Our first goal is selecting a key that will that will be granular enough to enable scaling the application just by adding more partitions, while making sure that we don't end up with a key that is too fine-grained -- making it sub-optimal for querying purposes.

If we use the timestamp key, for example, our application will be optimized for “inserts” (writes), however, even a simple query such “retrieve the tweets of a certain user” will force us to execute an aggregated query against all partitions. Alternatively, if we partition the data based on user-id, we'll be able to easily spread the load from different users across partitions. Retrieving the tweets of a certain user is going to be resolved in one call to a single partition. We may encounter a problem if a single user generates a significant higher load than average, however, in the case of Twitter, we can assume that this is not very likely. Partitioning by user-id is a good compromise.

Data capacity analysis

With such extreme requirements it is clear that storing all tweets in memory is going to require huge memory capacity. Very quickly this will become economically prohibitive, so we need to devise a scheme in which the IMDG acts as a buffer for most of the load on the system, and then offloads the data and queries to an underlying persistent storage.  In our Twitter example, it is fair to assume that most real-time queries (those that require fast access to the data) will be resolved in data from the last hour or 24 hours. Queries that require older data will need to hit the database for the initial call. However, subsequent access to fetch new updates should be resolved purely in-memory.

Using this approach, we'll need about 10 servers, each holding 10GB of data in memory to accommodate 24 hours of activity. If we also want to back up the data in memory, we will need double the amount of servers.

Choosing the right eviction policy

It's reasonable to assume that recent data is accessed most and older data is rarely used. To ensure that we get the maximum hit ratio on our memory front-end, let's choose a time-based eviction policy, which always holds the most recent updates in memory. When we will reach our memory capacity limit the oldest data will automatically get evicted from memory. The actual window of time in which we will be able to keep in memory is obviously dependent on the size of the cluster. With an IMDG implementation all tweets are stored in a persistent storage, which means that when tweets are evicted they are not deleted from the system.

Scaling tweet writes:

If we select user-id as the partitioning key, each user tweet will be sent to a specific partition. Multiple users may be routed to the same partition. Usually the algorithm to determine which partition fits a certain user is something like:

routing-key.hashCode() % #of partitions

In GigaSpaces, this is done by marking the routing attribute of our tweet class with an @SpaceRouting annotation.

The web front-end application will call space.write( new Tweet(..),..)  to send the tweets. This way there is nothing in our web client code that exposes the fact that the underlying implementation interacts with a cluster of partitions (spaces in GigaSpaces). Those details are abstracted within the space proxy. When the write method is called on the space proxy it parses the field that matches @SpaceRouting from our Tweet() object and uses this field value to calculate the partition it belongs to. It then uses that value to route the Tweet(..) object to the appropriate partition.

With this approach, the web application can be written in a very simple way and can interact with the entire cluster as if it was a single server.

The data from the memory partitions gets stored asynchronously into a persistent storage. The persistent storage could be a database, but it could also be other things, such as an index search engine based on Compass/Lucene.

Scaling tweet reads:

To those familiar with messaging system, at first glance Twitter looks like a classic publish subscribe application. A closer look, however, reveals that any attempt to implement Twitter with something like a JMS message queue is going to fail in achieving a scalable system. This is especially true if you consider that the system needs to maintain a durable queue for each user. That could easily lead to a scenario in which each tweet is published to thousands of subscribers and every re-tweet can potentially lead to a "message storm".

As I discuss above, the right way to think about this type of application is as a blackboard pattern, just as a blackboard (or these days, a whiteboard) is used by a group of people (followers, in the case of Twitter) to share information and collaborate. When someone writes something on the board, everyone sees it and can choose to react. Unlike messaging (take email for example), we don’t need to send separate messages to each subscriber. Instead everyone is looking at the same board. Everything is also copied from the board to paper. When the board runs out of space, we erase it. And we can always page through the paper copy to access the board history. 

In Twitter, this means that each follower that follows a group of people is basically polling for messages posted by those users from the last time he read them. To make things more tangible we can express this type of query with the following SQL syntax:

SELECT * FROM Post WHERE UserID=<id> AND PostedOn > <from date>.

The <from date> will normally be the last few minutes, if we're constantly looking for new messages.

But there's a caveat. Remember that we partitioned the application by user-id? This means that each user's tweets are stored in a separate partition. How can we read all users' posts? If we poll for each user individually, we will end up with a lot of network calls. The simplest approach would be to execute one call that looks for ALL the users we're following and look for updates (new tweets) from those users. The pattern we'll use to perform such this task is mapreduce. One way to do that with GigaSpaces is through the distributed task API:

The distributed task API is a modern version of the stored procedure. The following snippet shows what such a call would look like:

AsyncFuture<Long> future = gigaSpace.execute(new GetTweetsUpdates());
long result = future.get(); // result will be the number of primary spaces

The GetTweetsUpdates() class contains code that will be injected in each partition and will enable us to look for updates from the users we follow in a single call. Because the call runs in-process, and because the data is stored in-memory, executing such a task is extremely fast compared with the equivalent with database and stored procedure operations. Execution is aggregated to the caller implicitly. The caller can use a reducer to aggregate the results into a single result object.

Scaling the web front-end

Nothing really new here. We'll use a classic web front-end, which is comprised of a load-balancer and a cluster of web servers that act as a front end to our IMDG instances. The web application will use a single cluster-aware IMDG proxy to send new tweet posts. The IMDG proxy will be responsible for mapping the tweet with the actual partition that hosting the tweet. That logic is kept completely out of the application code. This allows us to keep our web tier clean and simple.

Keeping the web layer stateless to avoid session stickiness

One common pattern for keeping the web tier scalable is to use a Shared-Nothing Architecture, which basically means that the web tier will be stateless. This requires keeping the user session state external to the web-tier. As previously demonstrated, the IMDG can be used as high-performance, scalable data store for maintaining shared session state information. This allows us to avoid session stickiness and to scale the web tier without being locked in to a specific server throughout the entire session, in case the server is over-loaded.

For more information on how to scale the web tier, as well as other important capabilities such as self-healing and auto-scaling, see the following tutorial: Scaling Your Web Application.

Making it simple and cost-effective using cloud computing

Twitter is yet another example for a situation in which system load is highly variable and the difference between average load and peak load can be quite significant. In such cases, provisioning our system can be fairly hard and costly. This is where cloud computing and SLA-driven deployments can help us scale on demand and pay only for what we use.

Once we figured out a way to partition the application, it's going to be much simpler to package the application into self-sufficient units (referred to in GigaSpaces as processing-units) and scale the application simply by adding or removing these units on demand. You can learn more about this here

Final words

Scaling a real-time web application such as Twitter or Facebook introduces unique challenges that are are quite different from those of a "classic" database-centric application. The most profound difference is the fact that unlike with traditional sites, Twitter is a heavy read/write application, and not read-mostly. This seemingly minor difference can break most existing models for web application scalability. Using a combination of memcached + MySQL is not going to cut it for this type of application. 

The good news is that with the right patterns and set of tools, building a scalable architecture that meets such challenges isn’t that difficult.  There are already plenty of success stories that demonstrate that, such as the following example from highscalability.com: Handle 1 Billion Events Per Day Using a Memory Grid

The proposed architecture is by no means perfect and can be further optimized to meet even better performance and latency, but that will come at the cost of simplicity. I believe that the proposed architecture should get you pretty far as-is. Avoid going through more advanced optimizations until the point they are an absolute must.

References

• Twitter getting bigger - Scaling becomes more of an issue

• Memory-Based Architectures and the Cloud

• Handle 1 Billion Events Per Day Using a Memory Grid

• Are Cloud-Based Memory Architectures the Next Big Thing?

• Scaling Your Web Application

• Persistence on the Cloud – How and Why?

• Scaling twitter presentation by Guy Nirpaz

Challanges for Developing Enterprise Application on the Cloud

In the past few weeks I found myself  involved in various discussions centered around the challenges that enterprises face today when they want to deploy their application on the cloud.

These discussions were very timely as they gave me some interesting ideas for my talk next week Practical Guide for Developing Enterprise Application on the Cloud, taking place on Monday, April 20 at 11:00 am Eastern Daylight Time (GMT -04:00, New York), at the CloudSlam online conference.  

I thought that some of those discussions are worth sharing. In this post I'll try to summarize the main highlights from those discussions.

I’ll start by pointing to the following discussion thread on the cloud mailing list:  Challenges faced by developers and architects when moving to the cloud:

 Kent Langley responded by listing the following challenges in his response:

• Dealing with a lack of persistence in some cases Dealing with distributed programming models (prob. one of the most important ones imho) Having to think about the whole stack.  Not just the code.
• Caching considerations
• Messaging
• Using Memory Data Grids
• Understanding configuration management tools that might be involved Working more closely with the operations group in some case

Kevin Apte added the following comment:

There are many challenges- There is no out-of-the-box infrastructure for
hosting the typical J2EE and SOA Stack in the cloud. There is no Weblogic,
WebSphere, ALBPM, Message Bus like Tibco available in the cloud.

A development team could certainly move all of this into the cloud, but the
configuration, licensing issues etc. are all something the team would have
to solve on its own.  This is far too bleeding edge for many people.

Robert Hankel added other challenges

..   The problem of adding additional resources dynamically (e.g. more WebCache instances, or WebLogic servers) requires sophisticated distributed system management infrastructure where the entity being managed is no longer a physical or virtual box, but rather an array of boxes acting collectively as a single system..

Grig Gheorghiu posted an interesting write-up Experiences deploying a large-scale infrastructure in Amazon EC2, where he provide a very insightful summary of his lessons with deploying a large scale application on EC2.

Below are the main takeaways from Grig's summary, which I found relevant for this discussion:

1) Deploy multiple Web servers
2) Deploy multiple load balancers
3) Deploy several database servers.
4) Another way of dealing with databases is to not use them

Challenges summary?

It's easy to see that there is a common theme behind all those comments. Taking existing enterprise applications to the cloud can be very difficult simply because a) most of today's enterprise applications were built using frameworks and technologies not yet supported as first class citizen by cloud providers and b) most of those applications were not designed to take advantage of the cloud's elasticity.

Rather then pointing to my direct response to each of those challenges i thought that it would be better to provide a short summary of the main possible solutions that came through this discussion.

Does it have to be that difficult?

No. Below are two main approaches to those challenges.

- Packaging static images

The simplest approach would obviously be to package your local IT environment into images that could be easily ported to the cloud in the exact same way they run in your local IT environment; right? Well, yes, you can package anything in an image bundle and host your virtual machines in a reserved mode with fixed IP configuration. However, being able to technically do that doesn't mean that it makes sense. I would question what's the difference between this environment and any other hosting environment, and what do you expect to get by moving to such a hosted environment vs. running it in your local IT environment.

If you are going to try deploying your existing IT application on the cloud using static images, then most likely you'll end up "porting" not just the application but also the problems you were facing in your local IT environment; i.e. your application will be over-provisioned based on the peak load and you’ll end up with poorly utilized environment.

- Fully elastic application

The main driver for moving to cloud based environment in the first place was to be able to grow as you need and pay for what you use.

The question is whether you can deploy your application without changes to the application while at the same time leverage the elasticity that cloud brings.

Sounds impossible? Well, a good example that does just that is Storage. With storage you can take your existing application, run it with your local (static) disk, and then plug in a network storage and run the same application on that network device, without changing the application. In that world, instead of taking your existing local disk and virutalize it, you are taking the application and plug it to another device that has visualization built-in.

We can use the same approach as with storage; i.e. move your existing application code and run on top of a different underlying implementation that will enable you to capture the elasticity of the cloud without forcing you to re-write your entire application. If you're running in a JEE environment, it should be fairly easy. If your application has strong "ties" to back-end systems, you can use the hybrid model where your application front-end is running on the cloud, while being connected to the back-end system, using secured communication channel.

Is it really that simple?

Yes. My experience with the integration work that we had done between our Cloud Computing Framework (CCF) and EC2 showed me that getting a production-ready JEE application, including a load-balancer, self healing, auto scaling, security, database and even data grid capabilities plugged in, is actually much simpler than with any other environment I'm aware of. This is due to the built-in automation, predefined images, and the fact that I don't need to download and setup anything to get the entire system up and running. In fact, it's so simple that we decided to built our entire Demo as a Service framework around it; a framework that is used quite successfully and constantly, with customers, prospects, and now with partners.

Are there any production references?

Yes.  Read Jim Liddle's blog post and see a good example of an Enterprise JEE deployment that is already running on EC2, in production, on top of our new Cloud Computing Framework. In his post, Jim describes how this Telco operator were able to address some of the common challanges that were mentioned above such as security,flexability, cost, development complexity and lock-in as well as   high-avliability and scalability in a relatively simple manner .

What next?

In my discussion in the CloudSlam event I’ll try to provide a more detailed practical guide that demonstrating how you can take a step by step approach for porting existing JEE application to the cloud. I hope to get lots of questions and feedback during the discussion so that I can share those with you in one of my follow-up posts.

Is Cloud Computing a revolution or an evolution?

Over the past few months I occasionally got into heated debate with different colleagues regarding whether cloud computing is a revolution or an evolution. Will it change our world, or is it going to be more of the same thing with different flavors?

Well, I must admit that I didn’t made up my mind yet. One of the ways to answer this question is by looking at similar trends in other industries.

I recently came across an interesting post by Clay Shirky, Newspapers and Thinking the Unthinkable, where he discuss the impact of the internet on the newspaper industry and specifically the idea behind paying for news in similar fashion that people pay for music. I brought some of his thoughts below where he describes human behavior when a disruptive idea is brought to a well established industry:

Revolutions create a curious inversion of perception. In ordinary times, people who do no more than describe the world around them are seen as pragmatists, while those who imagine fabulous alternative futures are viewed as radicals. The last couple of decades haven’t been ordinary, however. Inside the papers, the pragmatists were the ones simply looking out the window and noticing that the real world was increasingly resembling the unthinkable scenario. These people were treated as if they were barking mad. Meanwhile the people spinning visions of popular walled gardens and enthusiastic micropayment adoption, visions unsupported by reality, were regarded not as charlatans but saviors. […]

When reality is labeled unthinkable, it creates a kind of sickness in an industry. Leadership becomes faith-based, while employees who have the temerity to suggest that what seems to be happening is in fact happening are herded into Innovation Departments, where they can be ignored en masse. This shunting aside of the realists in favor of the fabulists has different effects on different industries at different times. One of the effects on the newspapers is that many of their most passionate defenders are unable, even now, to plan for a world in which the industry they knew is visibly going away. […]

With the old economics destroyed, organizational forms perfected for industrial production have to be replaced with structures optimized for digital data. […]

That is what real revolutions are like. The old stuff gets broken faster than the new stuff is put in its place. The importance of any given experiment isn’t apparent at the moment it appears; big changes stall, small changes spread. Even the revolutionaries can’t predict what will happen. Agreements on all sides that core institutions must be protected are rendered meaningless by the very people doing the agreeing. (Luther and the Church both insisted, for years, that whatever else happened, no one was talking about a schism.) Ancient social bargains, once disrupted, can neither be mended nor quickly replaced, since any such bargain takes decades to solidify. […]

And so it is today. When someone demands to know how we are going to replace newspapers, they are really demanding to be told that we are not living through a revolution. They are demanding to be told that old systems won’t break before new systems are in place. They are demanding to be told that ancient social bargains aren’t in peril, that core institutions will be spared, that new methods of spreading information will improve previous practice rather than upending it. They are demanding to be lied to.

There are fewer and fewer people who can convincingly tell such a lie.

Now try to replace the term newspaper industry with “IT industry” and see what you get.

What are your thoughts on this matter, does cloud computing represent a revolution or an evolution?

Demo as a Service

Many of you out there who tried to run a demo online in front of a live audience are probably very familiar with the feeling when the demo is not working. Bill Gates trying to show Windows Media Center on Conan O’Brian is a good example of that :)

Having gone through this experience so many times, I know that getting a demo running successfully is not a trivial task at all.

There are many challenges involved in getting a demo right:

• Getting the demo inline with latest and greatest – every new version can break your demo. Since in most cases the demo is not going through the normal QA and testing procedures, the chance that your demo might break because of new changes get fairly high.
• Disseminating the demo among all the peers in the organization is also a challenge – if your demo is a little bit complex and requires installation of a database, a web server, etc., most likely you’ll end up with each sales rep having his own version of the demo, if at all. This is simply due to the fact that it takes time to get all those pieces to work and every update becomes a real project.
• Governing the quality of the demo – once the demo is on the hands of those sales rep you have absolutely no control over what they are running. Many will try to add their own tweaks, and without noticing end up with a broken demo or a demo that is not geared to highlight the right set of features.
• Demonstrating non-functional behavior such as scalability and availability on your own laptop is not that convincing. For example with GigaSpaces, which is a middleware product, scalability, performance and high availability are our main features. You can imagine how “impressed” I would be if you would showed me this type of feature on a single laptop.
• Now imagine what can happen in the ideal scenario, where everything worked out just fine, and the customer asked for a copy of the demo – ouch! This is where you’re going to try to find some really good excuses, because chances are that by giving the customer your demo she is going to be exposed to details you really didn’t want her to be exposed to at this stage.
• If you do get the chance to try and install the demo on the customer’s machines you might succumb to the “it’s not working on my machine” syndrome, which is often caused by environmental issues that are not so easy to spot.
• Last but not least is the collaboration challenge – in most cases demos are one of those tasks where we don’t have dedicated resources allocated to maintain and develop them. This is a classic scenario where collaboration can help to leverage the power of all the users and enable everyone to contribute to the demo in an effective manner. But this requires an initial effort to set such a collaborative environment, which the majority of us don’t spend when we set a demo. At the same time, there are only small number of users who would be willing to go through this investment themselves and contribute to such an environment.

The solution: “Demo as a Service”

The idea behind Demo as a Service is basically to apply the same principles of SaaS model to our demo.

This means that the demo would be fully accessible through the web (even if it’s a desktop application), there would be no installation required. It will be completely multi-tenant, meaning that each user can run on her own isolated environment even though she would share the same infrastructure with other users.

In this section I can refer to GigaSpaces as an example of a company that faced the obstacles I mentioned above and needed such a demo environment. I’ll share some of the principles behind the approach we eventually took to address these challenges. Assuming that others would be interested in creating a similar demo environment, we designed the system in a generic way that can be easily used by any user, even those that are not using our product, to run their own Demo as a Service without going through the same level of investment.

Here are the Demo as a Service platform design principles:

• Using cloud computing – as you can imagine, many of the things that I'll describe wouldn’t be possible if it wasn’t for cloud computing. Cloud computing enabled us to create a full production environment per demo in matter of two-three minutes, through a browser, from anywhere in the world. All you needed is an account set up and Internet access.
• No installation – only Internet access is required. I must admit that achieving this goal was a bit tricky at first, seeing as our product is not demonstrable through a web UI, and even if it was, there is not much you can see without running an application on top of it. So we had to create an environment that will enable you to deploy a demo application and watch what happens behind the scenes while the application was running. This included the ability to watch the cluster as it grows or shrink, crash machines and see how the application reacts to the event, etc. To achieve these goals, we set up the environment in such away that users can actually log into the servers running the application. We also needed to show the system view – CPU utilization, memory, etc. – of those specific machines. This meant that we needed to create system monitoring per demo and wire it on the fly so that it will monitor only the machines running that specific demo.
• Demo on demand – in our case we needed to support the option of creating a full demo environment that includes a load balancer, web containers, a Data Grid and a database, all in one click. Seeing as this happens while we’re facing users, we couldn’t afford any manual operations, and had to ensure that the entire setup would take less than five minutes. At the same time, when the demo is finished, we wanted to be able to clean the machines completely in one click. This had to be done in a way that wouldn’t conflict with other potential users running other demos at the same time.
• Fail proof – getting a reliable Internet connection is not always possible, and quite often the connection gets broken just a few minutes before the demo starts. We wanted to be able to recover easily from this type of failure. For this purpose, the web application we use to launch the demo is kept completely stateless. In addition, we keep the entire application in the cloud server repository, so we don’t need to ship any code from the desktop to the server. This enabled us to run the demo even under a low-bandwidth network. In case of connection failure, the deployment is not affected, and once the connection is re-established we are able to get the current state of the system as if nothing happened.
• Demos should be easily shared – one of the problems with demos, or any stand-alone application, is that it’s very hard to share the environment the application is running in, and therefore we found ourselves shipping different versions of the application when we wanted to collaborate or troubleshoot the application. One of the benefits of cloud computing is that I can easily share the resources I'm using over the Internet. This can also be useful if I want to interactively work with another user on the same machine, or even troubleshoot what she is doing.  All I need to do is log into the same machine and see what’s going on. There’s no need to ship any code. I can even fix issues immediately as I discover them, without needing to send long emails and package my patch for hours.
• Simple upgrade – we wanted to be able to launch a new version of our software, or the demo, very easily. We also wanted to enable users to use their current version and control when they do the switch to the new version. To achieve this, we used a shared repository model. The shared repository always keeps the latest update as well as the pervious version. The application only needs to indicate which version it wants to use, via an XML configuration file, and that’s it. In this case, we avoid the need to disseminate copies of our software with all our people or even force them to change their demos at the same time. All they need to do is to point the demo to the right version on S3, and when the demo runs it picks up that version and uses it.
• Governing the quality of the demo – now that the demo is running on a shared environment, we can ensure that everyone is using the same copy of the demo. Since we control not only the code but also the environment the demo is running in, we can eliminate any chances to the environment or human errors, which often happens when you ship a copy of the demo. If there is a fix that needs to be applied, it is applied immediately and everyone gets it. 
• Support desktop (fat-client) applications – there are cases in which the application needs to run a desktop application for its UI. In our case, part of our management tools needs to be accessed that way. For that purpose, each machine that is running can start the desktop application on the server and choose whether to launch only the application screen or the entire desktop. This is done without installing any software and without any manual setup.
• Simulate a real production environment – in order to demonstrate things like failover and scaling in a convincing way, we needed to use real machines and a real data center. This is another area where cloud computing came in pretty handy. We could easily instantiate real machines and allocate specific machines per demo and per user. This is extremely powerful, because it gives users the ability to experience the full product in a matter of minutes, without needing to download it or install it.
• Support multiple users at the same time (multi-tenancy) – multi-tenancy is a core principle in SaaS. It basically means that users can run on a shared environment while still being completely independent and isolated from each other. In our specific case, this means that:
  • All users share the same demo management portal.
  • The same user can run multiple application in complete isolation.
  • Different users can run at the same time in complete isolation.
  • Each application can be managed independently.

To achieve that goal, we had to design the management console to enable such isolation. In our case, we could use the same Amazon account to serve multiple distinct users, in other words there is no direct mapping between an Amazon user and our system user. The same user can run multiple application or different versions of the same application at the same time, and manage them separately.

Can I try it live now?

Yes. The built-in demos are available on www.gigaspaces.com/mycloud – to run the demos you don’t need to have an Amazon account, you can use a free trial user, which basically means that you’ll be running on our behalf. Note that this will only give you access to the built-in demos not your own application. A description of the available demos and how to run them is provided here.

Can I use it to deploy my own (non-GigaSpaces) demo?

Yes, you can use tihs same environment to deploy Tomcat or any other application fairly easily. Note that this option will require that you log in using your Amazon account.

An example on how to deploy your own Tomcat application is available here.

Note that this mode only provides the ability to launch new custom machines but doesn’t include all the elasticity, self-healing functionality that you would gain by deploying your web application on GigaSpaces XAP.

What’s the difference between a demo environment and a production environment?

One of the interesting thing is that there is not much difference. The difference would most likely be configuration – for example, security setup, or the size of the machines, which might be different in a demo because you would likely use low-class machines. Other than that, it’s pretty much the same environment. The beauty of that, IMO, is that you can move from demo to development to production without changing the environment.

Can I use this environment to run a POC and benchmark?

Yes. Since you get full access to the machines, you can keep them running as long as you wish, customize them and install your own software as well. This is actually the recommended way to run a POC, as it is more cost effective. You don’t spend time allocating the machines, setting up the environment and tuning the system, things that often take weeks. It is also easier to share the environment between two separate parties, e.g. the end-user and the vendor, without going through all the security processes required when you attempt to get access to the customer’s on-premise machines. In this way, you don’t need to ship code around when something is not working properly. I can log into the machine and see the exact behavior you’re seeing, and fix things immediately.

Final words

Demo as a Service is a very powerful model for overcoming many of the challenges involved in running demos and POCs. Cloud computing provides totally new capabilities, which allowed us at GigaSpaces to create a full demo environment for our product, at the click of a button. Having said that, based on our experience, setting up a full demo environment can take a while and would require investment. The mycloud environment was designed to get you the benefit of Demo as a Service while saving you the effort required to implement one. It is still in its early stages but is already being used extensively by our sales team and our partners. I’m eager to hear more feedback from users and suggestions on how to can make it better.