Nati Shalom's Blog

Is Cloud Computing a revolution or an evolution?

Over the past few months I occasionally got into heated debate with different colleagues regarding whether cloud computing is a revolution or an evolution. Will it change our world, or is it going to be more of the same thing with different flavors?

Well, I must admit that I didn’t made up my mind yet. One of the ways to answer this question is by looking at similar trends in other industries.

I recently came across an interesting post by Clay Shirky, Newspapers and Thinking the Unthinkable, where he discuss the impact of the internet on the newspaper industry and specifically the idea behind paying for news in similar fashion that people pay for music. I brought some of his thoughts below where he describes human behavior when a disruptive idea is brought to a well established industry:

Revolutions create a curious inversion of perception. In ordinary times, people who do no more than describe the world around them are seen as pragmatists, while those who imagine fabulous alternative futures are viewed as radicals. The last couple of decades haven’t been ordinary, however. Inside the papers, the pragmatists were the ones simply looking out the window and noticing that the real world was increasingly resembling the unthinkable scenario. These people were treated as if they were barking mad. Meanwhile the people spinning visions of popular walled gardens and enthusiastic micropayment adoption, visions unsupported by reality, were regarded not as charlatans but saviors. […]

When reality is labeled unthinkable, it creates a kind of sickness in an industry. Leadership becomes faith-based, while employees who have the temerity to suggest that what seems to be happening is in fact happening are herded into Innovation Departments, where they can be ignored en masse. This shunting aside of the realists in favor of the fabulists has different effects on different industries at different times. One of the effects on the newspapers is that many of their most passionate defenders are unable, even now, to plan for a world in which the industry they knew is visibly going away. […]

With the old economics destroyed, organizational forms perfected for industrial production have to be replaced with structures optimized for digital data. […]

That is what real revolutions are like. The old stuff gets broken faster than the new stuff is put in its place. The importance of any given experiment isn’t apparent at the moment it appears; big changes stall, small changes spread. Even the revolutionaries can’t predict what will happen. Agreements on all sides that core institutions must be protected are rendered meaningless by the very people doing the agreeing. (Luther and the Church both insisted, for years, that whatever else happened, no one was talking about a schism.) Ancient social bargains, once disrupted, can neither be mended nor quickly replaced, since any such bargain takes decades to solidify. […]

And so it is today. When someone demands to know how we are going to replace newspapers, they are really demanding to be told that we are not living through a revolution. They are demanding to be told that old systems won’t break before new systems are in place. They are demanding to be told that ancient social bargains aren’t in peril, that core institutions will be spared, that new methods of spreading information will improve previous practice rather than upending it. They are demanding to be lied to.

There are fewer and fewer people who can convincingly tell such a lie.

Now try to replace the term newspaper industry with “IT industry” and see what you get.

What are your thoughts on this matter, does cloud computing represent a revolution or an evolution?

Demo as a Service

Many of you out there who tried to run a demo online in front of a live audience are probably very familiar with the feeling when the demo is not working. Bill Gates trying to show Windows Media Center on Conan O’Brian is a good example of that :)

Having gone through this experience so many times, I know that getting a demo running successfully is not a trivial task at all.

There are many challenges involved in getting a demo right:

• Getting the demo inline with latest and greatest – every new version can break your demo. Since in most cases the demo is not going through the normal QA and testing procedures, the chance that your demo might break because of new changes get fairly high.
• Disseminating the demo among all the peers in the organization is also a challenge – if your demo is a little bit complex and requires installation of a database, a web server, etc., most likely you’ll end up with each sales rep having his own version of the demo, if at all. This is simply due to the fact that it takes time to get all those pieces to work and every update becomes a real project.
• Governing the quality of the demo – once the demo is on the hands of those sales rep you have absolutely no control over what they are running. Many will try to add their own tweaks, and without noticing end up with a broken demo or a demo that is not geared to highlight the right set of features.
• Demonstrating non-functional behavior such as scalability and availability on your own laptop is not that convincing. For example with GigaSpaces, which is a middleware product, scalability, performance and high availability are our main features. You can imagine how “impressed” I would be if you would showed me this type of feature on a single laptop.
• Now imagine what can happen in the ideal scenario, where everything worked out just fine, and the customer asked for a copy of the demo – ouch! This is where you’re going to try to find some really good excuses, because chances are that by giving the customer your demo she is going to be exposed to details you really didn’t want her to be exposed to at this stage.
• If you do get the chance to try and install the demo on the customer’s machines you might succumb to the “it’s not working on my machine” syndrome, which is often caused by environmental issues that are not so easy to spot.
• Last but not least is the collaboration challenge – in most cases demos are one of those tasks where we don’t have dedicated resources allocated to maintain and develop them. This is a classic scenario where collaboration can help to leverage the power of all the users and enable everyone to contribute to the demo in an effective manner. But this requires an initial effort to set such a collaborative environment, which the majority of us don’t spend when we set a demo. At the same time, there are only small number of users who would be willing to go through this investment themselves and contribute to such an environment.

The solution: “Demo as a Service”

The idea behind Demo as a Service is basically to apply the same principles of SaaS model to our demo.

This means that the demo would be fully accessible through the web (even if it’s a desktop application), there would be no installation required. It will be completely multi-tenant, meaning that each user can run on her own isolated environment even though she would share the same infrastructure with other users.

In this section I can refer to GigaSpaces as an example of a company that faced the obstacles I mentioned above and needed such a demo environment. I’ll share some of the principles behind the approach we eventually took to address these challenges. Assuming that others would be interested in creating a similar demo environment, we designed the system in a generic way that can be easily used by any user, even those that are not using our product, to run their own Demo as a Service without going through the same level of investment.

Here are the Demo as a Service platform design principles:

• Using cloud computing – as you can imagine, many of the things that I'll describe wouldn’t be possible if it wasn’t for cloud computing. Cloud computing enabled us to create a full production environment per demo in matter of two-three minutes, through a browser, from anywhere in the world. All you needed is an account set up and Internet access.
• No installation – only Internet access is required. I must admit that achieving this goal was a bit tricky at first, seeing as our product is not demonstrable through a web UI, and even if it was, there is not much you can see without running an application on top of it. So we had to create an environment that will enable you to deploy a demo application and watch what happens behind the scenes while the application was running. This included the ability to watch the cluster as it grows or shrink, crash machines and see how the application reacts to the event, etc. To achieve these goals, we set up the environment in such away that users can actually log into the servers running the application. We also needed to show the system view – CPU utilization, memory, etc. – of those specific machines. This meant that we needed to create system monitoring per demo and wire it on the fly so that it will monitor only the machines running that specific demo.
• Demo on demand – in our case we needed to support the option of creating a full demo environment that includes a load balancer, web containers, a Data Grid and a database, all in one click. Seeing as this happens while we’re facing users, we couldn’t afford any manual operations, and had to ensure that the entire setup would take less than five minutes. At the same time, when the demo is finished, we wanted to be able to clean the machines completely in one click. This had to be done in a way that wouldn’t conflict with other potential users running other demos at the same time.
• Fail proof – getting a reliable Internet connection is not always possible, and quite often the connection gets broken just a few minutes before the demo starts. We wanted to be able to recover easily from this type of failure. For this purpose, the web application we use to launch the demo is kept completely stateless. In addition, we keep the entire application in the cloud server repository, so we don’t need to ship any code from the desktop to the server. This enabled us to run the demo even under a low-bandwidth network. In case of connection failure, the deployment is not affected, and once the connection is re-established we are able to get the current state of the system as if nothing happened.
• Demos should be easily shared – one of the problems with demos, or any stand-alone application, is that it’s very hard to share the environment the application is running in, and therefore we found ourselves shipping different versions of the application when we wanted to collaborate or troubleshoot the application. One of the benefits of cloud computing is that I can easily share the resources I'm using over the Internet. This can also be useful if I want to interactively work with another user on the same machine, or even troubleshoot what she is doing.  All I need to do is log into the same machine and see what’s going on. There’s no need to ship any code. I can even fix issues immediately as I discover them, without needing to send long emails and package my patch for hours.
• Simple upgrade – we wanted to be able to launch a new version of our software, or the demo, very easily. We also wanted to enable users to use their current version and control when they do the switch to the new version. To achieve this, we used a shared repository model. The shared repository always keeps the latest update as well as the pervious version. The application only needs to indicate which version it wants to use, via an XML configuration file, and that’s it. In this case, we avoid the need to disseminate copies of our software with all our people or even force them to change their demos at the same time. All they need to do is to point the demo to the right version on S3, and when the demo runs it picks up that version and uses it.
• Governing the quality of the demo – now that the demo is running on a shared environment, we can ensure that everyone is using the same copy of the demo. Since we control not only the code but also the environment the demo is running in, we can eliminate any chances to the environment or human errors, which often happens when you ship a copy of the demo. If there is a fix that needs to be applied, it is applied immediately and everyone gets it. 
• Support desktop (fat-client) applications – there are cases in which the application needs to run a desktop application for its UI. In our case, part of our management tools needs to be accessed that way. For that purpose, each machine that is running can start the desktop application on the server and choose whether to launch only the application screen or the entire desktop. This is done without installing any software and without any manual setup.
• Simulate a real production environment – in order to demonstrate things like failover and scaling in a convincing way, we needed to use real machines and a real data center. This is another area where cloud computing came in pretty handy. We could easily instantiate real machines and allocate specific machines per demo and per user. This is extremely powerful, because it gives users the ability to experience the full product in a matter of minutes, without needing to download it or install it.
• Support multiple users at the same time (multi-tenancy) – multi-tenancy is a core principle in SaaS. It basically means that users can run on a shared environment while still being completely independent and isolated from each other. In our specific case, this means that:
  • All users share the same demo management portal.
  • The same user can run multiple application in complete isolation.
  • Different users can run at the same time in complete isolation.
  • Each application can be managed independently.

To achieve that goal, we had to design the management console to enable such isolation. In our case, we could use the same Amazon account to serve multiple distinct users, in other words there is no direct mapping between an Amazon user and our system user. The same user can run multiple application or different versions of the same application at the same time, and manage them separately.

Can I try it live now?

Yes. The built-in demos are available on www.gigaspaces.com/mycloud – to run the demos you don’t need to have an Amazon account, you can use a free trial user, which basically means that you’ll be running on our behalf. Note that this will only give you access to the built-in demos not your own application. A description of the available demos and how to run them is provided here.

Can I use it to deploy my own (non-GigaSpaces) demo?

Yes, you can use tihs same environment to deploy Tomcat or any other application fairly easily. Note that this option will require that you log in using your Amazon account.

An example on how to deploy your own Tomcat application is available here.

Note that this mode only provides the ability to launch new custom machines but doesn’t include all the elasticity, self-healing functionality that you would gain by deploying your web application on GigaSpaces XAP.

What’s the difference between a demo environment and a production environment?

One of the interesting thing is that there is not much difference. The difference would most likely be configuration – for example, security setup, or the size of the machines, which might be different in a demo because you would likely use low-class machines. Other than that, it’s pretty much the same environment. The beauty of that, IMO, is that you can move from demo to development to production without changing the environment.

Can I use this environment to run a POC and benchmark?

Yes. Since you get full access to the machines, you can keep them running as long as you wish, customize them and install your own software as well. This is actually the recommended way to run a POC, as it is more cost effective. You don’t spend time allocating the machines, setting up the environment and tuning the system, things that often take weeks. It is also easier to share the environment between two separate parties, e.g. the end-user and the vendor, without going through all the security processes required when you attempt to get access to the customer’s on-premise machines. In this way, you don’t need to ship code around when something is not working properly. I can log into the machine and see the exact behavior you’re seeing, and fix things immediately.

Final words

Demo as a Service is a very powerful model for overcoming many of the challenges involved in running demos and POCs. Cloud computing provides totally new capabilities, which allowed us at GigaSpaces to create a full demo environment for our product, at the click of a button. Having said that, based on our experience, setting up a full demo environment can take a while and would require investment. The mycloud environment was designed to get you the benefit of Demo as a Service while saving you the effort required to implement one. It is still in its early stages but is already being used extensively by our sales team and our partners. I’m eager to hear more feedback from users and suggestions on how to can make it better.

Leadership is not management

In one of my recent flights (and i had too many of them …)  to Melbourne Australia i pulled a book which caught my attention just few minutes before the plane took off.

Its one of those moments where you see something and it takes you few seconds to decide that “that’s the one”.

I’m referring to Seth Godin book Tribes.

Tribes is one of those inspiring books that makes you think during and long after you read it.

Here are some of Seth wisdom that i picked from the book

Leadership is not management:

Management is about manipulating resources to get a known job done,..leadership is about creating a change that you believe in.

Movement have leaders and movement makes things happen. Leaders have followers Managers have employees. Managers makes widgets. Leaders make change.

…

Managers manage by using authority the factory gives them. You listen to your manager or you loose your Job. A manager can’t make change because that’s not his Job. His Job is to complete tasks assigned to him by someone else in the factory.

Leaders on the other hand don’t care that much for organizational structure or the official blessing of whatever the factory they work for. They use passion and ideas to lead people, as appose to using threats and Burecreacrucy to manage them. Leaders must become aware of how the organization they works, because this awareness allows them to use it. Leaders doesn’t always start at the top but it always manage to effect the folks at the top.

How do you manage leaders?

Given that leaders can appear anywhere in an organization, it seems to me that the job of senior management is to find them and support them. Leaders have tribes of their own, and someone needs to lead those tribes. Which leads to the idea of positive deviance.
As a general rule, managers don't like deviants. By definition, deviance from established standards is a failure for a manager working to deliver on spec. So, most of the time, most managers work hard to stamp out deviance (and the deviants who create it). Managers stamp out deviants. That's what they do.

Leaders understand a different calculus. Leaders understand that change is not only omnipresent, but the key to success.
And it turns out that employees who are committed to change and engaged in making things happen are happier and more productive. Putting these two facts together, it's not hard to come to the conclusion that you desperately need more leaders, more deviants-more agents of change, not fewer

Change

Change isn't made by asking permission. Change is made by asking forgiveness, later.

Food for thoughts…

Its time for auto scaling – avoid peak load provisioning for web applications

Many web applications, including eBanking, Trading, eCommerce and Online Gaming, face large, fluctuating loads. Some of the load is predictable, for example, if you launch a new campaign you expect to see certain growth around that campaign; some of the load is seasonal, like in Christmas for most eCommerce sites or quarterly-based in other cases. But today, with social networks and the speed at which we can disseminate information around the globe, we’re seeing more and more unpredictable load. This effect was known in the financial industry as the “perfect storm,” which basically meant that a chain of events in the market led to a storm of events that brought many systems down. In social networks, this type of viral behavior is also referred to as the Digg effect. The Digg effect means that a site can generate a single news item that will generate more traffic than was ever anticipated to be handled by the site, in a fraction of a second. The barrier for generating such events has been significantly reduced, due to all the new tools available today such as twitter, Digg and Facebook, a trend is appearing across many industries where over time, online systems need to face ever-growing, unpredictable peak load events.

 

Peak load behavior in various industries

The effect of peak-load provisioning

Over-provisioning

A traditional way of dealing with loads is to provision the system for peak load. You try to estimate what’s going to be the peak load on the system, then double the number to have enough spare in case you were wrong, and buy the amount of hardware that will be able to handle this most extreme scenario. Interestingly enough, that number determines not just the amount of servers you’ll in the production environment, but also the staging and to a lesser degree the testing and development environments. It also determines the number of software licenses, etc. Now, when this number is low – say 2,3 servers – that’s not ideal but still manageable. But what happens when you need 10 servers or 100 servers? Also, what will the organization’s data center will look like if each application provisions its own spare capacity? Its really simple to imagine, right? On average, the data center will be extremely over-provisioned, which means the system will be extremely under-utilized most of the time, which means a huge waste of money.

Under-provisioning

The other side of being over-provisioned is being under-provisioned. Being under-provisioned means that you have too little resources to handle the traffic that the system is experiencing. Well, that’s just a way to put this in nice words. Remember the perfect storm?

 

Public failure due to under-provisioning

I've collected only a few examples that show what under provisioning can lead to – failure! The fact that the system is now online means that failure is also online, and quickly becomes a news item, resulting in a PR nightmare for your company.

“Right-sizing” – an oxymoron?

By right-sizing, I mean provisioning the system to fit its needs exactly. There's a small caveat here – how do you know the right size? With the viral behavior of traffic these days, trying to predict the load is almost like trying to predict the weather – you can probably have a good guess for next month, but your guess is going to be less accurate the longer the period you’re trying to estimate. So the question is, can we do right-sizing?

Right-sizing under fluctuating load

A basic assumption is that you can’t do right-sizing for a long period of time – even a year. You need to estimate the load for short window of time, and then be able to up-scale or down-scale the system system easily when you see there is a need for it. I think there are primarily two ways we can achieve this type of scaling – flexible scaling and auto scaling:

• Flexible scaling – this means that scaling up or down will be a matter of plugging a machine into the network and wouldn’t require re-architecture, changes to our code or even configuration changes. The scaling operation would require a manual process for adding a machine, but wouldn’t require complex development and tuning and, can be done in matter of minutes once the machine is plugged into the network.
• Auto-scaling – means that you can monitor the behavior of the system, and based on these metrics, decide in real time whether it needs to grow or shrink. To enable true auto-scaling, we need to have a pool of available resources that we can pull or release in an on-demand basis. This is basically cloud computing – a cloud (whether a public cloud, like Amazon EC2, or a private one running on-premise) provides the flexibility to add these machines on demand and pay for only for what we consume.

The missing piece – middleware virtualization

If you take pretty much any existing application, add another machine into the network, and watch what happens, you wouldn’t be surprised that nothing much would happen at all.

 Non-virtualized middleware

Today’s applications aren’t able to dynamically take advantage of new computing resources that become available. That’s also true for cloud based environments like EC2 – the fact that you can now add machines easily is nice, but it doesn’t mean that the application can do anything with it. What is missing is a layer that helps the application take advantage of these new resources dynamically, as they are being added to the system. This is where middleware virtualization comes to the rescue.

Virtualized middleware

Like in storage virtualization, where we can add physical disks easily without touching the application that is utilizing them, we can do the same for the web, business logic, messaging and data layers of our applications. The idea is very similar to storage: we decouple the application from the physical resources and then use a client-side proxy that maps between the application API calls and the appropriate resource that will be used to serve that call. On the web layer, you can use a load-balancer to get to this level of virtualization; on the messaging, business logic and data layers, you can use In-Memory-Data-Grids for the same purpose.

Where to start?

By now, you’re probably nodding your head with agreement, but at the same time you’re probably scratching your head thinking, “how the hell do I get there without re-writing my entire application?” Indeed, everything I’ve been saying is pure common sense, yet for years the industry has struggled with this challenge and hasn’t found a way to apply it effectively. So what’s different now?

The step-by-step approach

IMO, the key to successfully achieving flexible scaling or auto-scaling is by finding the right steps that will get you there with minimum risks. The first steps need to show value with very little effort. Once you get into a better comfort level, you can think of taking more serious steps.

If you take a simple web application and break it down to its components (web, business-logic, data) you can easily see that scaling the data layer would probably be harder then scaling the web layer. In this post I'll focus on the steps required to scale the web layer.

How to add auto-scaling to your existing web tier

Auto-scaling on the web tier means the following:

• Dynamically provisioning the web container
• Dynamically discovering the web containers and adding them to the load balancer

In a stateless application, these two should do the trick. However, in a more stateful application, you’ll also need to address the way session data is handled when adding or removing machines. For this, you need to be able to address high availability of session data and and data affinity between the load balancer and the web container.

1. Making the session reliable – to make session information reliable, you can back them in an In-Memory Data Grid, which is kept outside the web container. This way, if the web container changes location due to a scaling event, the session information is kept in that shared memory, and a new server can load it on-demand. To avoid the performance overhead of using remote memory, you can use a local cache that is loaded on-demand. For session information, this is fairly useful, because during a particular session it is very likely that you’ll need to read the same user data over and over again. The combination of local and remote Data Grid allows you to get to the great performance of local memory, while still keeping the memory reliable, and without having to overload each server will all the session data.

2. Maintaining data affinity between the load balancer and the web container: In many cases data affinity is done through “sticky sessions”. This basically means that the load balancer “remembers” the first server the user interacted with, and then during the entire session routes all requests from that particular user to the same server. This method is often based on identifying the server based on its IP – but in a dynamically-scalable environment, this assumption no longer holds. The same user session can be on one particular host at one point in time, and then on a different host in another point in time. To overcome this issue, you need to use a routing scheme that isn’t based on host names, but rather on logical IDs. This means that each web container needs to have a logical ID, and the load-balancer can then route the requests based on this ID. The ID will be consistent regardless of where this web container is running, so this solves the problem.

Further optimization – avoiding session stickiness

Imagine a scenario where a user is now associated with a particular server during a session, and that server becomes overloaded. This is where session stickiness becomes a scalability limitation. To avoid this limitation, you can keep the session data outside the web container on a shared In-Memory Data Grid. This means the same user session can be served at any point in time by any of the web containers – which gives you better scalability. The downside of this approach is performance, because you wouldn’t be exploiting the local memory of each web container. You’d also need to apply more stringent locking on the data, because in this scenario a concurrent update might happen under the same user session.

Try it now on the Amazon cloud

All of the above looks nice on paper – we all know how it works, right? :)

But does it really work? We’ve all gone through the experience of reading something we like, and then going and getting it and being disappointed. And in this case, “trying it out” can be fairly complex, because you need to download the software, read the manual, get several machines deployed and configure the environment, and only then we you really give it a try. If you’re like me, you’ll probably give up right there and go through these steps only if you don’t have a choice. This is another area where cloud-computing can be really handy – getting a production-ready test-drive environment.

We’ve just launched a new service on www.gigaspaces.com/mycloud where we give you access to Amazon EC2, with GigaSpaces XAP running as part of the environment, allowing you to experience how this model works without needing to download any software, setup machines, or configure the clustering and load balancer. You can get a fully production-ready environment with all the capabilities I outlined above above in matter of minutes.

If you want to deploy your existing web application on the cloud, check out the deployment instructions in our Cloud Computing Framework documentation. This process is quite straightforward but nevertheless is still not as easy as it could be. If you want to try out the auto-scaling capabilities really quickly, using a demo application, follow the quick steps below:

1. Visit the mycloud page and get your free access code – this will give you access to the cloud.

2. Select the Trader-Stock-Demo – when you click the demo, the system will start provisioning the web application on the cloud. This will include a machine for managing the cloud, a machine for the load-balancer and machines for the web and Data Grid containers. After approximately 3 minutes, the application will be deployed completely. At this point you should see the “running” link on the load balancer machine. Click on this link to open your web client application.

3. Test auto-scaling – click multiple times on the “running” link to open more clients. This increases the load (requests/sec) on the system. As soon as the requests/sec grow beyond a certain threshold, you’ll see new machines being provisioned. After approximately 2 minutes, the machine will be running and a new web container will be auto-deployed into that machine. This new web container will be linked automatically with the load balancer, and the load balancer in turn will spread the load to this new machine. This will reduce the load on each of the existing servers.

4. Test self-healing – you can now kill one of the machines and see how your web client is behaving. You’ll see that even though the machine was killed, the client was hardly effected and the system scaled itself down automatically.

What’s going on behind the scenes?

All this may seem like magic to you. If you want to access the machines and watch the web containers, the Data Grid instances and the machines, as well as real-time statistics, click the Manage button. This opens up a management console that is running on the cloud, and can be accessed through the web. With this tool you can view all the components of our systems. You can even query the data using an SQL browser and view the data as it enters the system. In addition, you can choose to add more services, and even relocate services from one machine to another just by dragging and dropping them in the UI.

The next steps

As you can see, the combination of middleware-virtualization and cloud computing can bring together the promise of right-sizing through auto-scaling. There are many things that can be done to make this entire process even simpler, not just for the demo application but for your own custom application.

At this point I'd really like to hear your feedback – in particular, I’d like to hear if you see areas that could be improved or areas that are missing in this approach.

To send your feedback you can either post a comment on this post or send an email to pm at gigaspaces dot com.

Additional References

• Quick start guide
• The Missing Piece in Cloud Computing: Middleware Virtualiztion
• Cloud Computing Framework documentation