Nati Shalom's Blog

Open Cloud Deconstructed. What does it mean to be OpenStack Native?

OpenStack is just not like any other cloud.

OpenStack is an open source cloud and that makes it quite unique in comparison to many of the closed source public clouds generally available today, the most common being AWS, Azure GCE, and others.

The fact that it is open source, allows us to integrate with the lower level of the stack and know exactly how each element of the infrastructure is designed and implemented.

As with any open source project we have the option to contribute and influence how the OpenStack cloud ultimately shapes out.

Given the above background, when we're designing an integration with OpenStack, we should approach it differently than we approach other clouds. A more native integration with OpenStack involves two main angles:

• Integration with OpenStack core and infrastructure services, which usually includes this full OpenStack component list - as well as those that are continually being developed and added with every release.

• Follow the OpenStack design principles - OpenStack is written in  Python and uses a certain stack for messaging, database and such . To be OpenStack native you should align yourself with the OpenStack design principles as closely as possible.

Current options to be a part of the OpenStack ecosystem

In addition to the general guidelines outlined above, there are couple of additional options to be part of OpenStack. These options mostly fit in with those looking to take an OpenStack *only* approach, and require heavy involvement in the OpenStack community project.

• OpenStack Core - be a part of the OpenStack distribution

• OpenStack StackForge is an incubation framework that allows you to add new projects into the OpenStack ecosystem. StackForge uses the same set of tools and frameworks that are used to develop the core OpenStack services, and as such it allows smoother transition from an incubation project into the core services. That makes this option more relevant for projects that are likely to be included as an OpenStack core service once they've successfully passed the incubation stages.

• OpenStack Marketplace - The marketplace is a web directory that lists the various ecosystem players and their OpenStack solutions. Currently the marketplace is focused on OpenStack distros, training, consulting and drivers, and doesn’t cover solutions on top of OpenStack.

What’s missing?

While there is a fairly clear process, and tool chain for handling core and incubated services on OpenStack, these options fit mostly with services that are primarily designed to only run on OpenStack. What’s missing is a clear certification process that will allow users to find solutions in the ecosystem, outside of OpenStack, and find out quickly what their level of integration and support for OpenStack is. This should include information such as version support, and platform the solution was certified on, which of the OpenStack services are integrated, how much of the stack of the solution is compatible with that of OpenStack’s, and other such important information. A good and simple starting point could be the OpenStack marketplace, however this is just the first step. What’s needed is a set of frameworks similar to StackForge but much more permissive that will provide a means for validating the compatibility with OpenStack.

Why should you care?

As with any popular framework there are many tools that claim support for OpenStack, however in many cases the integration mostly includes basic integration with the OpenStack compute API (Nova). A good example for this is Cloud Foundry, or other tools such as Chef, Puppet, Docker, just to name a few. While these tools do have support for OpenStack, the level of integration can vary quite dramatically. Quite often, each of these  solutions comes with a parallel stack for handling security, monitoring and other issues, which isn’t compatible with the OpenStack infrastructure. As an end user, it is important that to be able to find out quickly which of these products comes with basic support for OpenStack, and which one takes a more native approach to OpenStack. There are many implications to the level of complexity and also to the level vulnerability of those services based on the support level.

OpenStack should attract more ecosystem projects rather than competing with them

In my opinion, there has been too much focus developing more alternative projects for the OpenStack project in comparison to the focus that has been placed on projects that exist within the broader ecosystem. The main reason for this, is the lack of native support for OpenStack. A good example of this, is project Solum and its ecosystem equivalents Cloud Foundry and OpenShift. 

I think that it’s time that OpenStack realign its focus from growing all around the stack, to supporting and attracting more ecosystem projects, that focus on adding native support to OpenStack. In order to do this, there needs to be more emphasis on creating certification, compatibility and testing projects. Additionally, the marketplace initiative needs to receive more attention and investment in order to make it more than a web-based directory service, similar to the AWS marketplace.

I would like to propose a discussion on that subject during the next summit in Paris. Do you think this is a good or bad idea? I’d appreciate it if you could share your thoughts.

Mapping the Cloud/PaaS Stack

Cloud is evolving at an incredible pace, changing almost every aspect of our industry. It’s only natural then, that with this change also comes a continuous evolution of how we classify and categorize the various components of the cloud stack.

In earlier days, it was enough to draw the lines between IaaS, PaaS, and SaaS, and while this categorization is still very handy, we are beginning to see the lines between these categories start to blur. On the one hand, we see more IaaS providers starting to move up the stack, and on the other hand, we are seeing new categories such as DevOps gaining popularity and bridging the gaps between the IaaS and the PaaS layers.

 

I was recently asked to draw a map of how I see the current cloud stack. This ended up being an interesting exercise. To make things simpler I decided to focus on the IaaS/PaaS piece, both because this is my area of expertise, and also because IMO this is where many interesting developments are taking place. I ended up with the diagram below, which I thought would be worth sharing in order to get feedback and different perspectives.

It’s important to note, that this diagram is not based on any scientific scoring of different products in the market. I'll leave that part to the analysts. The specific products I chose to use are only there to illustrate and support my analysis of the components in the stack and where they map to within each category.

What's Driving the Cloud Stack?

The two main (and often conflicting) driving forces behind the evolution of the cloud stack are Productivity and Control. As cloud started to emerge it was productivity that took center stage, often by giving up almost completely on the level of control. As the industry started to mature we started to see an evolution where we could have various degrees of tradeoffs between these two driving forces. A good indicator of this, is the rise of DevOps and Private Clouds. We could also see the new open source movement as yet another major step in that direction - the most notable one is OpenStack, and the recent Citrix announcement about the moving of Cloud Stack into the Apache Foundation.

We could also see similar movement happening on the PaaS layer, where VMware launched Cloud Foundry as both a self-service PaaS for development productivity, and as an open source project to provide users with more control. Cloudify takes similar yet more explicit moves in that direction by bringing DevOps and PaaS together, in addition to making it open source.

 

Mapping the PaaS Stack

Geva Perry wrote a good summary mapping the current Java PaaS players in his post Listing the Java PaaS Public and Private Clouds . Geva chose to classify the different players based on their private/public cloud position and language support (Java in this particular case). Obviously the correlation behind public/private cloud and productivity/control is fairly close, so I can see where he was coming from. While this makes a lot of sense, I believe that it’s not accurate enough as there are still solutions that work on both private and public clouds and are not limited to just Java - Cloudify included. So is there a better way to classify the PaaS providers?

Here is my suggestion:

I would classify the PaaS providers based on those who focus on delivering PaaS through a self-service portal I’ll refer to them as PaaS service providers, where they assume full control over the underlying infrastructure and own the entire operation of your application - this is where I would put GAE, Heroku etc. Next will be  those who gives you the tools to build your own PaaS on private or public clouds, I’ll refer to them as PaaS stack providers - this is where Cloudify and I believe, Cloud Foundry and Stackato also fit in.

The second sub-category of classification would be on the level of openness and control i.e. open source, language support, cloud environment support, DevOps support and such.

In any case, we can easily get into a complex matrix of features/mapping for even more fine-grained classification but this far beyond the scope of this post which is merely drawing a rough map rather than providing a complete competitive analysis.

How would you categorize the Cloud/PaaS Stack?

As I noted earlier, this is just my own personal view on the subject. I hope that this will spark an interesting debate in which we may either come out smarter, or possibly even more confused. :)

GigaSpaces Cloudify & VMware Cloud Foundry the new PaaS Jailbreaker

I was reading Krishnan Subramanian's post, Two Events That “Clouded” Our Thinking In 2011. The thing that caught my attention was Krishnan's comments on why PaaS is a superior alternative to DevOps: 

The shift in the thinking about the enterprise cloud consumption also poured water into the “DevOps” concept advocated by vendors and pundits with their foot in the IaaS world. When organizations embrace PaaS instead of infrastructure services, we don’t need the DevOps marriage and the associated cultural change (believe me, this cultural change is giving sleepless nights to many IT managers and some consultants are even making money helping organizations realize this cultural change). With PaaS, organizations can keep the existing distinction between the Ops and Dev teams without worrying about the cultural change. In fact, with cloud computing, the role of the Ops is not going away but it stays in the background offering an interface which developers can manage themselves.

Krishnan represents one of the common attitudes and subjects of debate between two main paradigms for developing and managing applications on the cloud:

• PaaS -- PaaS takes a developer, application-driven approach. A PaaS platform provides generic application containers to run your code. The PaaS platforms deals with all the operational aspects needed to run your code such as deployment, scaling, fail-over, etc.
• DevOps -- DevOps takes a more operations-driven approach. With DevOps, you get tools to automate your operational environment through scripts and recipes, and keep full visability and control over the underlying infrastructure.

The Difference Between PaaS and DevOps

Both PaaS and DevOps aim toward the same goal -- reducing the complexity of managing and deploying applications on the cloud. But they take a fairly different approach to deliver on that promise.

Christoper Knee summarized it in his blog post DevOps and PaaS -- Friend or Foe? as the difference between Developers and SysAdmin:

• Developers may ask: "if I have a self-service portal for deploying applications (aka PaaS), do I need SysAdmins at all?"
• SysAdmins may ask: "isn't PaaS just a monstrous black box that prevents me from provisioning the specific services we need to deploy real-world apps?" ... The typical SysAdmin thinks that they can get to 75% of PaaS functionality with DevOps tools like Chef without giving up any systems architecture flexibility.

In one of my earlier post on the subject, Productivity vs. Control tradeoffs in PaaS, I tried to outline the main limitation of most of the current "blackbox PaaS" implementations:

I thought that Carlos Ble's post Goodbye Google App Engine (GAE) is a good example that illustrates why the initial perception behind GAE as a simple platform that provides extreme productivity can be completely wrong.

...developing on GAE introduced such design complexity that working around it pushes us 5 months behind schedule.

Part of the reason that brought Carlos through that experience IMO is that in the course of trying to make GAE extremely productive the owner made the platform too opinionated, to the point where you lose all the potential productivity gains by trying to adopt their model. In addition, with a platform like GAE you have very little freedom to leverage existing frameworks such as your own database, or messaging system, or any other third-party service that can in itself be a huge contributor to productivity.

Instead, you're completely dependent on the platform provider's stack and pace of development, and that in itself can work against agility and productivity in yet another dimension. In this specific example, Carlos couldn’t use a specific version of a Python library that would have made his productivity higher, and instead had to work around issues that were already solved elsewhere. This is a good example how the lack of flexibility leads to poorer productivity even in the case of simple applications.

Putting DevOps & PaaS togatehr 

It looks like more people in the the industry have come to recognize that rather than looking at DevOps and PaaS as two competing paradigms, it might be best to combine the two, as Christoper Knee pointed out in his post:

 

What if you could get a PaaS that wasn't a black box, enabling developers to deploy apps easily while still giving SysAdmins the ability to provision any services they needed (a la Cloud Foundry)?

I also came to that realization myself, as I pointed out in my  2012 predictions: 

In 2012, we’ll see many of the DevOps tools, such as Chef and Puppet, integrated into application platforms, making it easier to deploy complex applications onto the cloud. In the same way, we’re going to see more Application Platforms adopting the automation and recipe model from the DevOps world into the application platform. The latter have the potential to transform the opinionated PaaS offerings as we know them today, with Heroku and GAE leading that trend, into a more open PaaS offering that better fits the way users develop apps today, and provide more freedom to choose your own stack, cloud, and application blueprint.

What Makes a Cloudify and CloudFoundary a PaaS Jailbreaker?

The CIO Maginze article Cloud computing disrupts the vendor landscape defines a new class of PaaS platforms that I think represent the definition for a DevOps PaaS:

A PaaS that allows developers to use whatever tool they want to build their cloud applications and the platform tackles the deployment, scaling and management of these apps in the cloud data center.

VMware CloudFoundry is one of the more notable references in that category. Quoting Christoper Knee:

CloudFoundry runs anywhere, incuding on your laptop. CloudFoundry's service container concept is particularly strong, kind of an appliance on steroids.

These ideas were the founding concept behind Cloudify, i.e. putting DevPops & PaaS togather in a single framework. As with CloudFoundry, Cloudify enables you to break away from the "blackbox PaaS". However, even though CloudFoundry is significantly more open than most other PaaS alternatives, at the core it is still based on the "my way or the highway" approach (aka opininated architecture), which forces you to fit into a specific blueprint that is mandated by the platform. Cloudify, on other hand, pushes the envelope even further by adopting the concpet of recipes that was first introduced by DevOps frameworks such as Chef and Puppet. It introduces  more application-driven recipes through the introduction of Domain Specific Language that extends upon the groovy language.

The Cloudify recipes give you the full power to plug in any application stack on any cloud (including a non-virtualized environment) and manage them in similar way to the way you would manage them in your own datacenter or machines. You can also call Chef and Puppet from within the recipes. All this, without hacking the framework itself. As with other similar DSLs, the Cloudify DSL was designed to express even the most complex application management tasks, such as <recovery from a data center failure> in a single line and avoid all the verbose script and API calls that are often involved when you work at the infrastructure level.

All this makes Cloudify an even more open alternative that fits a large variety of the current enterprise application stacks, including:

1. JEE applications
2. Big Data applications
3. Multi-tier applications
4. Native applications (C++,..)
5. .Net, Ruby, Python, PHP applications
6. Multi-site applications
7. Low-latency applications (that can't run on VMs)

It also makes Cloudify more open for special customization of the existing stack, like in the case of:

1. An application that needs a certain version of MySQL (not the one that comes with the framework)
2. An application that needs to run on Redhat (not Ubuntu). or even more interesting -- a case where there are mutiple applications, each needing a different OS served at the same time.

Cloudify also provides a more advanced level of control geared for mission-critical apps, including:

1. Monitoring the application stack and topology
2. Adding custom application metrics
3. Adding custom SLAs

It can also work on wide variety of cloud environments, including Microsoft Azure and non-virtual enviroments.

One of the great powers of the recipe is that it is a great collaboration tool. Once you develop a recipe, it is very easy to share it wtih different groups -- whether internal groups like developement, QA, and operations, where the recipie provides a programmatic definition of thier environment, or it can be shared between the product team and the pro-services team, where your sales and pro-services can easily install and update product versions in a consistent way, as well as reproduce customer scenarios and share them with the support team. Recipes are also a great tool for collaboration through a wider community network, where users can collaborate by sharing common recipes and best practices over the web.

Quick Introduction to Cloudify Recipes

Below is a short snippet that shows a simple application recipe, of a typical java-based web application based on JBoss as the web container and MySQL as a database. The application recipe describes the services that comprise the application, and thier dependencies. The details on how to run MySQL and JBoss is provided in a seperate recipe for each of the individual services. A more detailed description of how a service recipe would look like can be seen here. 

application {
       name="simple app"

       service {
              name = "mysql-service"
      }
       service {

              name = "jboss-service"
              dependsOn = ["mysql-service"]
       }
}

To get a taste, you can try one of the available recepes such as Cassandra, MongoDB, Tomcat, JBoss, Solar, etc., just as a simple way to try out these products on your own desktop or on any of the supported clouds, without the hassle that is often involved in doing so and without even direct relation to Cloudify per-se.

 

References

• Productivity vs. Control tradeoffs in PaaS
• DevOps and PaaS - Friend or Foe?
• PaaS shouldn’t be built in Silos
• Two Events That “Clouded” Our Thinking In 2011
• Cloud computing disrupts the vendor landscape
• Cloudify docs

Multi-tenancy: emulation or the real thing? - My Take

Phil Wainewright wrote and interesting take on Zdnet (Multi-tenancy: emulation or the real thing?) where he outlines different multi-tenancy levels:

• Hosting single-tenant applications on multi-tenant infrastructure
• Redistributing application processes across virtualized machines
• Injecting a user-specific column in the database driver

Phil also addressed two approaches to achieve multi-tenancy, to which he referred as emulation - multi-tenancy that is pre-backed into the infrastructure as in the case of GigaSpaces, or the "real thing," which stands for a custom build where multi-tenancy is handled at the application level as in the case of Salesforce.

In his final takeaway, Phil suggested that to get the most out of multi-tenancy, you have to deal with it at the application level.

The build vs buy dilemma

Phil's point reminded me of the classic build vs buy dilemma – at first sight you would think that a custom build that is tailored to your specific domain and application would always be better than a generic solution. But is it?

I addressed "build vs buy" in a previous post on the subject (The impact of cloud computing on build vs. buy behavior), using Fred Brooks's article No Silver Bullet. There are many lessons that Fred points out in his article that are worth reading, but here's a relevant point I'd like to highlight:   

The hardest single part of building a software system is deciding precisely what to build.

Fred suggests that we tend to justify custom builds without properly realizing the associated costs. In many cases, what we tend to refer to as “specific requirements” aren’t that specific after all - or to be more accurate, the differences don’t justify the costs.

What to choose?

In the context of multitenancy, we should consider the following things:

• Does the difference justify the cost?

How different would Salesforce multi-tenancy be compared to the multitenancy of any other SaaS or PaaS provider?

I would argue that most multitenancy providers are the same. Differences are minor; the goal is multitenancy, not a mix-and-match of potential features.

• Complexity consideration

Dealing with data multi-tenancy at the application level can be error prone and can be significantly more complex than dealing with it at the infrastructure level, with serious effects. For example, I heard from one SaaS provider that a bug in their system led to one customer accidentally getting access to another customer's data. In addition to the complexity, there are things that just can’t be dealt with properly at the application level – for example, in the case of Salesforce, a single tenant is bound to a single database instance and can’t scale beyond that single database.

• Time to Market consideration

When you launch a new SaaS or PaaS application, time to market often plays a critical role. Dealing with multi-tenancy at the application level requires lots of effort which can lead to a significant delay in time to market.

Joseph Ottinger points out that this effort can be addressed in lots of ways - perhaps by programming language choice, for example. But even these solutions carry costs, in terms of your team's capabilities or even in scalability. The language you choose, for example, might be able to help your time to market, but might cost you in performance, or even address how your architecture is built. These choices can be made well, but you should go in with eyes open to what choices you're making.

• Cost consideration

When were dealing with multitenancy at the application level, we're often limited to cross-grain multi-tenancy as we don’t have full control over the underlying infrastructure.  For example, in the case of data-multi-tenancy we can easily scale out and and scale down our data services based on the actual capacity that is being used – doing that at the application level would be much harder, maybe close to impossible. That means that we can achieve better density and thus control the number of resources and thus the cost at each given point in time by using platform multitenancy.

It’s not a mutually exclusive decision

In the context of multi-tenancy, there are always going to be things better dealt with at the infrastructure level such as data-multi-tenancy – and application specific multi-tenancy such as the user interface layout.

By delegating the generic part of the challenges of multi-tenancy to the infrastructure layer we can devote more time and effort at areas more specific to our business. 

The full story in 15 min

Phil's article triggered a more in-depth discussion with Uri Cohen, GigaSpaces product manager, which later led to a recorded interview (15 Min on Multi-Tenancy) in which Uri address the following topics:

• Multi-tenancy: emulation or the real thing? the classic build vs buy dilemma
• What is multitenancy anyway?
• Why is it such a popular topic?
• Is it a feature or best practice?
• What is GigaSpaces Multi-tenancy?

Enjoy..

 

References:

• Multi-tenancy: emulation or the real thing?
• The impact of cloud computing on build vs. buy behavior
• Multi-tenancy: does it have to be that hard?

GigaSpaces New Cloud Platform - Sneak Preview

The presentation below provides a sneak preview into one of the cool features of our upcomming Cloud Enabled Application Platform - Uniersal Service Manager (USM).

The universal service manager enable to handle the deployment, elasticity, continues avliability, scalabiltiy of existing application on any cloud or local data center. In this specific presentation were demonstrating how we can deploy NoSQL Service - Cassandra, using the USM , scale-it, monitor it, handle failure scenario etc. All that without writing any line of code and without any need for prior knowlage of GigaSpaes!

All you need to do is point to your cassandra directory, write few shell scripts for handling the pre-post deployment and the rest is taken care of by the platform.

Were showing a live demo of that feature on Cloud Connect, so if your in the area stop by our CloudConnect booth (106).  

For those interested in trying it out themselfs were going to make the first milestone avliable for public download by the end of this month.

You can also send an email to (pm at gigaspaces dot com) and register for updates on this new release!

GigaSpaces new cloud platfom - sneak preview I

View more presentations or Upload your own.

PaaS shouldn’t be built in Silos

Last year I wrote a post titled Platform as a Service: The Next Generation Application Server?  where I was referring to the shift from  application server into platforms.

On my 2011 predictions  I referred to a new emerging PaaS category targeted for those who be looking to build their own custom PaaS offering.

The demand coming from many SaaS providers to offer their own platform to their ecosystem players (a good example on that regard is Salesforce force.com) as well as the continues pressure of enterprises on their IT to enable faster deployment of new application and improve the utilization of their existing resources will force those organizations to build their own specialized PaaS offering. This will drive a new category of PaaS platform known as Cloud Enabled Application Platform (CEAP) specifically designed to handle multi-tenancy, scalability, and on-demand provisioning but unlike some of the public PaaS it needs to come with significantly higher degree of flexibility and control.

One of the immediate question that came up in various follow-up  discussion was  “how is that different from the Application Servers of 2000?”

I’ll try to answer that question in this post.

The difference between PaaS and Application Servers 

Lori MacVittie wrote an interesting post, "Is PaaS Just Outsourced Application Server Platforms?," which sparked an interesting set of response on twitter.

Lori summarized the aggregated response to this questions as follows:

PaaS and application server platforms are not the same thing, you technological luddite. PaaS is the cloud and scalable while application server platforms are just years of technology pretending to be more that it is..

Now let me try to expand on that:

The founding concept behind application servers was built around the concept of Three Tier architecture where you break your application into presentation, business-logic and data-tier. It was primarily an evolution of Client/Server architecture, which consists of a rich client connected to centralized database.

The main drive was the internet boom that allowed million of users to connect to the same database. Obviously having million of users connected directly to a database through a rich client couldn’t work. The three-tier approach addressed this challenge by breaking the presentation tier into two parts – a Web Browser and Web Server - and by separating the presentation business logic tier into two separate parts.

In this model, Clients are not connected directly into the database except through a remote server.

Why is PaaS any different?

A good analogy to explain the fundamental difference between existing applications servers and PaaS enablement platform is the comparison between Google Docs vs Microsoft office. Google came up with a strong collaboration experience that made sharing of documents simple. I was willing to sacrifice many of the other features that Office provided just because of the ease of sharing.

The same goes with PaaS – when I think of PaaS I think of built-in scaling, elasticity, efficiency (mostly through Multi Tenancy) and ease of deployment as the first priority and I’m willing to trade many other features just for that.

In other words, when we switched from desktop office to its Google/SaaS version we also changed our expectations from that same service. It wasn’t like we took the same thing that we had in our desktop and put it over the web – we expected something different,.  This brings me to the main topic behind this post:

PaaS shouldn’t be built in Silos

If I take the founding concept of Application Server – i.e. Tier based approach - and try to  stretch it into the PaaS world the things that pops up immediately in my mind is the impedance mismatch between the two concepts.

Tiers are Silos – PaaS is about sharing resources.

Tiers were built for fairly static environment – PaaS is elastic.

Tiers are complex to setup – In PaaS everything is completely automated. Or should be.

Now this is an important observation – even if the individual pieces in your Tier based platform is scalable and easy to deploy the thing that matters is how well they can:

1. Scale together as one piece,
2. Dance in an elastic environment as one piece.
3. Run on the same shared environment without interfering with one another... as one piece through multi-tenancy.

Silos are everywhere ..

In the application server world, every tier in our infrastructure is built as a complete silo. By "silo," I mean that they are not just built separately, but that each component comes with its own deployment setup, scaling solution, high availability solution, performance tuning and the list goes on...

All that individual deployment and tuning in the context of a single application, well, if you stretch that into multi-tenant application, it becomes much worse.

Silos are not just the components that make our products. Silos are also a feature of those who build the components.

If you think of Oracle Fusion as an example or IBM for that matter you’ll see silos. Messaging, data, analytics, all broken into a separate development group with different management, individual development groups.

Each component has a different set of goals, different timelines, different egos, and in most cases they are even incentivized to keep those silos in place simply since they are not measured on how well their products work with other pieces but solely on how well their thing does its (siloed) job which leads into even greater focus on silos - functionality that works as an end to itself and not a contributor to general application development.

Oracle Fusion is an interesting attempt to break those silos. In the course of creating Fusion, they even came up with an interesting announcements of bringing Hardware and Software together.

Is "bringing hardware and software together" good enough? It's certainly an attempt but if you look closely you’ll see that most of it is a thin glue that tries to put together pieces that were bought separately through acquisitions and are now packaged together and labeled as one thing.

However most of it is still pretty much the same set of siloed products. (Oracle RAC cluster is still very different than Weblogic cluster, etc.) That doesn’t apply to Oracle only – think of VMware and its recent acquisition now labeled under VFabric.

Now imagine how you can scale together an erlang based messaging system with other pieces of their stack as one piece.. The sad thing about VMware is that it looks like they went and built a new platform targeted specifically for PaaS world but inherited the same pitfalls from the old Tier-based application server world.

Silos are also involved in the process by which we measure new technology.

Time and time again I see where even those who need to pick or build a PaaS stack go through the process of scoring different pieces in the puzzle independently, with different people measuring each piece rather than looking on how much those pieces work together.

A good analogy is comparing Apple with Microsoft: Microsoft would have probably score better on feature by feature comparison (they can put a checkmark in more places) but the thing that brings many people to Apple is how well all of its pieces work together.

Apple even went farther and refused to put in features that might have been great - just because they didn’t fit the overall user experience. They often win with users because of the scarcity of features, even though it may not look as nice on a bullet list of features.

Sharing is everything.

The thing that makes collaboration so easy in the case of Google Docs is the fact that we all share the same application and underlying infrastructure. No more moving parts. No more incompatible parts.

All previous attempts to synchronize two separate documents through versioning highlighting and many other synchronization tools pale compared to the experience that we get with Google Docs. The reason is that sharing and collaboration in Office was an afterthought. If we work in silos, we’ll end up… in silos. Synchronizing two silos to make them look as one is doomed to fail, even if we use things like revision control systems for our documents.

We learned through the Google docs analogy that sharing of the same application and its underlying infrastructure was a critical enabler to make collaboration simple.

There is no reason that that this lesson wouldn’t apply to our Application Platform as well. Imagine how simple our lives could be if our entire stack could use one common clustering mechanism for managing its scaling and high availability. All of a sudden the amount of moving parts in our application would go down significantly, along with lots of complexity and synchronisation overhead.

We can measure the benefits of having a shared cluster based on the fact that it comes with less moving parts compared with those that are based on Tier based model:

• Agility – fewer moving parts means that we can produce more features more quickly without worrying if once piece will break another piece.
• Efficiency – fewer moving parts means that will have less synchronization overhead and network calls associated with the each user transaction.
• Reliability – fewer moving parts means lower chance for partial failure and more predictability in the system behavior.
• Scaling –  fewer moving parts means that scaling is going to be significantly simpler, because scaling would be a factor of a "part" designed for that purpose.
• Cost – fewer moving parts means fewer products, a smaller footprint, less hardware, reduced maintenance cost, even less brainpower dedicated to the application.

First Generation PaaS – Shared platform

If we look at the first generation PaaS platforms such as Heroku, Engine Yard, etc., we see that they did what Oracle Fusion does. They provide a shared platform and carve out a lot of the deployment complexity associated with the deployment of those multi-tiered applications. (This makes is marginally better than Oracle Fusion, but not good enough... certainly not as good as it could or should be.)

I refer to these as “PaaS wrappers”. The approach that most of those early-stage PaaS providers took was to “wrap” the existing tier-based frameworks into a nice and simple package without changing the underlying infrastructure, to provide an extremely simple user experience.

This model works well if you are in the business of creating lots of simple and small applications but it fails to fit the needs of the more “real” applications that need scalability, elasticity and so forth. The main reason it was built this way was that it had to rely on an underlying infrastructure that was designed with the static tier-based approach in mind. An interesting testimonial on some of the current PaaS limitations can be found in Carlos Ble's post, "Goodbye Google App Engine (GAE)."

Toward 2nd Generation PaaS

In 2010, a new class of middleware infrastructure emerged, aimed at solving the impedance mismatch at the infrastructure layer. They were built primarily for a new, elastic world, with NoSQL as the new elastic data-tier, Hadoop for Big data analytics, and the emergence of a new category (DevOps) aimed at simplifying and automating the deployment and provisioning experience.

With all this development, building a PaaS platform in 2011 will - or should - look quite different than those that were built in 2009.

The second-generation PaaS systems would be those who could address real application load and provide full elasticity within the context of a single application.

They need to be designed for continuous deployment from the get-go and allow shorter cycles between development and production.

They should allow changes to the data model and introduction of new features without exposing any downtime.

The second-generation PaaS needs to include built-in support for multi-tenancy but in a way that will enable us to control the various degrees of tradeoffs between Sharing (utilization and cost) and Isolation (Security).

The second-generation PaaS should come with built-in and open DevOps tools that will enable us to gain full automation but without losing control, open enough for customization and allowing us to choose our own OSes and hardware of choice as well as allowing us to install our own set of tools and services, not limited to the services that come out of the box with the platform.

It should also come with fine-grained monitoring that will enable us to track down, trouble-shoot  our system behavior through a single point of access as if the platform was a single machine.

The 2nd generation PaaS systems are not going to rely on virtualization to deliver elasticity and scaling but instead provide a fine grain level control to enable more efficient scaling.

At Structure 2010, Maritz said that “clouds at the infrastructure layer are the new hardware.” The unit of cloud scaling today is the virtual server. When you go to Amazon’s EC2 you buy capacity by the virtual server instance hour. This will change in the next phase of the evolution of cloud computing. We are already starting to see the early signs of this transformation with Google App Engine, which has automatic scaling built in, and Heroku with its notion of dynos and workers as the units of scalability.

Final words

Unlike many of the existing Platforms, in this second-generation phase, its not going to be enough to package and bundle different individual middleware services and products (Web Containers, Messaging, Data, Monitoring, Automation and Control, Provisioning) and brand them under the same name to make them look as one. (Fusion? Fabric? A rose is a rose by any other name - and in this case, it's not a rose.)

The second-generation PaaS needs to come with a holistic approach that couples all those things together and provide a complete holistic experience. By that I mean that if I add a machine into cluster, I need to see that as an increase in capacity on my entire application stack, the monitoring system needs to discover that new machine and start monitoring it without any configuration setup, the load-balancer need to add it to its pool and so forth.

Our challenge as technologists would be to move from our current siloed comfort zone. That applies not just to the way we design our application architecture but to the way we build our development teams, and the way we evaluate new technologies. Those who are going to be successful are those who are going to design and measure how well all their technology pieces work together before anything else, and who look at a solution without reverence for past designs.

References

• Is PaaS Just Outsourced Application Server Platforms?
• PaaS choices today – ZDnet
• Platform as a Service: The Next Generation Application Server?
• VMware Knows the Cloud Doesn’t Need Server Virtualization
• Extreme and extensible applications Gigaspaces XAP andVMware vFabric GemFire –ZDnet
• Goodbye Google App Engine (GAE) 

Written by Nati Shalom and Joseph Ottinger

2011 Cloud, PaaS, NoSQL Predictions

2010 was a turning point year in the IT landscape – specifically in comparison to the downturn of 2009. Lots of major acquisitions changed the IT vendor landscape beyond recognition, the most notable ones in the Java world are Sun acquisition by Oracle and to a lesser degree SpringSource acquisition by VMWare. Cloud computing has moved tremendously fast through the hype cycle to become a true viable reality with the recent addition Mr. Larry Elison who made a 180-degree turn to acknowledge Oracle wished to be a cloud vendor too.

I was recently asked to give my prediction to 2011 to one the the tech magazine. Even though I’m not a big fan of those predictions, but it turns out to be an interesting exercise. From obvious reasons i chosen to focus the predictions on areas that are close to our day to day life in GigaSpaces such as  Cloud computing, PaaS, SaaS enablement, NoSQL, Tera Scale  etc. enjoy..

2011 Predictions

Cloud prediction

Hybrid is the Way to Go: The Public-Private Cloud Discussion Isn’t Relevant Anymore

In 2010, IT organizations realized that the cloud wasn’t just a buzzword for the latest and greatest IT trend but rather the future of an organization’s IT infrastructure. The discussion centered on whether a public or private cloud infrastructure would be the best format. The issues of bottom line versus considerations for SLAs, confidentiality, security, latency, responsiveness, regulation, and availability provided pros and cons in each direction.

In 2011, the debate will become moot as the adoption of the hybrid cloud will become more widespread. Organizations have certain processing and data requirements for which they must maintain complete control – the internal, data-center based private cloud – and others where control is not an issue – leveraging the resources of the public cloud. The hybrid allows for both. 

Openstack will dominate the open IaaS offering

Openstack is gaining momentum quickly and has a great potential to become the next “Linux” of the cloud (Think of VMware as Windows in that analogy). Unlike other similar projects like Eucalyptus, Openstack is backed by a relatively large organization such as Rackspace, NASA, Citrix, all planning to “eat their own dog food” and use the platform in anger to run their own business. Many smaller IaaS platform providers will either join the OpenStack initiative or seize to exist.

PaaS predictions

2011 is the Year of PaaS

Platform as a Service is the largest growing business opportunity on the cloud. By creating a flexible infrastructure in a multitenant environment, PaaS companies will be able to offer a wide range of services to start-ups and enterprise organizations who want to outsource or “hybridize” their IT operations. In addition, existing Software as a Service providers will leverage their own infrastructures of hardware, software, applications, and processing to outsource the same services to other organizations.

A new PaaS category will emerge – Building your own PaaS

The demand coming from many SaaS providers to offer their own platform to their ecosystem players (a good example on that regard is Salesforce force.com) as well as the continues pressure of enterprises on their IT to enable faster deployment of new application and improve the utilization of their existing resources will force those organizations to build their own specialized PaaS offering. This will drive a new category of PaaS platform known as Cloud Enabled Application Platform (CEAP)  specifically designed to handle multi-tenancy, scalability, and on-demand provisioning but unlike some of the public PaaS it needs to come with significantly higher degree of flexibility and control.

Application Servers will Change Their Name to PaaS – But Won’t Change Their Stripes

Many of the application servers available on the market support a limited scalability and work on a case-by-case basis without necessary scalability and the flexibility required for a multitenant environment. A limited number of application server companies, by the nature of their integration partners, will find the market wide open to step up when “traditional” application servers fail. 

VMForce Will Fail to deliver on its promise

Salesforce already signaled through a series of platforms acquisitions that its not going to rely on VMware and SpringSource to deliver its core piece of infrastructure.  The future of VMForce is therefore remains vague. Interestingly enough since the announcement in April this year VMForce remains in pre-beta stage. IMO this is more to do with the inherit conflict between the two companies. Both wants to become dominant player in the emerging PaaS market. Salesforce cannot afford to be marginalized by VMware who wants to see itself in the center and Salsefroce as one of the many providers on its own stack. With the recent acquisitions Salseforce has all it needs to build VMForce without any reliance on VMware and that’s IMO the only way VMForce could become successful.

NoSQL (+Big Data) predictions

NoSQL will become compatible with SQL

Many of the NoSQL implementations gained momentum as a scalable alternative to relational databases. Most of those NoSQL implementations started as a simple Key/Value store. In 2011 will see  a big push toward SQL support and more standard interfaces and at the same time we’ll see traditional databases adding support for dynamic data structure and schemaless API.

More applications will run entirely In-Memory

The emergence of the new Tera-Scale machines in 2010 will enable to store Terra bytes of data in-memory on a single box. In 2011 well see more applications that will run their entire data in-memory.

Real-time Analytics Big Will Replace majority of the Map/Reduce Batch Processing

Real-time/stream-based analytics has to become the norm. No one has time for batch processing anymore. For example, e-commerce customers want to know what their friends think about products or services – right in the middle of their shopping experience. If sites cannot keep up with their thousands of users in real-time, they will lose their customers. Yahoo S4 , and Google Percolator is a strong indication on that movement..

References

• 2011 Cloud Computing Predictions For CIO’s And BusinessTechnology Leader
• 10 Cloud Predictions for 2011 from Forrester's James Staten
• Enterprises Consider Business Agility As A PrimaryDriver Towards Clou
• A Private Party
• Top 10 Cloud Stories Of 2010
• The TeraScale Effect
• Google Percolator – global search jolt sans MapReduce comedown
• Yahoo’s Open Sourced S4 Could be a Real-time Cloud Platform

Multi-tenancy: does it have to be that hard?

Multi-tenancy is a term that is used often in the context of SaaS applications. In general, multi-tenancy refers to the ability to run multiple users of an application on a shared infrastructure. The main motivation for doing this is efficiency, or in other words -- reducing the cost per user in comparison to a dedicated system where each user has their own dedicated environment.

Over the past few months I've had discussions with various SaaS providers, in which I tried to learn about the main challenges that SaaS providers face when they deal with multi-tenancy. As it happens, I came across an interesting article on this very topic, Why Multi-tenancy Matters in the Cloud by Alok Misra. The article later sparked an interesting discussion thread on the cloud mailing list, which I also find to be a good reference for diverse thoughts on this subject.

In this post I want to summarize some of my findings on multi-tenancy, as well as suggest a model that can make multi-tenancy significantly simpler.

The common challenge: Multi-tenancy at the data layer

There are many approaches and levels of multi-tenancy, depending on the application layer and type of application you're dealing with. For example, trying to separate user accounts in a CRM application is quite different than separating various batch processing jobs in a risk management application. That said, what seems a common challenge in almost every SaaS application is multi-tenancy at the data layer. To put it simply, the challenge is how to share data resources (database in most cases) among multiple users, while at the same time ensuring data isolation between those users, as if they are running on completely physically seperated servers.

In the remaining part of this post I'll address how multi-tenancy is currently being dealt with at the data layer, and how the existing challenges can be met.

How is multi-tenancy being done today?

For this part of the story, I think it might be useful to first look at the two extreme approaches: The fine-grained approach and the cross-grained approach.

Fine-grain multi-tenancy (Salesforce.com model)

For many SaaS providers the “poster child” of multi-tenancy is Salesforce.com. There is a very useful videocast explaining the Salesforce approach to multi-tenancy. In a nutshell, their approach is to share the same database instance between different users, and define the data model so that each table has a primary key identifying the customer ID. In addition, there is a separate metadata table that translates every query into a user-specific query before the query hits the database.

• Pros:
  • Very fine-grained multi-tenancy resulting in optimum cost efficiency per user.
• Cons
  • Complexity: Requires changes in the data model.
  • Doesn’t fit to existing application: Existing applications require a complete rewrite.
  • Modest degree of isolation: Users sharing the same resource can impact one another under heavy load situations. A maintenance/patch release required for a particular user might affect other users. In a more extreme scenario, there is also exposure to a higher degree of vulnerability. For example, a bug in the query mapping could point one user to data belonging to other users just by picking up the wrong customer ID.

Cross-grain multi-tenancy (hosted service model)

Most hosting providers (including hosted IT providers) enable outsourcing of certain IT applications. Efficiency is gained by sharing maintenance overhead, labor costs, and other data center costs such as network, electricity, and more.

• Pros:
  • No code changes are required, resulting in greater simplicity.
  • High degree of isolation.
• Cons
  • Lower efficiency and therefore limited cost margins.

Challenges with today's approach to multi-tenancy

The challenge with the fine-grained model is that it is fairly complex to maintain and implement. If you have an existing application, it requires a complete rewrite and also forces fairly significant changes in your existing data model. The challenge with the cross-grained model is that cost margins per user are still rather high. In a competitive environment, even a single competitor offering similar servicse to yours using the fine-grained approach could easily put you out of business.

The elasticity challenge

The other challenge that neither approaches addresses today is dynamic elasticity. What happens when a customer grows beyond their allocated shared capacity? Both approaches seem to rely on certain capacity assumptions, and can scale by splitting multiple users between their shared resources. However, none of the approaches seem to handle a situation where a particular user's capacity grows beyond the allocated resources.

Jayarama Shenoy provides a good introduction to that challenge in his comment in the discussion thread:

So, how does elasticity work in a multi-tenancy case? Say 100 tenants share an app on a server (by which I would think they go all the way down to a shared database). Then,say, five of them grow quite rapidly such that they need to be moved off that server, and the other 95 can continue to exist on the original server.
So 'something' in the application has to know how to pick off the records that belong to these five, copy it off into a second instance of that database and kill, move & restore application service for those tenants. (Of course, without being privy to the data contained in those records). And of course there's an implied load balancer element in front of all this to keep up appearances to the clients.

Based on this analysis I believe that the main challenge with the current approaches is that we have too few options available. No option seems to provide a good solution to the elasticity challenge, and this is what is making multi-tenancy “so damn hard”. This is especially true for existing businesses with existing products.

The solution: Making multitenancy simple

Learning from the past operating system experience

Some of us probably remember the days when Bill Gates came up with DOS, which was basically a single-user system. At the time, if you wanted to provide a multi-user application you had to write it yourself, and sometimes that required going down to the core interrupt level and program in assembly language. This made multi-user systems on DOS very uncommon. Unix, on the hand, was built for multitasking, multi-user environments, and therefore it was quite common to build multi-user systems in the Unix environment. Later on, when Windows NT came out, all that changed. Today we can’t even imagine a scenario where we would build an application that wouldn’t support multiple users. 

Lesson learned:

As long as the underlying infrastructure (the operating system in the example, and the database in our specific discussion) is not built for multi-tenancy, trying to solve the multi-tenancy challenge at the application level is doomed to be as complex as writing a multi-user system on DOS. 

In fact, once multi-tenancy was dealt with properly at the operating system level, writing a multi-tenant application became significantly simpler and therefore widely adopted. The same should apply to databases as well.

The database is the problem!

Almost all of the attempts I've seen in both the Salesforce approach and the hosted environment approach were built under the assumption that the database is a fixed “concrete” construct that cannnot be broken down easily into “smaller bricks”, let alone scale dynamically. It is therefore not surprising that most of the attempts to deal with multi-tenancy have tried to address the challenge outside the database.

Now, imagine what your approach to the challenge would be if a database could be easily broken down into small “bricks” without huge overhead, and if it could scale dynamically?

Multi-tenant data service

Similar to the experience we had with operating system, I believe that the solution for multi-tenancy has to be dealt with at the infrastructure level. In our specific discussion, that would be the database. In this section I’d like to draw the principles for a multi-tenant data service:

• Simplicity: With a multi-tenant database, applications can be written to a single database tenant just as they are today. How tenants are allocated and shared with other tenant needs would be completely abstracted from the application.
• Efficiency: Efficiency is maintained by the fact that all data tenants can share the same hardware (memory, CPU) with other tenants.
• Dynamic scaling: Each database tenant can be distributed and scale dynamically across multiple machines, if necessary, to meet demand. This includes moving data to other machines if necessary. Scaling must be supported seamlessly without any downtime.
• Support for multiple isolation/sharing levels: With a multi-tenant database, the level of sharing/isolation is set on demand, based on the specific requirements. At that level, a user can choose a dedicated tenant, in which case the multi-tenant database would allocate tenants on a set of machines that is dedicated to that specific user. Clearly, the benefit of this is isolation, at the cost of efficiency. Another type of sharing is available to the user at the same time: Choosing to share with any individual or group under the user's control. In other words, rather then dictating either extreme (full isolation or full sharing) for all users, with a multi-tenant database we can dynamically choose the right level that best fits our purpose.

Where we are today?

Cloud providers such as Amazon, Microsoft, and Google have already realized that they can’t expect their users to deal with multi-tenancy themselves. So they've come up with data services that were designed with multi-tenancy in mind. Amazon recently introduced Amazon RDS (Relational Database as a Service) and Microsoft introduced SQL Azure. Both are an attempt to take existing databases (MySQL in the case of Amazon and SQL Server in the case of Microsoft) and add some of the characteristics that I discussed above. For example In both cases the user works with a single tenant and is completely abstracted from the underlying resource sharing optimizations. 

And yet, both solutions are still fairy limited when it comes to providing elasticity: With the Amazon solution, you're expected to scale-up (use a bigger machine) and go through a period of downtime when you switch between one degree of scaling and another. Google AppEngine has taken a different approach. Instead of relying on an existing RDBMS and wrapping it up with multi-tenancy and dynamic scaling, they took their own NoSQL alternative (BigTable) that was already built for dynamic scaling and, through a mapping layer, provided SQL semantics on top of it. As with RDS and SQL Azure, the user is completely abstracted from the way Google manages multi-tenancy in their underlying infrastructure.

Niether Amazon, Microsoft, nor Google provide WRT flexibility to the isolation level, and more importantly, all these solutions seem to be completely tied into their own cloud data center, which means that I cannot port it to my own local environment. From a throughput efficiency perspective, SQL Azure and Google seem to trade scaling for throughput by introducing an additional layer of indirection (load balancer in the case of Azure) that makes their multi-tenant solution fit only low-end applications.

At GigaSpaces we took an approach very similar to the Google approach with our new Elastic Middleware Services, where we use our existing distributed NoSQL In-Memory Data Grid and used it as a muti-tenant data service.

That is it for similarity. Here are the differences:

• You can easily download and install GigaSpaces in your local environment or use it in the cloud and get the full multi-tenancy and dynamic scaling experience. We provide an open SPI that enables you to plug in any resource pool, such as starting from a simple SSH call through a complete virtualization pool.
• You can control the level of isolation, scaling, and other SLA directly through an API.
• Our data services are designed for extreme efficiency by maximizing the utilization of memory resources. This normally translates into the ability to run at least 10x more user transactions on the same hardware, which in turn translates into huge cost savings.
• We provide the same consistent approach for other layers of the stack such as messaging and map/reduce through the same environment.

You can learn more about the Elastic Middleware solution in my previous post.

References

• SaaS Enablement of existing applications (Videocast)

• The Missing Piece in the Virtualization Stack (Part 2)

• Why Multitenancy Matters In The Cloud by Alok Misra

• Multi Tenancy - Keep It Simple, REALLY Simple by Ayende Rahien
• Multitenancy definition – Wikipedia
• Elastic Data On the Cloud: Hype or Reality 
• GigaSpaces 7.1 Release
• Multitenant Magic - Under the Covers of the Force.com Data videocast by Craig Weisseman Chief Software Architect salesforce.com
• Security and multitenancy with GigaSpaces XAP (docs)
• Security and multitenancy with GigaSpaces XAP (videocast)

The Missing Piece in the Virtualization Stack (Part 2)

In the first part of this post, I discussed how virtualization and cloud computing, as we know it today, is only a small part of the solution for today’s IT inefficiencies. While new technologies and delivery models have made it much simpler to manage the infrastructure, this is not where our core inefficiencies lie. Virtualization principles must be extended to higher levels of the application stack, to make it easier for all of us to manage, tune and integrate applications. Otherwise we will continue to spend most of our time on things that don’t provide real value to the business – infrastructure, installation, management, tuning, and integration.

In this post, as promised, I’ll show how this missing piece can be filled in practice, by something called Elastic Middleware Services. These are middleware services that can be deployed with one API call, and are completely abstracted from the applications that use them. Elastic Middleware Services are very similar to cloud-based middleware like Amazon SimpleDB, but the major difference is that they are enterprise-grade, and can be purchased and installed in your private data center, for use by your internal enterprise applications.

To better illustrate this idea, I’ll discuss a reference implementation of Elastic Middleware Services which will shortly be released as part of GigaSpaces XAP 7.1. The discussion is intended to illustrate in concrete terms how to fill the missing piece in the virtualization stack. I’ve intentionally focused on the conceptual and fundamental elements of the GigaSpaces implementation, not on specific features and benefits, to make the discussion useful even if you are not using or considering GigaSpaces products.

By the way, I recently hosted a webinar on this topic, together with Uri Cohen, GigaSpaces’ product manager. In the webinar we presented GigaSpaces’ upcoming Elastic Middleware Services. The recorded version of this presentation is available below:

Elastic Middleware Services

The idea is to provide middleware services (messaging, data, Map/Reduce, and more), like those the ones provided by cloud providers such as Amazon and Microsoft Azure, in your own local environment. These services simply extend the basic principles of virtualization, as I outlined in the previous post, to the middleware stack:

Basic virtualization principles

1. Break big physical resources into smaller logical units
2. Decouple the application from the physical resources
3. Provide an abstraction that makes all the small units look like one big unit

From a middleware perspective, this means that instead of having one big database, you have lots of small units of data services (also called partitions or shards), which are grouped together through a client side proxy that exposes all of these small units as one big unit to the client that is using them. The client proxy is also responsible for abstracting the client from the physical location of those resources. To scale, all you need to do is add more of those units. The client experiences these additional units as extra capacity added to the service.

Deploying middleware with one API call

From an API perspective, this is a much simpler approach than traditional middleware, because it introduces a higher-level abstraction to the user, as can be seen in the code snippet below:

With this new elastic middleware API, users define requirements in a language that is closer to their domain. For example, “I want to create a data service that will have 10GB of data and can grow up to 100GB of data, I need hot failover and I'm willing to share my deployment with other users of my own organization but not with users of other organizations.”

In response, the Elastic Middleware Services would automatically do the following:

• Allocate the right number of partitions, based on machine availability and the available memory in each machine, to address the capacity required.
• Automatically launch a hot backup in for each partition and ensure that it runs on separate machine than the primary partition, to address the high-availability requirement. 
• Make sure that if another organization’s application is already running on the machine, the application won’t deploy on that machine, to address the security requirement.

Fully automated through built-in SLA

As I noted in my previous post, one of the major costs of enterprise applications over a period of three years is the cost of maintenance and operations. The main component of maintenance costs is the labor cost, closely tied to the amount of manual work you have to put into your system to make sure that it meets the application’s SLA. The cost grows proportionally to the demand for scale.

With Elastic Middleware, maintenance cost can be substantially reduced through built-in SLA and automation that covers the following aspects:

• Scaling SLA – scale when there is a memory shortage, high CPU utilization, Garbage Collector hiccups, etc. Users can choose to scale automatically when any of these events occurs, and customize their own thresholds (e.g. scale when CPU utilization hits 90%). The Elastic Middleware is integrated with a cloud or virtualization framework, to enable it to automatically pull or release of machines as needed.
• Failover – when a machine fails, the Elastic Middleware does one of two things:
  • Scale down – if there are available resources to meet the current workload, the system will automatically scale down and continue to service the application.
  • Rebalance – if there are not enough resources to serve the application, the Elastic Middleware calls the cloud/virtualization pool and starts a new machine. As soon as the new machine starts, it is added to the existing pool of resources, and the application is re-balanced to take advantage of the additional capacity available through the new machine. It is important to point out that if you don’t have a dynamic pool of resources available through virtualization or cloud computing, you could still start a machine manually. Once the machine is started, adding it to the application’s resources and rebalancing would happen in exactly the same way.
• Continuous rebalancing – when new machines are added to the Elastic Middleware, it immediately detect them and re-balances the assets currently running on the new machines (if necessary), ensuring optimum utilization of all services.

Carrying out this dynamic SLA was deigned to happen while the application is running, ensuring no transaction or data loss during the process. In-flight transactions continue to be served with no noticeable hick-ups. 

Concerned of losing control?

Whenever I introduce the concept of built-in SLA and automation, I get two type of responses. One is “Wow, that’s cool!”, the other is “It sounds like I’ll lose control.” The concern about losing control is very valid, as in many cases, especially in non-cloud environments the concept of spare capacity rarely exist and therefore adding a new machine would often involve some manual intervention. In addition, when something goes wrong, other parts of the system or even other parts of the organization need to be involved, so we can’t always assume that full automation is possible.

The cruise control analogy

The approach GigaSpaces has taken is similar to the way cruise control works in our cars. You can choose to give up full control under a certain threshold, but at each point in time you can take the wheel back and resume full control. Like many of the cloud providers, we provides this type of cruise control for our services through an open API that enables management and control of every aspect of our cluster. You can query the available node, CPU, application partitions and service components. GigaSpaces Elastic Middleware was built as a layer on top of that API, which means that if the built-in functionality that comes with the Elastic Middleware doesn’t fit your needs, you can go one layer down and write your own custom behavior. For example, you can specify that in your system, scaling or failover events send an alert that will trigger a manual process for adding a new machine. In other words, you have full flexibility to choose when to turn on “cruise control” and when to take ownership.

SaaS-enabled with built-in multi-tenancy

Multi-tenancy often means that can run multiple users/customer applications on a shared resource, thereby reducing the cost of ownership per user. This is considered an area of high complexity, specifically in SaaS applications, because with today’s middleware, the application itself is responsible for mapping between users and application tenants. With Elastic Middleware Services, multi-tenancy is built into our implementation and API. Thus the burden of sharing resources is moved from the application down to the middleware. Each application works with its own dedicated middleware service (data, messaging, etc.) but that middleware service can share resources with other middleware services. In other words, instead of having one big database running on a dedicated machine and split it out between users at the application level, you can have lots of small databases spread between machines, where each application can have its own dedicated database and still have that database shared with other database instances that runs on that same machine.

This not only significantly simplifies your code, it also provides better isolation between multiple users, as well as independent life cycle management for each tenant.

There are various tradeoffs between sharing and isolation. With isolation you get better security and control of your own environment, but lower utilization and higher cost. With sharing, you can reduce cost and still achieve reasonable isolation, but not at the same level as when running on a dedicated resource. The Elastic Middleware Services make it possible to define the isolation/sharing at the application level. It does this by introducing the notion of an isolation level:

Currently we support three isolation levels:

• Dedicated – guarantees a dedicated machine allocated per instance of the application.
• Shared private – multiple instances of the application or organization share the same resources, but other departments or organizations are isolated.
• Shared public – everyone shares everything.

 

Available both on your local network and on the cloud

To try the new service you don’t need to have a private cloud or run your application on a public cloud. You don’t even need a virtualization layer. All you need is to launch a single GigaSpaces agent (Java process) per machine (normally that process will be started automatically at machine boot).

Once you’ve done this, you can start interacting with your machines and an create the desired middleware service through a simple API call. The following snippet shows how you might use this API:

Admin admin = new AdminFactory().createAdmin();

ElasticServiceManager elasticServiceManager = admin.getElasticServiceManagers().waitForAtLeastOne();

// Start a new data-grid

ProcessingUnit pu = elasticServiceManager.deploy(new ElasticDataGridDeployment("mygrid") // give it a name

.isolationLevel(IsolationLevel.DEDICATED) //isolation level

.highlyAvailable(true) // set the high availability level

.elasticity("2GB", "6GB") // set the required capacity range

.jvmSize("512MB") // configure the size per VM

.addSla(new MemorySla(70))); // define Memory SLA

That’s it! – you just got a full cluster deployed and ready to use.

What does it mean for you?

To sum up, I here are the main benefits that Elastic Middleware Services bring to each type of user:

For developers

As a developer, you can get access to the service you want just by calling an API, without worrying about installation or cluster setup. You get access to the service from your existing platform. In other words, you don’t need to run your application on GigaSpaces XAP to use any of those services. You can also pick and choose – use only the services you want, e.g. just the data, data and messaging, etc.

For the (private) data center

From a data-center perspective, you could install Elastic Middleware Services only once in your data-center. The same installation and resources would be shared amongst all users in your organization. If you happen to have virtualization in place, you can do this very easily using the GigaSpaces virtual appliances. 

Once you install the system, other users in the organization don’t need to go through the installation process – they can start consuming the middleware services just by calling an API, like they would do on the Amazon cloud when they launch a SimpleDB instance.

In a way, the Elastic Middleware Services gives the data center the power to provide higher-lever services to the business, rather than just plain infrastructure services, enabling the organization to become more agile.

For public cloud providers

Public cloud providers can be viewed as an outsourced version of the local data center, so they should experience much the same benefits as data centers. In addition, with Elastic Middlware Services, public cloud providers can offer a set of middleware services that are enterprise-ready with extremely high utilization per service. Because the Elastic Middleware Services are fully memory-based, you can use as much as 10X less machines to achieve the same throughput requirements, compared to a disk-based approach.

Ricky Ho published some interesting performance characteristics of Amazon SQS and SimpleDB in one of his recent posts:

• Network latency and throughput: 20 - 100 ms for SQS access, SimpleDB domain write throughput is 30 - 40 items/sec.
• Eventual consistency: 2 simultaneous requests to dequeue from SQS can both get the same message. SQS sometimes reports empty when there are still messages in the queue

While both SQS and SimpleDB provides a scalable data store and messaging services, their performance are at least 10-50X slower than an equivalent GigaSpaces data and messaging service. Implementing a Map/Reduce scenario with GigaSpaces is going to be significantly simpler and closer to real-time compared to Amazon Elastic Map Reduce (and that’s a topic for a separate post). Another cool feature is the ability to execute your code within the service container, which gives you an even bigger improvement in performance and latency.

The Elastic Middleware Services have much higher utilization and simpler maintenance, compared to equivalent services offered separately to the end-user, because they are provided using a shared cluster with a unified clustering model. With Elastic Middleware Services you get JPA, memcached, key/value (NoSQL), Spring, JMS, Remoting and Map/Reduce in Java or .Net, in a single deployment.

Existing GigaSpaces XAP users

Existing GigaSpaces XAP users can use the Elastic Middleware Services as part of XAP.  They’ll benefit from the simplicity provided through the high-level abstraction. In addition, the Elastic Middleware Services make it easier to plug-in the GigaSpaces components into existing application servers or development environments. They also make it significantly simpler to manage and deploy GigaSpaces XAP in various groups in the organization, because they can all share the same virtual pool of machines in the data center.

Availability

The Elastic Middleware Services will be available as part of our upcoming XAP 7.1 release, due for end of March 2010. It is currently available for private beta testing. We will also release a public beta of the Elastic Middleware Services through our upcoming 7.1 release candidate version. As always, we’ll welcome feedback and will be very happy to hear about your specific requirements. You could either post comment to this post or send an email to <pm at gigaspaces dot com>.

Application Monitoring as a Service with New Relic and GigaSpaces

Application monitoring has become a core component of IT infrastructure. It gives you a view of what’s happening to your applications at the higher level. With this information, you can detect anomalies and prevent failure before it happens, analyze trends which will help you predict growth and better estimate the sizing of your application, and so on.

Understanding how application monitoring works

Application monitoring includes three main parts:

• Agent – the agent acts as a sensor. It is the entity that plugs into the application, collects data at various points of the application flow, and reports them to the monitoring server.
• Monitoring server – the monitoring server is the part that collects all the data from the various application tiers, and stores it in a database, to enable aggregated analysis of application behavior and transaction flow over a period of time.
• Monitoring/reporting dashboard – the monitoring dashboard is the part that lets you view various reports on application behavior, based on the data that has been collected from the agent to the server.

New Relic – Application Monitoring as a Service

One of the main challenges involved in many of the existing application monitoring systems is the complexity involved in setting up a monitoring environment, as well as well the cost that is normally associated with it. To overcome these limitations, New Relic is offering real-time application monitoring, as a service, over the internet. What’s cool about it is that you can plug-in monitoring to your application in matters of minutes.

You can find a good overview of the New Relic architecture in Bernd Harzog’s overview here.

New Relic architecture overview

As can be seen in the above diagram, the New Relic agent plugs into your application and reports the state of the application to the New Relic services over the Internet. You can login through a browser and view your application behavior and statistics any time, by logging into their service.

Beyond the regular use cases of application monitoring (troubleshooting, business activity monitoring, alerting, etc), application monitoring as a service opens new usage opportunities:

1. SaaS/cloud application monitoring – SaaS providers are already used to an “As a Service” model – so monitoring as a service fits well with the way they run their own business and fits nicely into their cost model. As a matter of fact, this is how I got familiar with New Relic in the first place. When we built our cloud offering on Amazon, I was looking for a tool to monitor our cloud infrastructure. Geva Perry first introduced me to the New Relic concept, and it took me only a few hours to plug it into our platform. Bernd Harzog’s overview outlines nicely how we are now benefitting from New Relic in a cloud/virtualization environment:

Issues Unique to Monitoring Cloud Hosted Applications

Since RPM is designed to monitor applications that live in one or more clouds, we should explore exactly what it means to deal with the unique aspects of APM in cloud environments. The first set of challenges which must be addressed when performing APM for cloud hosted applications are the same challenges that must be addressed when monitoring applications that live on a virtual infrastructure – since we can safely assume that most clouds either today already live on a virtual infrastructure or will do so shortly. These issues are explored in detail in the white paper available for download at the end of this review, and are summarized below:

• Dynamic capacity. In virtual environments, capacity can be added automatically, and in many cases while the application is running. Therefore inferring application performance as a reciprocal of capacity utilization no longer works once an application is virtualized.
• Shared capacity. Virtualization puts guests into resource pools which share and pool CPU and memory capacity. Furthermore some virtualization platforms (like VMware) actually share memory across guests. Therefore whatever the number is that gets reported as the amount of a resource that is being used by the application can be warped, or made irrelevant by the degree of sharing that occurs in virtualized environments.
• Timekeeping issues. Virtualizing a guest causes its perception of elapsed time to warp as a function of how much that guest gets scheduled out by the hypervisor. This impacts time based metrics (like CPU utilization) collected by the guest OS and makes these metrics suspect and of dramatically reduced value.
• Dynamic configuration. In a virtual infrastructure, a guest may move between physical hosts, creating new “maps” of how the application is constructed. These moves may be driven by automated management solutions like VMware DRS. They may be driven by a decision to move an application from an internal cloud to an external cloud. APM solutions need to keep working as these moves are made, and if they include application topology mapping features, need to automatically update these maps to reflect changes in the deployment architecture of these applications.

The net effect of these issues is that when an application is hosted on a virtual infrastructure the old method of inferring performance as a reciprocal of resource utilization no longer works. A functional approach must start with an understanding of response time on a per transaction and user action basis within the application. This approach is essential not only because it is the only one that will work, but because it is the one that users and applications teams will insist upon in order to feel comfortable about “their” application residing on a shared/virtualized platform.

2. Real Time Collaboration - sharing your dashboard with third parties – in many troubleshooting situations you are asked to send your logs, CPU utilization traces, and so on, to the third party you are working with, so they can trace the root cause of your problem. The main reason you are asked for these things in the first place, is that you can’t share your local monitoring service with your partner – simply because you will need to worry about security, etc. With New Relic the fact that the monitoring service lives in a secured space outside your organization makes it easier to share this information in real-time, and make collaboration with other parties much simpler. A good analogy to this is Office and Google docs – it’s easy to share a Google document with someone, without worrying about security and without needing to actually send anything to anyone.

How to setup New Relic monitoring with GigaSpaces XAP on the cloud

To start using New Relic you should go through the following steps:

1. Create an account (If you don’t already have one).

2. Obtain the New Relic agent files.

3. Install the New Relic Agent and configuration file in your Java application.

After those two very simple steps you can start monitoring your application. More details on each step are below.

Step 1: Create a New Relic Account

To create a New Relic account, go to this page: http://www.newrelic.com/get-RPM.html

You can start with a free account by choosing RPM light – the free account is limited to basic monitoring usage.

For production monitoring, it is recommended to upgrade to one of the options listed in the page linked above. New Relic and GigaSpaces have announced a partnership that will enable you to get a discount on those other options. For details on how to obtain GigaSpaces/New Relic account, click here.

Step 2: Obtain your NewRelic agent files

The NewRelic agent setup consist of two files, newrelic.jar and a configuration file named newrelic.yml. You need to download them from the newrelic site and place them in your application environment.

You can find a good reference for this step in the NewRelic site here. There is also a nice videocast that shows how to do that with Tomcat here.

Step 3: Add the New Relic Agent to your GigaSpaces environment

You can add the NewRelic agent simply by adding the following environment variable.

Unix:

export EXT_JAVA_OPTIONS=-javaagent:/full/path/to/newrelic.jar

windows:

set EXT_JAVA_OPTIONS=-javaagent:/full/path/to/newrelic.jar

To verify that your New Relic agent has started properly, look at the log file located on the same directory as newrelic.jar. If everything worked properly, you should see a message indicating that the agent established connection with the New Relic site. Once this step is done, you can start monitoring your GigaSpaces application simply by logging in to your New Relic account.

Note that by default, New Relic uses a default application name, “MyApplication”. To provide a more meaningful name you should change the app_name: attribute in the newrelic.yml configuration file.

Setting New Relic in GigaSpaces XAP on the cloud

GigaSpaces XAP is integrated with various cloud and virtualization environments. Mostly recently we have integrated with GoGrid and VMware. In these environments, you’ll generally boot the system through a GigaSpaces agent, gs-agent. The agent makes it easy to deploy an application cluster using a simple symmetric configuration across all machine nodes. You can set up the New Relic agent through the same environment variable, as mentioned above. For example, EXT_JAVA_OPTIONS=-javaagent:/full/path/to/newrelic.jar. Since the agent normally starts automatically when the machine boots, you should make sure that this environment variable is set before the agent is called. In a Linux environment, this would normally be your init.d script.

The GigaSpaces MyCloud service is built on Amazon EC2 and uses an application deployment XML file to automate the deployment.

To set the New Relic agent in this environment, follow these steps:

1. Copy the newrelic.jar and newrelic.yml into your application S3 directory

2. Add the following transfer-file elements to your deployment xml:

<transfer-files>

<!-- new relic --> 

 <file>

 <source>$CPD/newrelic.jar</source> 

 <target>newrelic.jar</target> 

 </file>

 <file>

 <source>$CPD/newrelic.yml</source> 

 <target>newrelic.yml</target> 

 </file>

</transfer-files>

3. Set the environment variable to point to add your New Relic agent:

 <variable>EXT_JAVA_OPTIONS=-javaagent:/home/gsadmin/newrelic.jar</variable> 

To set the New Relic application-name attribute in your newrelic.yml configuration file to the cloud cluster-name, you can use the following shell utility:

sed s/_APP_NAME_/${clusterName}/ /home/gsadmin/newrelic.yml > /home/gsadmin/newrelic.temp
cp /home/gsadmin/newrelic.temp /home/gsadmin/newrelic.yml 

You should place this command line in the deployment initialization script of your GigaSpaces cloud XML file.

 <script-source>$CPD/deployScript.sh</script-source> 
 <script-target>deployScript.sh</script-target> 

Your feedback is needed

Adding application monitoring as a first class citizen to the GigaSpaces environment is very exciting for me. We started this journey by adding more visibility and control to our environment through our cluster management API (also called the administration API). We are only scratching the surface on how the two technologies can work together and produce more valuable information for our customers. There are lots of options and tradeoffs involved in getting a productive metering and control system, so it would be extremely valuable to get feedback based on your specific experience and expectations. Feel free to send even your craziest wish list to pm at gigaspaces dot com or simply post a comment on this post.