Nati Shalom's Blog

PaaS on OpenStack

In my last post (GigaSpaces OpenStack Explained) I introduced our plan to add support for OpenStack in our platform:

One of the goals for our second-generation PaaS/SaaS enablement platform was to enable smooth migration between different cloud providers. We were able to achieve this goal through the use of our own abstraction (the Scaling Handler) and through the integration with the JClouds project that provides common abstraction to most of the existing cloud providers. With that, we can ensure that any application can be moved from the likes of Amazon to OpenStack or to an organization's own private cloud with zero changes to the application code or configuration.

The only change involves is setting the user/key of the specific cloud.

By adding support for OpenStack, we now enable users to safely move to an OpenStack-based cloud when they're ready and with little effort, yet they gain all the benefits that comes with it in terms of cost, openness etc.

Yesterday, I had a session during the OpenStack Summit where I tried to present a more general view on how we should be thinking about PaaS in the context of OpenStack.

The key takeaway :

The main goal of PaaS is to drive productivity into the process by which we can deliver new applications.

Most of the existing PaaS solutions take a fairly extreme approach with their abstraction of the underlying infrastructure and therefore fit a fairly small number of extremely simple applications and thus miss the real promise of PaaS.

Amazon's Elastic Beanstalk took a more bottom up approach giving us better set of tradeoffs between the abstraction and control which makes it more broadly applicable to a larger set of applications.

The fact that OpenStack is opensource allows us to think differently on the things we can do at the platform layer. We can create a tighter integration between the PaaS and IaaS layers and thus come up with better set of tradeoffs into the way we drive productivity without giving up control. Specifically that means that:

• Anyone should be able to:
  • Build their own PaaS in a snap
  • Run on any cloud (public/private)
  • –Gain multi-tenancy, elasticity… Without code changes.
• Provide a significantly higher degree of control without adding substantial complexity over our:
  • Language choice
  • –Operating System
  • –Middleware stack
• Should come pre-integrated with a popular stack:
  • Spring,Tomcat, DevOps, NoSQL, Hadoop...
  • Designed to run the most demanding mission-critical apps

The slides below illustrate a very high-level overview of the problems as well as how OpenStack can help provide a solution, from the presentation at the Summit:

PaaS on OpenStack

View more presentations or Upload your own.

We’ve recorded a short demo that shows how all different pieces pieces actually work together in the context of OpenStack:

 

You should note that since the demo was mainly targeted at illustrating the OpenStack integration through JClouds/OpenStack provider, it doesn’t cover much of the feature set, such as Multi-tenancy, fine-grained monitoring, or fail-over, nor does it show deploying full-blown web-apps and big-data apps, etc.

The actual code for the JClouds/OpenStack provider should be available through the JClouds project shortly.

Call for action

Today, there is a lot of work and interesting innovation being done in the PaaS world by different providers. Unfortunately a lot of that work is done with very little collaboration. The OpenStack community can be a great environment to put all those great ideas into something meaningful and open.

I hope that our initial joint work with OpenStack, Citrix, JClouds, and GridDynamics can be a good start in that direction.

I’m trying to figure out what’s should be the right way to establish a more formalized Open-PaaS group as part of the OpenStack community. Any ideas/help would be greatly appreciated...

GigaSpaces OpenStack Explained

One of the major concerns of many IT organizations is cloud vendor lock-in. This concern was expressed recently in "Banks fear cloud vendor lock-in," from IT Wire:

The onset of cloud computing gives vendors the chance to lock customers in to their infrastructure, using proprietary protocols to ensure they’re on the monthly billing cycle as long as possible.

The OpenStack project emerged with a mission to address this concern by creating a community-led open source project enabling any organization to create and offer cloud computing services running on standard hardware. 

GigaSpaces joined the OpenStack project with the mission to enable any organization to build its own Platform-as-a-Service (PaaS), with its own choice of language and best-of-breed middleware stack.

In this post, I’ll try to provide more insight into our current and future plans around OpenStack, and more specifically our joint collaboration with the Citrix OpenCloud initiative.

GigaSpaces OpenStack Explained

• What does GigaSpaces' OpenStack support mean?

One of the goals for our second-generation PaaS/SaaS enablement platform was to enable smooth migration between different cloud providers. We were able to achieve this goal through the use of our own abstraction (the Scaling Handler) and through the integration with the JClouds project that provides common abstraction to most of the existing cloud providers. With that, we can ensure that any application can be moved from the likes of Amazon to OpenStack or to an organization's own private cloud with zero changes to the application code or configuration.

  The only change involves is setting the user/key of the specific cloud.

By adding support for OpenStack, we now enable users to safely move to an OpenStack-based cloud when they're ready and with little effort, yet they gain all the benefits that comes with it in terms of cost, openness etc.

• Can I use the OpenStack integration outside of GigaSpaces' platform?

Yes. The OpenStack integration was developed in close collaboration with GridDyanmics and Adrian Cole from JClouds. We intend to make the integration with OpenStack available to the entire Java community through the open source JClouds project. In this way, any Java application can easily run on OpenStack.

• What specific application stack is currently supported?

Today, there are a wide variety of tools and frameworks available throughout the ecosystems in Java, .Net, Ruby, and more. We feel that limiting the platform to a specific stack that we control -- as in the case of Google App Engine -- is too restrictive. Instead, we wanted to create a foundation that enables users of the platform to easily host any service or framework they choose on the platform. We also wanted to do this in a way that provides consistent behavior in terms of deployment, monitoring, elasticity, and scaling. Therefore, we developed a Universal Service Manager and Service Orchestration framework. The framework makes it easier users to plug in their own choice of services that comprise the application stack (Tomcat, MySQL, NoSQL through Cassandra, Hadoop, Ruby...).

In addition, we will provide a set of built-in recipes. At first we will be targeting Big Data stacks, which include integration with NoSQL and Hadoop, and also reporting tools to make the deployment of Big Data applications significantly simpler.

The integration with OpenStack will make it possible to bring these benefits to any application running on OpenStack.

• How is this related to the previously announced Open Elastic Platform with Citrix?

The new platform builds on the collaboration announced in October 2010 by Citrix and GigaSpaces, highlighting the integration between Citrix® NetScaler®, Citrix® XenServer® -- both part of the Citrix OpenCloud solution -- and GigaSpaces eXtreme Application Platform (XAP).

In this new release we are adding the following new development:

Greater Openness

Openness at the IaaS layer -– The specific integration with NetScaler (load balancer) and XenServer would be done through more open interfaces provided through the Citrix/Openstack contribution, which means that OpenStack users can use those intefaces to plug in any hypervisor or load balancer.

Openness at the application layer –- The previous version of GigaSpaces XAP offered a limited set of application stack support features, mostly geared to services that we control and mostly in Java. With the new platform we offer significantly greater flexibility on various fronts:   

• • Open to any application stack and language: As I mentioned earlier, users can now easily host any application or service and build their stack of choice, while at the same time managing and controling their application in a consistent way in terms of the deployment, monitoring, elasticity, etc.
  • Users can choose their container of choice: The new platform will include support for Tomcat in addition to our existing Jetty support.
  • Users can choose standard APIs to scale their data: In most cases users looking for data scalability have had to rewrite their applications. This is still the case with most of the NoSQL and in-memory data grid solutions. Through standard JPA support in our platform, we can finally reduce that lock-in barrier.

Better Application Monitoring, Specifically Geared for PaaS

One of the challenges with many of the existing monitoring systems is that they were not geared for PaaS-based deployment. In this new offering we provde PaaS-driven monitoring that is tightly integrated with the platform, and which can interact with the platform to deal with failure or scaling events without any human intervention. In addition, it provides a holistic view that includes the application and infrastructure, as can be seen in these screenshots:

 

Furthermore, the new monitoring was designed to integrate with the existing set of data-center monitoring tools.

Better Performance and Scalability

By leveraging the Citrix OpenCloud solution, GigaSpaces will provide tighter integration between the platform and the infrastructure layers to up-level services offered through OpenStack clouds. In this way, we leverage the years of experience of both platforms in delivering high performance and low latency to mission-critical applications and remove the hassle involved in tuning the entire stack. We believe that through this joint work we can bring a lot of that experience into the OpenStack community. 

We can also now offer more fine-grained multi-tenancy support –- by combining process and VM-based multi-tenancy. This makes it possible to achieve significantly higher density and utilization of existing resources, and reduces the amount of resources -- and therefore the cost -- associated with serving a particular application load.

• What does this means for Enterprises, ISV/SaaS, and Cloud/IaaS providers?

Any user of this new offering will benefit from the greater openess and reduced lock-in concern. They  wouldn’t be relying on single vendor for their future. This also means that they will have better control over their own stack and cost margins, and will therefore gain the ability to offer more competitive prices with other equivalent cloud providers. More specifically

• Enterprises can use this offering to build their own Enterprise PaaS layer that is specifically geared for big-data analytics, e-commerce, and financial applications.

• SaaS ISVs can use this offering to SaaS-enable their application and provide the same solution off- and on-premises.

• Infrastructure/Cloud providers can use this offering to offer an Amazon-like services stack, including RDS, SimpleDB, SQS, Cloudwatch, and Elastic Beanstalk as a pre-packaged solution.

 

Final Words

I’m very excited about all of these new developments. OpenStack fills in the missing piece in the cloud –- as its name suggests, it unlocks the cloud by providing a truly open cloud stack and provides an essential substrate that drives innovation and collaboration -- which couldn’t be done before. It's interesting to compare the level of effort that we had to invest in the past when we did the first integration with Citrix to the one with OpenStack. Openstack enables us to work completely in parallel and make progress without too much coordination and still achieve substantial progress. Moreover, every delivery of our work can be relevant to wider spectrum of users. It also enabled us to join forces with other members of the community, such as Adrian Cole from JClouds and GridDynamics, simply because we share a similar goal to support the OpenStack open cloud mission.

This is only the beginning. I hope that with this new development we can continue to work toward greater openess at the application platform layer together with Citrix and the rest of the OpenStack community.

I’m going to give a talk on on the subject (PaaS on Openstack) during the upcoming OpenStack Design Summit on April 26^th, 2011, in Santa Clara, California, where I hope to share some of these thoughts and hopefully get more work on this happening within the community. 

Availability

I will also be announcing the first GigaSpaces code contribution to OpenStack at the Design Summit. Note that the integration with OpenStack will be made available to the entire Java community through the open source JClouds project. 

You can also get an early look at our upcoming 2nd-generation CEAP release by registering for the Early Access Program.

References:

• Spring is Sprung by Simon Crosby -Citrix CTO)
• PaaS shouldn't be built in Silos
• GigaSpaces 2nd generation PaaS/SaaS enablement platform FAQ
• GigaSpaces CEAP sneak preview
• GigaSpaces/Citrix  Open Elastic Platform

Multi-tenancy: emulation or the real thing? - My Take

Phil Wainewright wrote and interesting take on Zdnet (Multi-tenancy: emulation or the real thing?) where he outlines different multi-tenancy levels:

• Hosting single-tenant applications on multi-tenant infrastructure
• Redistributing application processes across virtualized machines
• Injecting a user-specific column in the database driver

Phil also addressed two approaches to achieve multi-tenancy, to which he referred as emulation - multi-tenancy that is pre-backed into the infrastructure as in the case of GigaSpaces, or the "real thing," which stands for a custom build where multi-tenancy is handled at the application level as in the case of Salesforce.

In his final takeaway, Phil suggested that to get the most out of multi-tenancy, you have to deal with it at the application level.

The build vs buy dilemma

Phil's point reminded me of the classic build vs buy dilemma – at first sight you would think that a custom build that is tailored to your specific domain and application would always be better than a generic solution. But is it?

I addressed "build vs buy" in a previous post on the subject (The impact of cloud computing on build vs. buy behavior), using Fred Brooks's article No Silver Bullet. There are many lessons that Fred points out in his article that are worth reading, but here's a relevant point I'd like to highlight:   

The hardest single part of building a software system is deciding precisely what to build.

Fred suggests that we tend to justify custom builds without properly realizing the associated costs. In many cases, what we tend to refer to as “specific requirements” aren’t that specific after all - or to be more accurate, the differences don’t justify the costs.

What to choose?

In the context of multitenancy, we should consider the following things:

• Does the difference justify the cost?

How different would Salesforce multi-tenancy be compared to the multitenancy of any other SaaS or PaaS provider?

I would argue that most multitenancy providers are the same. Differences are minor; the goal is multitenancy, not a mix-and-match of potential features.

• Complexity consideration

Dealing with data multi-tenancy at the application level can be error prone and can be significantly more complex than dealing with it at the infrastructure level, with serious effects. For example, I heard from one SaaS provider that a bug in their system led to one customer accidentally getting access to another customer's data. In addition to the complexity, there are things that just can’t be dealt with properly at the application level – for example, in the case of Salesforce, a single tenant is bound to a single database instance and can’t scale beyond that single database.

• Time to Market consideration

When you launch a new SaaS or PaaS application, time to market often plays a critical role. Dealing with multi-tenancy at the application level requires lots of effort which can lead to a significant delay in time to market.

Joseph Ottinger points out that this effort can be addressed in lots of ways - perhaps by programming language choice, for example. But even these solutions carry costs, in terms of your team's capabilities or even in scalability. The language you choose, for example, might be able to help your time to market, but might cost you in performance, or even address how your architecture is built. These choices can be made well, but you should go in with eyes open to what choices you're making.

• Cost consideration

When were dealing with multitenancy at the application level, we're often limited to cross-grain multi-tenancy as we don’t have full control over the underlying infrastructure.  For example, in the case of data-multi-tenancy we can easily scale out and and scale down our data services based on the actual capacity that is being used – doing that at the application level would be much harder, maybe close to impossible. That means that we can achieve better density and thus control the number of resources and thus the cost at each given point in time by using platform multitenancy.

It’s not a mutually exclusive decision

In the context of multi-tenancy, there are always going to be things better dealt with at the infrastructure level such as data-multi-tenancy – and application specific multi-tenancy such as the user interface layout.

By delegating the generic part of the challenges of multi-tenancy to the infrastructure layer we can devote more time and effort at areas more specific to our business. 

The full story in 15 min

Phil's article triggered a more in-depth discussion with Uri Cohen, GigaSpaces product manager, which later led to a recorded interview (15 Min on Multi-Tenancy) in which Uri address the following topics:

• Multi-tenancy: emulation or the real thing? the classic build vs buy dilemma
• What is multitenancy anyway?
• Why is it such a popular topic?
• Is it a feature or best practice?
• What is GigaSpaces Multi-tenancy?

Enjoy..

 

References:

• Multi-tenancy: emulation or the real thing?
• The impact of cloud computing on build vs. buy behavior
• Multi-tenancy: does it have to be that hard?

Scaling Social Ecommerce: Architecture Case study

In February this year I had the opportunity to meet Tomer Gabel, an Application Engineer at SHC Israel, who gave an excellent talk at one of our customer road shows about Social E-Commerce and the scalability challenges associated with handling of social graphs. Tomer later joined me for a case study, published on InfoQ.

In this post I tried to summarize the main takeaway from the interview. Enjoy..

What is Social E-Commerce anyway?

According to wikipedia, Social Shopping is a mechanism for e-commerce where shoppers' friends become involved in the shopping experience. Social shopping attempts use technology to mimic the social interactions found in physical malls and stores.

The opportunity

A recent study showed that over 92 percent of executives from leading retailers are focusing their marketing efforts on Facebook and subsequent applications. Furthermore, over 71 percent of users have confirmed they are more likely to make a purchase after “liking” a brand they find online. (source)

I recently came across an interesting analysis on the Subscribers, Fans, & Followers: The Social Break-Up. The report analyzed social behavior, and more specifically why consumers end brand relationships. Even though the focus of this post is not about ending brand relationships, I thought that some of the statistics presented in this report are quite useful for quantifying where we are today with social e-commerce in terms of the market size and its potential.

Facebook statistics:

• 73% of U.S. online consumers have created a profile on Facebook.
• 65% of U.S. online consumers are currently active on Facebook.
• 42% of U.S. online consumers (64% of those on Facebook) are “FANS” (have “liked” a company on Facebook).

Twitter statistics:

• 17% of U.S. online consumers have created a Twitter account.
• 9% of U.S. online consumers are currently active on Twitter.
• 5% of U.S. online consumers (56% of those on Twitter) are FOLLOWERS  (use Twitter and have “followed” at least one company).
• 71% of FOLLOWERS expect to receive marketing messages from companies through Twitter.

Clearly the adoption of social commence is quite staggering.

The focus of this post wasn’t to convince you should invest in social commerce but to give a context for one of the common scalability challenges associate with building social ecommerce platform - the "Social Graph."

The Social Graph challenge

The Social Graph is “the global mapping of everybody and how they're related.”

Processing social networks is not an easy proposition:

• Massive amounts of branching data
• No data locality
• Very few assumptions can be made about the data

In other words, to meet the capacity scaling demand you can’t store the data in a centralized location. That forces us to distribute the data. On the other hand, to access (or query) the data, we can’t assume that the data for our query is located in a single node. That forces us to look for the data on multiple nodes.

Let’s take a simple scenario to get some sense of the complexity of the problem:

 Example:

• Imagine every Facebook user (500 million)
• Imagine each person is only connected to 100 others (conservative estimate)

Query: How is user X connected with Y?

• X has 100 friends
• Each of them has 100 friends
• 10,001 nodes visited!
• 101 reads from the underlying storage system!

Clearly even if we can scale at the capacity level we can’t scale at the query level.

The crux of the problem:

• High branch factor necessitates many loads to serve even a simple request
• No data locality + high branch factor means very high random I/O
• Traditional storage models (RDBMS, flat files etc.) are a poor fit

The Solution:

• Use Memory as the main storage
  • Random I/O access works much better on memory devices than on disk (See more details here)
• Execute the code with the data - Using Real Time Map/Reduce
  • To reduce the number of iterations required to execute a particular query we use the executor API. The executor API enables us to push the code to the data. By doing that we can execute fairly complex data processing on the data node at memory speed vs network speed.
• De-normalize the data
  • To reduce the amount of traversal access and network hops per query on the graph we need to copy elements of the graph into each node. For example the list of Friends and friends of friends (up to a certain degree) could be stored in each node and thus become available to any element of the graph  without the need to consult with other nodes.

The operational perspective

Scaling Social Ecommerce involves not just the application architecture but also the operational aspect. This is of particular importance as ecommerce and social sites tend to evolve quite rapidly and therefore the time it takes to release a new feature from development to production is critical.  This process is often referred to as Continuous Deployment or in our specific case it can be referred to also as  "Continuous Scaling."

There are basically two factors that are important to achieve continuous scaling/deployment:

• Automation – if the process of deployment and scaling involves lots of human intervention than the time it would take to release a new feature would get significantly higher. It is therefore important that the entire process can be fully automated. In some cases, you may want to have manual check points in the process but even in that case the expectation is that manual intervention would involve clicking on a <continue> button and nothing else.  To achieve this level of automation we need to interact with the our application infrastructure through an API. This process of automation and integration between the development and operational environment is often referred to as DevOps. The GigaSpaces reference for dev-ops API is provided here.
• Schema evolution –There are many cases in which we would want to add or change our existing data structure to our existing application as part of the upgrade process without brining the system down. This is considered one of the more complex challenges to deal with with many of the existing data bases. A document model is schema-less and therefore more suited for this sort of rapid data change.

 The full story

Tomer did a a great job describing in more details the scalability challenges and the approach that they have taken in Sears to address those challenges as well as providing the specific insight on the performance figures that they achieved with their current system. The full interview recording is provided below:

 

References

• Designing a Scalable Twitter
• Memory is the New Disk for the Enterprise
• Why Existing Databases (RAC) are So Breakable!
• The Common Principles Behind the NOSQL Alternatives