Nati Shalom's Blog

The Missing Piece in the Virtualization Stack (Part 2)

In the first part of this post, I discussed how virtualization and cloud computing, as we know it today, is only a small part of the solution for today’s IT inefficiencies. While new technologies and delivery models have made it much simpler to manage the infrastructure, this is not where our core inefficiencies lie. Virtualization principles must be extended to higher levels of the application stack, to make it easier for all of us to manage, tune and integrate applications. Otherwise we will continue to spend most of our time on things that don’t provide real value to the business – infrastructure, installation, management, tuning, and integration.

In this post, as promised, I’ll show how this missing piece can be filled in practice, by something called Elastic Middleware Services. These are middleware services that can be deployed with one API call, and are completely abstracted from the applications that use them. Elastic Middleware Services are very similar to cloud-based middleware like Amazon SimpleDB, but the major difference is that they are enterprise-grade, and can be purchased and installed in your private data center, for use by your internal enterprise applications.

To better illustrate this idea, I’ll discuss a reference implementation of Elastic Middleware Services which will shortly be released as part of GigaSpaces XAP 7.1. The discussion is intended to illustrate in concrete terms how to fill the missing piece in the virtualization stack. I’ve intentionally focused on the conceptual and fundamental elements of the GigaSpaces implementation, not on specific features and benefits, to make the discussion useful even if you are not using or considering GigaSpaces products.

By the way, I recently hosted a webinar on this topic, together with Uri Cohen, GigaSpaces’ product manager. In the webinar we presented GigaSpaces’ upcoming Elastic Middleware Services. The recorded version of this presentation is available below:

Elastic Middleware Services

The idea is to provide middleware services (messaging, data, Map/Reduce, and more), like those the ones provided by cloud providers such as Amazon and Microsoft Azure, in your own local environment. These services simply extend the basic principles of virtualization, as I outlined in the previous post, to the middleware stack:

Basic virtualization principles

1. Break big physical resources into smaller logical units
2. Decouple the application from the physical resources
3. Provide an abstraction that makes all the small units look like one big unit

From a middleware perspective, this means that instead of having one big database, you have lots of small units of data services (also called partitions or shards), which are grouped together through a client side proxy that exposes all of these small units as one big unit to the client that is using them. The client proxy is also responsible for abstracting the client from the physical location of those resources. To scale, all you need to do is add more of those units. The client experiences these additional units as extra capacity added to the service.

Deploying middleware with one API call

From an API perspective, this is a much simpler approach than traditional middleware, because it introduces a higher-level abstraction to the user, as can be seen in the code snippet below:

With this new elastic middleware API, users define requirements in a language that is closer to their domain. For example, “I want to create a data service that will have 10GB of data and can grow up to 100GB of data, I need hot failover and I'm willing to share my deployment with other users of my own organization but not with users of other organizations.”

In response, the Elastic Middleware Services would automatically do the following:

• Allocate the right number of partitions, based on machine availability and the available memory in each machine, to address the capacity required.
• Automatically launch a hot backup in for each partition and ensure that it runs on separate machine than the primary partition, to address the high-availability requirement. 
• Make sure that if another organization’s application is already running on the machine, the application won’t deploy on that machine, to address the security requirement.

Fully automated through built-in SLA

As I noted in my previous post, one of the major costs of enterprise applications over a period of three years is the cost of maintenance and operations. The main component of maintenance costs is the labor cost, closely tied to the amount of manual work you have to put into your system to make sure that it meets the application’s SLA. The cost grows proportionally to the demand for scale.

With Elastic Middleware, maintenance cost can be substantially reduced through built-in SLA and automation that covers the following aspects:

• Scaling SLA – scale when there is a memory shortage, high CPU utilization, Garbage Collector hiccups, etc. Users can choose to scale automatically when any of these events occurs, and customize their own thresholds (e.g. scale when CPU utilization hits 90%). The Elastic Middleware is integrated with a cloud or virtualization framework, to enable it to automatically pull or release of machines as needed.
• Failover – when a machine fails, the Elastic Middleware does one of two things:
  • Scale down – if there are available resources to meet the current workload, the system will automatically scale down and continue to service the application.
  • Rebalance – if there are not enough resources to serve the application, the Elastic Middleware calls the cloud/virtualization pool and starts a new machine. As soon as the new machine starts, it is added to the existing pool of resources, and the application is re-balanced to take advantage of the additional capacity available through the new machine. It is important to point out that if you don’t have a dynamic pool of resources available through virtualization or cloud computing, you could still start a machine manually. Once the machine is started, adding it to the application’s resources and rebalancing would happen in exactly the same way.
• Continuous rebalancing – when new machines are added to the Elastic Middleware, it immediately detect them and re-balances the assets currently running on the new machines (if necessary), ensuring optimum utilization of all services.

Carrying out this dynamic SLA was deigned to happen while the application is running, ensuring no transaction or data loss during the process. In-flight transactions continue to be served with no noticeable hick-ups. 

Concerned of losing control?

Whenever I introduce the concept of built-in SLA and automation, I get two type of responses. One is “Wow, that’s cool!”, the other is “It sounds like I’ll lose control.” The concern about losing control is very valid, as in many cases, especially in non-cloud environments the concept of spare capacity rarely exist and therefore adding a new machine would often involve some manual intervention. In addition, when something goes wrong, other parts of the system or even other parts of the organization need to be involved, so we can’t always assume that full automation is possible.

The cruise control analogy

The approach GigaSpaces has taken is similar to the way cruise control works in our cars. You can choose to give up full control under a certain threshold, but at each point in time you can take the wheel back and resume full control. Like many of the cloud providers, we provides this type of cruise control for our services through an open API that enables management and control of every aspect of our cluster. You can query the available node, CPU, application partitions and service components. GigaSpaces Elastic Middleware was built as a layer on top of that API, which means that if the built-in functionality that comes with the Elastic Middleware doesn’t fit your needs, you can go one layer down and write your own custom behavior. For example, you can specify that in your system, scaling or failover events send an alert that will trigger a manual process for adding a new machine. In other words, you have full flexibility to choose when to turn on “cruise control” and when to take ownership.

SaaS-enabled with built-in multi-tenancy

Multi-tenancy often means that can run multiple users/customer applications on a shared resource, thereby reducing the cost of ownership per user. This is considered an area of high complexity, specifically in SaaS applications, because with today’s middleware, the application itself is responsible for mapping between users and application tenants. With Elastic Middleware Services, multi-tenancy is built into our implementation and API. Thus the burden of sharing resources is moved from the application down to the middleware. Each application works with its own dedicated middleware service (data, messaging, etc.) but that middleware service can share resources with other middleware services. In other words, instead of having one big database running on a dedicated machine and split it out between users at the application level, you can have lots of small databases spread between machines, where each application can have its own dedicated database and still have that database shared with other database instances that runs on that same machine.

This not only significantly simplifies your code, it also provides better isolation between multiple users, as well as independent life cycle management for each tenant.

There are various tradeoffs between sharing and isolation. With isolation you get better security and control of your own environment, but lower utilization and higher cost. With sharing, you can reduce cost and still achieve reasonable isolation, but not at the same level as when running on a dedicated resource. The Elastic Middleware Services make it possible to define the isolation/sharing at the application level. It does this by introducing the notion of an isolation level:

Currently we support three isolation levels:

• Dedicated – guarantees a dedicated machine allocated per instance of the application.
• Shared private – multiple instances of the application or organization share the same resources, but other departments or organizations are isolated.
• Shared public – everyone shares everything.

 

Available both on your local network and on the cloud

To try the new service you don’t need to have a private cloud or run your application on a public cloud. You don’t even need a virtualization layer. All you need is to launch a single GigaSpaces agent (Java process) per machine (normally that process will be started automatically at machine boot).

Once you’ve done this, you can start interacting with your machines and an create the desired middleware service through a simple API call. The following snippet shows how you might use this API:

Admin admin = new AdminFactory().createAdmin();

ElasticServiceManager elasticServiceManager = admin.getElasticServiceManagers().waitForAtLeastOne();

// Start a new data-grid

ProcessingUnit pu = elasticServiceManager.deploy(new ElasticDataGridDeployment("mygrid") // give it a name

.isolationLevel(IsolationLevel.DEDICATED) //isolation level

.highlyAvailable(true) // set the high availability level

.elasticity("2GB", "6GB") // set the required capacity range

.jvmSize("512MB") // configure the size per VM

.addSla(new MemorySla(70))); // define Memory SLA

That’s it! – you just got a full cluster deployed and ready to use.

What does it mean for you?

To sum up, I here are the main benefits that Elastic Middleware Services bring to each type of user:

For developers

As a developer, you can get access to the service you want just by calling an API, without worrying about installation or cluster setup. You get access to the service from your existing platform. In other words, you don’t need to run your application on GigaSpaces XAP to use any of those services. You can also pick and choose – use only the services you want, e.g. just the data, data and messaging, etc.

For the (private) data center

From a data-center perspective, you could install Elastic Middleware Services only once in your data-center. The same installation and resources would be shared amongst all users in your organization. If you happen to have virtualization in place, you can do this very easily using the GigaSpaces virtual appliances. 

Once you install the system, other users in the organization don’t need to go through the installation process – they can start consuming the middleware services just by calling an API, like they would do on the Amazon cloud when they launch a SimpleDB instance.

In a way, the Elastic Middleware Services gives the data center the power to provide higher-lever services to the business, rather than just plain infrastructure services, enabling the organization to become more agile.

For public cloud providers

Public cloud providers can be viewed as an outsourced version of the local data center, so they should experience much the same benefits as data centers. In addition, with Elastic Middlware Services, public cloud providers can offer a set of middleware services that are enterprise-ready with extremely high utilization per service. Because the Elastic Middleware Services are fully memory-based, you can use as much as 10X less machines to achieve the same throughput requirements, compared to a disk-based approach.

Ricky Ho published some interesting performance characteristics of Amazon SQS and SimpleDB in one of his recent posts:

• Network latency and throughput: 20 - 100 ms for SQS access, SimpleDB domain write throughput is 30 - 40 items/sec.
• Eventual consistency: 2 simultaneous requests to dequeue from SQS can both get the same message. SQS sometimes reports empty when there are still messages in the queue

While both SQS and SimpleDB provides a scalable data store and messaging services, their performance are at least 10-50X slower than an equivalent GigaSpaces data and messaging service. Implementing a Map/Reduce scenario with GigaSpaces is going to be significantly simpler and closer to real-time compared to Amazon Elastic Map Reduce (and that’s a topic for a separate post). Another cool feature is the ability to execute your code within the service container, which gives you an even bigger improvement in performance and latency.

The Elastic Middleware Services have much higher utilization and simpler maintenance, compared to equivalent services offered separately to the end-user, because they are provided using a shared cluster with a unified clustering model. With Elastic Middleware Services you get JPA, memcached, key/value (NoSQL), Spring, JMS, Remoting and Map/Reduce in Java or .Net, in a single deployment.

Existing GigaSpaces XAP users

Existing GigaSpaces XAP users can use the Elastic Middleware Services as part of XAP.  They’ll benefit from the simplicity provided through the high-level abstraction. In addition, the Elastic Middleware Services make it easier to plug-in the GigaSpaces components into existing application servers or development environments. They also make it significantly simpler to manage and deploy GigaSpaces XAP in various groups in the organization, because they can all share the same virtual pool of machines in the data center.

Availability

The Elastic Middleware Services will be available as part of our upcoming XAP 7.1 release, due for end of March 2010. It is currently available for private beta testing. We will also release a public beta of the Elastic Middleware Services through our upcoming 7.1 release candidate version. As always, we’ll welcome feedback and will be very happy to hear about your specific requirements. You could either post comment to this post or send an email to <pm at gigaspaces dot com>.

Interview with Michael Di Stefano from Integrasoft on their CEP Cloud Services using Esper & GigaSpaces

During the past few weeks I had the honor to have a discussion with one of our partners Integrasoft who developed a distributed Complex Event Processing engine on top of Esper a popular opensource Complex Event Processing engine and recently integrated their solution into GigaSpaces.

As many of GigaSpaces users already design their application in an event driven fashion I thought that the CEP solution proposed by Integrasoft may  be a natural fit.

Below is a summary of a discussion i had with Michael Di Stefano where he provides an overview on Integrasoft and  CEP and later explain their specific solution and how GigaSpaces users can benefit from it:

Michael can you tell us a bit about yourself and Integrasoft?

We help clients and product companies optimally match business need with the technology that best meets that need. This is the fundamental philosophy is embedded in Integrasoft, the company I founded in 1997. Integrasoft embraced Grid computing when hit Wall Street almost a decade ago, and quickly promoted the importance of data affinity and the shift of the most important resource in an IT environment from CPU to the network. Today Clouds and Complex Event Processing (sometimes called CEP, Event stream Processing, and Business Event Processing) are the leading technologies addressing business need in not just Financial Services but across industry sectors.

True to form Integrasoft has taken CEP’s “server” approach to event processing and have adopted it to a pure distributed computing architecture. CEP Service Clouds abstracts CEP engines and provides a framework to network them together forming a Cloud of CEP that can host “services” in the cloud. These services can take advantage offered by CEP -AND- the virtualization and scale of a cloud. With CEP Service Cloud one can push the processing out to the fringes of the cloud eliminating unnecessary data movement from a data source to today’s centralized CEP Servers.

What CEP stands for?

A definition of Complex Event Processing can be found on Wikipedia

Complex event processing, or CEP, is primarily an event processing concept that deals with the task of processing multiple events with the goal of identifying the meaningful events within the event cloud. CEP employs techniques such as detection of complex patterns of many events, event correlation and abstraction, event hierarchies, and relationships between events such as causality, membership, and timing, and event-driven processes. CEP is to discover information contained in the events happening across all the layers in an organization and then analyze its impact from the macro level as "complex event" and then take subsequent action plan in real time.

And like all “buzz words” CEP is now commonly applied to many products. Is CEP a pure marketing buzz or something else? What is different about CEP form other middleware infrastructures of recent years (Grid, Data Grids, distributed caches, etc) is that CEP itself is an evolution of what IT professionals have been doing on Wall Street for almost 15 years. Custom code and rules engines are the fore runners to today’s CEP products. Today’s products are general-purpose tools that allow developers to take advantage of all it has to offer without having to know the intricacies happening inside the engine. Is this to say that a rules engine or custom code is not CEP? Not at all, they well address event processing in the scope of the problems they are designed to meet.

Events or transactions occur throughout the enterprise. These events may or may not be dependent on time, may or may not be dependent on sequence, and may come from one or many systems or applications within a business group or across business groups. So in order to consume the events regardless of source, understand the event, correlate it across other events with regards to time and source, or sequence in events, one need to leverage CEP as part of the architecture.

The CEP technology has been around for years and is leveraged by many firms. With the technology evolving over the years with a family of CEP engines available through vendors and open source, the choices of which engine(s) to leverage are wide. The key things to be concerned about as this technology is being leveraged is the consumption of these events in a way that is scalable and out of the box thus creating a distributed CEP Cloud Services.

What is a typical architecture of a CEP solution?

As with most architectures, the 50,000 foot view is similar to all other event driven systems. Event sources, processing of the events, event consumers and all the integration necessary to join (either tight or loose coupling) the systems together.

• Event source: – This would be the actual messaging provider (GigaSpaces in our specific case)
• CEP engine – This is the heart of the CEP. Matching engine is responsible for processing incoming event against users queries. Users queries may require a history of events to fulfill a given query. Users can express their queries in an SQL like semantics. The matching engine will trigger the appropriate listener when a query conditions has been met.
• Event consumers– The specific logic components (e.g. external systems or other middleware components such as GigaSpaces) that are be triggered when a certain condition happens.

However as you dive closer into”CEP” architecture significant differences emerge. Joining event streams together in logically similar fashion as joining tables in a database greatly simplifies implementations while reducing latency and increasing throughput. The result is smaller and simpler code base therefore a faster delivery of function. However, if not careful these benefits can quickly erode by trying to do too much inside the CEP Engine where the complexity of the streaming rules affect performance and maintainability.

What are the differences between a complex event and regular event?

Quoting CEP provider Esper:

“Regular events normally represents a concrete state, a complex event is normally an aggregation of multiple events (not necessarily of the same type) that identify a meaningful event.”

What are the difference between standard queries and continues queries?

A standard query, associated with a database operates over the tables in the database and returns a result data set. The tables stand still and the query traverses the data set (tables in the database). They run once and exit.

Continuous Queries are always “on”. However the query itself is stationary and the data (events) streams through the query, when the conditions of the query are met a resulting event is generated into the event engine for further rule processing and/or output to subscribers to that event. Visualize the query as a 2 dimensional plane and the various data streams orthogonally passing through the query plane for evaluation. The resulting events emerging on the other side of the query plane are new events resulting from the query conditions evaluating as true.

What are the difference between CEP and other MOM such as JMS?

MOM and JMS are transports moving data across a network. CEP is a process that may uses these middleware tools as a mechanism to consume and publish event streams.

What does Integrasoft add to the current CEP engine?

CEP engines have been leveraged in many business applications as siloed solutions. With a great push into the Cloud infrastructure for High Performance Computing, many firms are finding the need to run across multiple CEP engines for Services regardless of the nature of the Cloud.

Integrasoft CEP Cloud Services product offers the infrastructure required to run CEP Cloud(s) to meet the business application demands and required services via CEP engines “networked” together running transparently for virtualized services. The solution offers holistic and intelligent CEP Services within the Cloud.

The CEP Cloud Services form “Clouds of CEP” to be leveraged as needed by applications within a heterogeneous environment that is typically found in any infrastructure. Integrasoft CEP Cloud Services allows for the processing of complex events, scalable with the cloud, and transparent to the applications (see diagram below).

As illustrated in the diagram below, the solution offers:

Transparency – The applications need not know about the underlying architecture.

Scalability – “Sub-Clouds of CEP Services” within the cloud if needed.

Holistic & Intelligent Services – Intelligent processing of all events against defined rules. Holistic view of what is going on in the cloud.

Inter-CEP Engine communication – CEP engines framework to establish the needed communication for total transparency, scalability, and manageability.

What is the value for GigaSpaces users?

Leveraging the CEP engines in a public or private cloud for services comes another degree of challenges. The business applications development and deployment are concerned about:

· Underlying middleware technology

· Underlying Database technology

· Underlying business event processing technology

These challenges impact development time and cost as well as time to market. With the integration of Integrasoft CEP Cloud Services and Gigaspaces XAP, the solution offers a complete services infrastructure solution to handle all business events with:

· Integrated solution for event processing

· Virtualized CEP Cloud(s) Services

· Distributed cache for HPC

· Underlying distribution messaging

· Underlying database persistence

· Easier deployment and management

The solution offers high performance CEP + Messaging + Distributed Cache which is typically need by HPC applications. The tight binding of these three functionalities offers a new class of applications that are high event rates, data intensive, complex relations and correlations of events.

Now, the business SLAs demands can be met with a linear performance scale of a Cloud, constant latency as event rates increase, where event sources and processing of events are physically close together to maximize efficiency and performance. In summary, the solution is geared to control costs and minimize operational risks which are inherent with Cloud infrastructure.

My take

Event Driven Architecture (EDA) is becoming more popular in recent years due to the demand for greater scalability. CEP provides a way to extend the use of EDA into the way we process and access our data. Having said that most of the existing CEP solutions relies on a centralized event coordinator and could become a scalability bottleneck as a result of that. What’s interesting in the approach taken by integrasoft  is that it brings the scalability of GigaSpaces to CEP through the integration of Esper and GigaSpaces. 

I would be very interested to learn about your specific requirements in that area and work collaboratively with Integresoft to make sure that the integrated solution can address your needs.  Feel free to contact me or Michael directly on that regard or simply post a comment on this post.