Nati Shalom's Blog

Data agregation pattern for effective monitoring

In my previous post I wrote about two patterns for using a GigaSpaces cluster to solve some of the issues involved in managing distributed applications:

• Using the space as a scalable alternative to a directory service. With this approach each service publishes itself to the space directory service and a JMX proxy acts as a wrapper that virtualizes the different managed services from the client accessing them.
• Using the space as a management repository aggregating management data.
  

Steve Colwill from PSJ published a second post on this topic titled Data Aggregation via JMX and the Grid that covers the second option outlined above.  In summary, Steve suggests that each managed agent will report its aggregate statistics to the space. A JMX façade is used to expose the statistics through a standard JMX API as described in the diagram below:

If you think about it, this is yet another proof of the old aphorism by Butler Lampson that "All problems in computer science can be solved by another level of indirection".

The thing about indirection though is that it basically moves the problem from one layer (the application layer or management layer in our specific case) to another layer (infrastructure layer).  Now if that infrastructure layer is not in place, we have to create it ourselves, which makes Butler's statement kind of empty. This is where Space-Based Architecture becomes handy. It serves as a general purpose infrastructure layer that provide a means for solving data distribution, high availability and scalability. In this case we applied the abstraction principle as a way to expose the generic space capabilities through a specific set of APIs (JMX in this case). The combination of the two creates what I often refer to as a middleware virtualization layer. We use the same API, but the implementation of this API is virtualized. In this specific case, our JMX API doesn't point to a physical JMX server, instead it points to a virtualized cloud of servers.

Quoting Steve: 

One of the reasons I'm a fan of GigaSpaces and space-based architectures is that a number of architectural choices that are traditionally hard-wired: transactional/non-transactional, sync or async replication can be changed through configuration only. This enables common design patterns (and therefore components) to be applied to a wide range of application problems, by enabling the data integrity/performance equation to be tweaked at a late stage of application assembly.

 

Summary

The main benefit of this approach compared to the one described in the previous post is that it decouples the client and the managed services. A client doesn't need to maintain direct a connection to the managed service. Most of the logic is kept server-side. In this way we can keep the client-side --  which acts as the management façade -- stateless and thin.

Because the space acts as a distributed data-store, it is easier to build aggregated statistics, as all the statistics are placed in one logical entity - the space.

Decoupling enables us to easily add new services/policies to our system without changing the management service. For example, we can add SLA monitors that can listen to the statistics in the space and take action once a certain threshold is breached. This capability makes the space an ideal solution for those looking to build their next-generation management and monitoring application.

Cloud management frameworks (for Infrastructure-as-a-Service) are ideal candidates for this, as they have the need for proximity of the management information and application behavior. The management layer acts as a loopback mechanism that can tell the application how it is actually behaving. The application can use this information to adjust itself to meet a given SLA without human intervention.

Having said all that, it is also important to note that the two options presented in Steve's posts are not necessarily mutually exclusive. I could easily see how one could use the approach presented in this post for maintaining the managed dashboard information, and the approach in the previous post to invoke specific operations on a managed service, in case you want to drill down to the managed service level. Having one underlying technology that can be used to serve the invocation, virtualization and data virtualization is yet another benefit of using the space.

Private/Public Cloud

Most data centers of today run applications on dedicated machines. This is often referred to as static provisioning. In addition, applications are typically provisioned to handle expected peak loads. Both lead to over-provisioning and low resource utilization.

John Foley wrote an article back in August Private Clouds Take Shape in which he describes how data centers are reshaping themselves by taking ideas from public cloud providers, such as Amazon and Google.  The idea is to make the data center more cost-effective by enabling on-demand utility-based computing rather than dedicated machines.

The shift towards a utility data-center is a game changer. It will change the way IT operates, the way applications run in the data center as well as the culture of IT organizations.

The push to private clouds has a strong momentum these days, as all the major players, starting with the hardware vendors and ending with virtualization vendors, realize that their future rests in how well they fit in this model. Microsoft's Azure announcement one of the most significant announcements from a major vendor so far.

The need for Private/Public Clouds

At the same time, it is clear that to make IT operations more effective, it doesn't make sense to run all the applications that are currently hosted in a company's data center in the private cloud.  Not all applications in the data center are mission-critical or production systems. For example, take staging or testing environments. Such environments are supposed to be a mirror of the production environment. This is reasonable when our production system runs on a single dedicated server, but what if it runs on 10 or even 100 servers? Does it make sense to have another 10 or 100 dedicated servers just for that purpose? Another good example is disaster recovery. Disaster recovery sites require us to double our resources, let alone the cost associated with maintaining two separate data centers. These are classic scenarios in which running applications on a public cloud could lead to huge cost savings.

A recent InformationWeek survey (which Foley mentions in his piece) provides a more detailed view on the types of applications likely to move from private clouds to public clouds.

Making your application ready for the private/public cloud

The challenges

There are a few challenges to be aware of if we want as ready applications for a hybrid private/public cloud:

1. How do we design applications to be cloud-agnostic: how do we perform application testing on a public cloud and then run that exact same application in production on a private cloud. For the application to be cloud-agnostic we need to ensure that neither our application code or configuration is going to change by the transition and that our application is going to behave the same in both environments.

2. How do we enable seamless fail-over to a public cloud? To enable a disaster recovery scenario, the public and private clouds need to be connected in a way that enables seamless fail-over from the private to the public cloud

3. Future-proofing: There are many cases in which we can't make a clear decision as to where our application should be running at the time of writing or developing the application. We would like to be in a position to change the decision as to where our application will be running even after our application has been completely developed.

The solution

1. Enterprise-ready Platform-as-a-Service (PaaS)

Many recent discussions on cloud computing have been centered on the low-level infrastructure, such as virtualization. This is sometimes referred to as Infrastructure-as-a-Service (e.g., Amazon EC2). It is clear that to address the the first and the third challenges mentioned above we need a new middleware stack that will provide generic services for running applications in a virtualized environment, or a platform-as-a-service.

You can read more about this layer in GigaSpaces as Alternative to Google AppEngine for the Enterprise. The role of the enterprise-ready PaaS will be similar in nature to that of the application server of today, only that it will broaden to support the needs of private cloud environments, as I outlined in my earlier post.

2. Cohesive FT's VPN Cubed

While entperise-ready PaaS shields application code from the underlying cloud infrastructure, CohesiveFT's VPN Cubed is responsible for connecting one or more cloud networks through a secured channel in a way that makes them all appear as one big cloud, even if they are not owned by the same provider.

See for yourself how it works live!

My colleague Dekel Tankel  blogged about the joint solution by GigaSpaces and CohesiveFT aimed at addressing these challenges. The solution will be presented in a webinar next week Making Cloud Portability a Practical Reality.

We will show how you can write and deploy standard applications on top of GigaSpaces' Cloud Framework and use CFT's VPN Cubed for a seamless transition across clouds. Even more interesting is that using this solution you can even use multicast discovery across clouds.

By "standard application" I mean that you can deploy a standard JEE web application packaged as a WAR and deploy it in the multi-cloud environment. It doesn't need to be a "GigaSpaces application". In the webinar next week we will show live how we can deploy an application across both Amazon EC2 and Flexiscale, kill one of the machines and see how the application fails-over seamlessly between the clouds with zero downtime.

The webinar will take place next week: November 18, 2008, 1:00 PM EST.

You can register online here

The impact of cloud computing on build vs. buy behaviour

Last week I took part in an interesting discussion with a group of architects, and the question of build vs. buy came up. It came up specifically in the context of the recent experience with many of new Internet companies. I was wondering why it is that that many of them seem to spend so much in developing their own proprietary infrastructure, when it's clear that their needs are not that unique and that such development is not really part of their core IP. Many of them seem to continuously go through difficult experiences until they get their infrastructure right. And it seems that they all stumble into the same pitfalls along the way.

The typical answers as to why they build vs. buy were:

• It's core to our intellectual property and therefore we have to own all of our infrastructure
• We didn't find a solution that fits our needs since our needs are very unique
• We had a bad experience with Product FooBar which made us reevaluate build vs. buy

I can see how I'd react in exactly the same way; the most basic human instinct, when entering uncharted territory, is to rely only on yourself.

But looking at the amount of repeated failure over the past few years, it's pretty clear that this pattern isn't really proving itself too well either. Even when we choose to build it ourselves, according to our own specific in-house requirements, we still end up falling into the same trap over and over again.

Where to draw the line of build vs. buy?

To answer that question, I looked at Fred Brooks's article "No Silver Bullet" which was pointed out to me again by one of our lead architects few weeks ago.

One of the interesting points was the drastic impact of the economy on the build vs. buy decision pattern:

"The development of the mass market is, I believe, the most profound long-run trend in software engineering. The cost of software has always been development cost, not replication cost. Sharing that cost among even a few users radically cuts the per-user cost. Another way of looking at it is that the use of N copies of a software system effectively multiplies the productivity of its developers by N. That is an enhancement of the productivity of the discipline and of the nation.

The key issue, of course, is applicability. Can I use an available off-the-shelf package to perform my task? A surprising thing has happened here. During the 1950's and 1960's, study after study showed that users would not use off-the-shelf packages for payroll, inventory control, accounts receivable, and so on. The requirements were too specialized, the case-to-case variation too high. During the 1980's, we find such packages in high demand and widespread use. 

What has changed? Not the packages, really. They may be somewhat more generalized and somewhat more customizable than before, but not much. Not the applications, either. If anything, the business and scientific needs of today are more diverse and complicated than those of 20 years ago.

The big change has been in the hardware/software cost ratio. In 1960, the buyer of a two-million dollar machine would have felt that he could afford $250,000 more for a customized payroll program, one that slipped easily and nondisruptively into the computer-hostile social environment. Today, the buyer of a $50,000 office machine cannot conceivably afford a customized payroll program, so he adapts the payroll procedure to the packages available. Computers are now so commonplace, if not yet so beloved, that the adaptations are accepted as a matter of course."

The impact of cloud computing on the buy vs. build decision

I think Fred's analysis above is much more than just a historic curiosity. Exactly the same process is playing out today, with the advent of cloud computing and virtualization techniques that are turning IT infrastructure into a commodity, on the road to becoming a utility, and dramatically reducing its total cost. 

As Fred says in his paper - when the hardware gets cheap, development becomes very expensive. Under these new conditions, we're all going to have to change how we evaluate off-the-shelf products compared to the alternative of developing in-house. Proper TCO measurements need to be put in place at an early stage of the decision making process. 

For example, it will no longer be sufficient to choose a product based on the "best performance" or even "best reliability," because each of those factors has a direct cost associated with it. Instead, we are forced to have a better picture of the business requirements, so that we can choose the right product to meet our business needs, and it's not always going to be that the best product from a technical perspective is the right product - and the cheapest product won't be the right product either.

"The hardest single part of building a software system is deciding precisely what to build. No other part of the conceptual work is as difficult as establishing the detailed technical requirements, including all the interfaces to people, to machines, and to other software systems. No other part of the work so cripples the resulting system if done wrong. No other part is more difficult to rectify later."

It is quite surprising to see how much of the current decision-making process is not based on real business requirements. It is even more surprising to see how little we as architects and business people know about their system requirements and real application behavior.

A good example that was given in the architect meeting is the user experience. One participant in the discussion said that at one point, he was focusing on making the latency of serving his site pages as fast as possible and did a good job at that, but at the end of the day, when measured against a competing site that was performing slower, the impression of the user was that the competing site was performing better - the reason was simple, the other site was focused on user experience which led to less clicks per request and not how much time a single request is being executed.

If using off-the-shelf products can cut costs dramatically, why are there are so many product failures?

Fred provide an interesting answer to that question as well:

"Much of present-day software-acquisition procedure rests upon the assumption that one can specify a satisfactory system in advance, get bids for its construction, have it built, and install it. I think this assumption is fundamentally wrong, and that many software-acquisition problems spring from that fallacy. Hence, they cannot be fixed without fundamental revision--revision that provides for iterative development and specification of prototypes and products."

Final words

You might be thinking by now that these are all new lessons learned from the recent changes in the economy, right? – wrong. Go check when Fred Brooks' article was written.

If anything, I would strongly recommend that everyone reading this post would spend time reading Fred's article from start to finish, because I've only covered a small part of the philosophy behind his paper. I think the paper's viewpoint is extremely relevant today -- perhaps even more relevant today then it was when he originally wrote it.

Managing application on the cloud using a JMX Fabric

One of the challenges of managing application in a distributed environment such as Cloud/Grid is that collecting or finding the management information of each part of the application is a relatively complex task.

JMX provides a standard way to expose the management information (MBean) of a particular server. However, the way the client-side finds all the MBeans that comprise the application, or the way a single client might interact with the distributed parts of the application, is left open.

Steve Colwill from PSJ wrote a detailed blog, JMX for Grid Based Applications,
where he outlines a solution that uses JMX JSR-160 connectors and GigaSpaces to create a JMX Fabric. According to the proposed solution, the managed agent (server side) use the connector to add a reference of each MBean stub to the space. The client uses a FederatedMBeanServerConnection class that picks up those references from the space, connects to them and then delegates operations to the set of Mbean servers, effectively acting as a multiplexer.

Using the space as a JMX directory service

The above diagram illustrates how the model described by Steve works. The client is abstracted from the physical location of each server and can easily discover services that join the network. The connection from the client to the servers uses peer-to-peer communication, which means that once the service is discovered, no additional overhead is needed for communication between the client and the managed service. In this case, the space is used as a directory service. We leverage the fact that it can be distributed and dynamically discovered to simplify the discovery process in a distributed environment.

Using the space as a management data repository

The above model is quite useful for cases in which we want to expose federated services which have an existing remote interface. But this is not always the case. If it isn't, we can use the space as a management data repository, which contains full management information for each agent and exposes that information to the client or to any management application. In this method too, the client application is abstracted from the managed service. But unlike the first option, the client gets the information about the managed entity directly from the space, and doesn't need to maintain a connection with the managed service. The space in this case is used as a distributed database, so the application can not only obtain management information about an individual server but can also gather aggregate statistics and perform other aggregate data queries, directly on the data model.

Summary

Steve's solution to managing application in a distributed environment is an interesting one, as it enables applications that are already using a standard JMX interface to use a new federated model without changing the application and without adding a performance overhead. This is achieved just by plugging a new space-based connector. It is a good example that shows how a space can be used as a distributed directory service. It is important to note this is only one pattern in which a space can be used to solve this type of challenge. There are other ways; using the space as a management data repository, as I suggested in this post, is just one of them. The nice thing is that implementing any of these patterns becomes fairly simple once the space is brought into the picture.

I would like to end this post by thanking Steve specifically and PSJ in general for being a great partner for such a long period of time, and for sharing your experience in such meticulous detail.