Nati Shalom's Blog

Our Open Source Cloud (Re) Architecture with OpenStack + TOSCA

Cloudify OpenStack Native approach - The Story behind the Scenes

In the previous post I outlined what I mean by native-approach to the OpenStack cloud. In a nutshell, a native approach represents out of the box tighter integration with OpenStack, all without being limited only to OpenStack.

In this post I want to share some of the stories behind the design of Cloudify 3.0 and the approach that we have taken to make Cloudify *native* to OpenStack.  I would first start by outlining our motivation behind this approach.

OpenStack-only or Not-only-OpenStack?

It is our core belief that for the foreseeable future supporting hybrid cloud is going to be important for many of the OpenStack users for the following reasons:

1. Transitioning to OpenStack is a long journey - Many enterprises have VMware-based stacks as their core infrastructure. Where many of these organizations have often started their journey toward OpenStack by creating a dev environment, and then gradually moving more workloads into their OpenStack environment as they get more comfortable with it. This process usually spans over months and possibly even years. It became apparent to us, that during this time these customers would still be using VSphere and VCloud in parallel to OpenStack. For such users, having a common platform to manage their app deployment across two environments can smoothen the transition and also reduce the risk of getting stacked if one of the environments doesn’t meet their needs.
2. The public cloud market is going to be dominated by non-OpenStack clouds - while OpenStack is on a trajectory to dominate the private cloud market, the public cloud market looks quite different with AWS as a strong and long-standing leader and other major clouds emerging such as GCE and Azure - hoping to close in. Many organizations wouldn’t want to host their entire data center on a private cloud, and would therefore need to take a hybrid cloud approach. In order to support hybrid cloud, we had to include an abstraction layer that will allow us to integrate with clouds other than OpenStack.

At the same time we wanted to have a more intimate integration with OpenStack in order to leverage the fact that it is open source and allows for a deeper integration with its core services. Through that we were able to enable smooth and simple integration of Cloudify into OpenStack.

Extending or Rewriting?

Cloudify 2.x already took a step toward deeper integration with OpenStack and we were one of the first to announce our support for OpenStack. Recently, we also added support for OpenStack Neutron.

We realized that if we want to make Cloudify fit natively into the OpenStack infrastructure it is not enough to have integration with the OpenStack components.  We also need to ensure that the design behind Cloudify will be consistent with the way other projects and services of OpenStack are implemented. This led us to the conclusion that we wouldn’t be able to be a first class citizen with OpenStack unless we underwent a major redesign of Cloudify.

Going Python?

Our first step in this direction was to move from Java to Python. Moving to Python wasn’t as painful as you would think for us, because many of our developers already knew the language. This decision quickly proved itself, as it became much simpler not just to develop the new version of Cloudify faster, but it also enabled us to join other OpenStack projects as well as reuse some of the same components and frameworks that are already used by OpenStack.

Integrating with Core OpenStack Services

Many of the products that declare support for OpenStack mostly integrate with the compute API (Nova), and quite often come with their own security, messaging, configuration, and other such considerations. We felt that this is not enough, and doesn’t fit in with the way other services are built within OpenStack. Hence, we decided to deepen our integration with other services such as Keystone, Neutron, Heat and later with Ceilometer, Mistral and others.

Going TOSCA?

Cloudify 2.x was designed around a proprietary Groovy-based domain specific language (DSL) for defining application topology and configuration. It became apparent that the DSL had become a central piece of Cloudify architecture. As a result, maintaining a proprietary DSL would only serve as a barrier going forward. TOSCA (Topology Orchestration Specification for Cloud Applications led by the Oasis Foundation) defines a standard templating language. TOSCA was similar and richer in concept to our previous 2.x specification. Because of this, using TOSCA seemed like a natural evolution.

At the time when we decided to use TOSCA, the specification was provided in a fairly complex XML. Our first step for adopting TOSCA was to come with a simplified YAML based version which better fits into the spirit of OpenStack. After the OpenStack summit in Hong Kong, when we first introduced the idea, it became apparent that members from the Oasis group were already thinking along this line.We decided to join forces and join the Oasis organization. We started working with the team to incorporate TOSCA as an official part of OpenStack. During the Atlanta summit, it was decided to integrate the TOSCA project into the Heat project and now there is an official definition of the TOSCA 2.0 specification based on YAML. Our goal is to use the TOSCA 2.0 specification as the official templating language for Cloudify instead of our current TOSCA like DSL.

Integrating with OpenStack Heat

OpenStack is growing, and with this the services that are included as part of the core OpenStack project continue to grow up the stack. One of these services is OpenStack Heat, which started as the equivalent of Amazon Cloud Formation, and is becoming more of a general purpose infrastructure orchestration these days.

We realized that since Heat is closely integrated with OpenStack it will follow the OpenStack release cycles and API, and therefore would provide a useful tool for setting up the OpenStack infrastructure. The approach that we have taken with Heat is an “overlay approach”. This means that users can use Heat as they do today to set up their infrastructure. Then, Cloudify integrates with Heat in a way that will allow it to discover any resource that was provisioned through Heat, and ultimately add the monitoring, logging and software stack on top of that environment. 

Integrating with Neutron - Network Orchestration

Networking becomes a core service in any cloud deployment. Networking refers to an element such as security-group, private IP, floating IP as well as routers, DNS, vLans, load balancers, and such.

Cloudify 2.7 included basic integration with Neutron that allows users to attach floating IPs and set the availability zone configuration.

With Cloduify 3.0, we included support for all the networking elements and can now create vLans, security groups and more, as part of the application deployment.

Design for Scale

Application orchestration often requires intimate and continuous integration with the application so that it can detect failure in sub-seconds and take corrective actions as needed. This often imposes a more complex scalability challenge.

Cloudify 2.x uses a hierarchy of managers, where each manager controls 100+ nodes as a scaling architecture. One of the things that we have experienced over the last year is that there is a growing class of services such as  Big Data and NFV. With these services, a single service can be comprised out of more than 100s and potentially even 1000s of instances. This is why the manager of managers approach didn’t fit well to these use cases.

To enable scaling of 1000s of nodes per manager, we decided to use a message broker approach based on AMQP as well as to separate the provisioning, logging and real-time monitoring tasks into separate services that can scale independently. This allows us to also control the level of intimacy between the Cloudify manager and the application services. As an example, we can tune Cloudify to handle the provisioning and logging only, and use the real-time information that has already been gathered by the infrastructure. This is especially important in the context of OpenStack, as we expect that a large part of the metrics can be gathered through the infrastructure itself.

Integrating your DevOps toolchain of choice

One thing that has become clear through the various OpenStack statistics based on their annual survey, is that many of the OpenStack users are using a toolchain that is comprised of tools available through OpenStack infrastructure, as well as tools that are not specific to OpenStack such as Docker, Chef, Puppet, Ansible, JClouds and others.

Integrating all these tools as part of a deployment system becomes a fairly complex task that can takes weeks and even months.

We realized that if we want to provide a full lifecycle management of the application, it wouldn’t be right to force a particular stack or toolchain on a user, but rather provide an open pluggable architecture that could enable you to easily integrate your choice of tools into the same deployment.

With Cloudify 2.x we included support for Chef, Puppet and also had a cloud driver that allows us to integrate with various cloud infrastructures, as well as with tools like JClouds.

With Cloudify 3.x we created a more generic plugin architecture that allows users to plug in almost any element of their application, starting from the cloud plugin to configuration management, orchestration, monitoring etc.

To make this simple we included a Bash plugin that provides a simple framework for integration with external tools, just by pointing the plugin to the relevant Bash script.

Final Notes

In this post I thought sharing the story behind the scenes that we went through to make Cloudify OpenStack native, and the rationale and decision process behind them, would be relevant to other users that are considering their OpenStack strategy.  Especially for those still deciding how deep they should integrate with OpenStack.

The good news is that, it is quite possible to integrate natively with OpenStack without limiting yourself only to OpenStack. Having said that, the initial investment required to get there is quite big. I hope that this lesson from our own experience will save others time.

Going Hybrid Cloud with OpenStack

OpenStack Silicon Valley is just a few short weeks away and we are looking forward to talking OpenStack with the local community.

A main focus for Team Cloudify at the event will be around the topic of portability AKA hybrid cloud, not only between environments, but also between OpenStack and non-OpenStack environments. In July we announced the release of Cloudify 3, a complete rewrite and redesign of Cloudify.  This change provides an intelligent orchestration and management platform that is even more compatible with the open source world of OpenStack. The platform is designed to integrate with any tool chain and provide interoperability across stacks and environments.

Why the focus on portability? It is our core belief that for the foreseeable future, supporting hybrid clouds is going to be important for many OpenStack users. Here are a couple of reasons why:

1. Transitioning to OpenStack is a long journey - Many enterprises are based on VMware as their core infrastructure. Often, those organizations begin their journey to OpenStack by creating a dev environment and gradually moving more of their workload onto OpenStack as they become more accustomed to the new environment. This process typically spans the course of months and sometimes years. It is clear that during this period of transition, customers will continue to use vSphere and vCloud in parallel with OpenStack. For those users, having a common platform with which to manage their app deployment across those two environments can ease the transition and reduce the risk of hitting a roadblock if one of the environments does not meet their needs.

1. The public cloud market is going to be dominated by non-OpenStack clouds - While OpenStack is on a trajectory to dominate the private cloud market, the public cloud market is led by AWS with other major clouds emerging, such as GCE and Azure. Many organizations would not want to host their entire data center on a private cloud and would, therefore, need to take a hybrid cloud approach. In order to support hybrid cloud environments, an abstraction layer is needed to enable integration with clouds other than OpenStack.

While we strongly believe in the necessity of supporting hybrid deployments, the integration with OpenStack was a major consideration when redesigning the Cloudify platform. By switching to Python code, developing compatibility with Heat and supporting the TOSCA specification, Cloudify is now more than ever more deeply integrated with OpenStack’s core services.

This deeper integration with OpenStack technology makes Cloudify even more equipped to handle the task of multi-cloud deployment and management between OpenStack and not-only-OpenStack environments.

Open Cloud Deconstructed. What does it mean to be OpenStack Native?

OpenStack is just not like any other cloud.

OpenStack is an open source cloud and that makes it quite unique in comparison to many of the closed source public clouds generally available today, the most common being AWS, Azure GCE, and others.

The fact that it is open source, allows us to integrate with the lower level of the stack and know exactly how each element of the infrastructure is designed and implemented.

As with any open source project we have the option to contribute and influence how the OpenStack cloud ultimately shapes out.

Given the above background, when we're designing an integration with OpenStack, we should approach it differently than we approach other clouds. A more native integration with OpenStack involves two main angles:

• Integration with OpenStack core and infrastructure services, which usually includes this full OpenStack component list - as well as those that are continually being developed and added with every release.

• Follow the OpenStack design principles - OpenStack is written in  Python and uses a certain stack for messaging, database and such . To be OpenStack native you should align yourself with the OpenStack design principles as closely as possible.

Current options to be a part of the OpenStack ecosystem

In addition to the general guidelines outlined above, there are couple of additional options to be part of OpenStack. These options mostly fit in with those looking to take an OpenStack *only* approach, and require heavy involvement in the OpenStack community project.

• OpenStack Core - be a part of the OpenStack distribution

• OpenStack StackForge is an incubation framework that allows you to add new projects into the OpenStack ecosystem. StackForge uses the same set of tools and frameworks that are used to develop the core OpenStack services, and as such it allows smoother transition from an incubation project into the core services. That makes this option more relevant for projects that are likely to be included as an OpenStack core service once they've successfully passed the incubation stages.

• OpenStack Marketplace - The marketplace is a web directory that lists the various ecosystem players and their OpenStack solutions. Currently the marketplace is focused on OpenStack distros, training, consulting and drivers, and doesn’t cover solutions on top of OpenStack.

What’s missing?

While there is a fairly clear process, and tool chain for handling core and incubated services on OpenStack, these options fit mostly with services that are primarily designed to only run on OpenStack. What’s missing is a clear certification process that will allow users to find solutions in the ecosystem, outside of OpenStack, and find out quickly what their level of integration and support for OpenStack is. This should include information such as version support, and platform the solution was certified on, which of the OpenStack services are integrated, how much of the stack of the solution is compatible with that of OpenStack’s, and other such important information. A good and simple starting point could be the OpenStack marketplace, however this is just the first step. What’s needed is a set of frameworks similar to StackForge but much more permissive that will provide a means for validating the compatibility with OpenStack.

Why should you care?

As with any popular framework there are many tools that claim support for OpenStack, however in many cases the integration mostly includes basic integration with the OpenStack compute API (Nova). A good example for this is Cloud Foundry, or other tools such as Chef, Puppet, Docker, just to name a few. While these tools do have support for OpenStack, the level of integration can vary quite dramatically. Quite often, each of these  solutions comes with a parallel stack for handling security, monitoring and other issues, which isn’t compatible with the OpenStack infrastructure. As an end user, it is important that to be able to find out quickly which of these products comes with basic support for OpenStack, and which one takes a more native approach to OpenStack. There are many implications to the level of complexity and also to the level vulnerability of those services based on the support level.

OpenStack should attract more ecosystem projects rather than competing with them

In my opinion, there has been too much focus developing more alternative projects for the OpenStack project in comparison to the focus that has been placed on projects that exist within the broader ecosystem. The main reason for this, is the lack of native support for OpenStack. A good example of this, is project Solum and its ecosystem equivalents Cloud Foundry and OpenShift. 

I think that it’s time that OpenStack realign its focus from growing all around the stack, to supporting and attracting more ecosystem projects, that focus on adding native support to OpenStack. In order to do this, there needs to be more emphasis on creating certification, compatibility and testing projects. Additionally, the marketplace initiative needs to receive more attention and investment in order to make it more than a web-based directory service, similar to the AWS marketplace.

I would like to propose a discussion on that subject during the next summit in Paris. Do you think this is a good or bad idea? I’d appreciate it if you could share your thoughts.

The next big thing in big data: fast data

The big data movement was pretty much driven by the demand for scale in velocity, volume, and variety of data. Those three vectors led to the emergence of a new generation of distributed data management platforms such as Hadoop for batch processing and NoSQL databases for interactive data access. Both were inspired by their respective predecessors Google (Hadoop, BigTable) and Amazon (Dynamo DB).

As we move to fast data, there’s more emphasis on processing big data at speed. Getting speed without compromising on scale pushes the limits in the way most of the existing big data solutions work and drives new models and technologies for breaking the current speed boundaries. New advancements on the hardware infrastructure with new flash drives offer great potential for breaking the current speed limit which is bounded mostly by the performance of hard drive devices.

Why using existing RDBMS, NoSQL on top of flash drive is not enough

Many of the existing databases — including more modern solutions such NoSQL and NewSQL — were designed to utilize the standard hard drive devices (HDD). These databases were designed with the assumption that disk access is slow and therefore they use many algorithms such as Bloom filters to save access to disk in case the data doesn’t exist. Another common algorithm is to use asynchronous write to commit-log. A good insight into all the optimizations that are often involved in a single write process on NoSQL databases is provided through the Cassandra Architecture. Let’s have a look at what that entails below:

 

The Cassandra write path

 

What happens when disk speed is no longer the bottleneck?

When the speed of disk is no longer the bottleneck, as in the case of flash devices, then a lot of the optimization turns into overhead. In other words, with flash devices it will be faster and simpler to access the flash device directly for every write or read operation.

Flash is not just a fast disk

Most of the existing use cases use flash devices as faster disk. Using flash as a fast disk was a short path to bypass the disk performance overhead without the need to change much of the software and applications. Having said that, this approach inherits many unnecessary disk driver overhead. So to exploit the full speed potential of flash devices, it would be best to access the flash devices directly from the application and treat flash devices as a key/value store rather than a disk drive.

Why can’t we simply optimize the existing databases?

When we reach a point in which we need to change much of the existing assumptions and architecture of the existing databases to take advantage of new technology and devices such as flash, that’s a clear sign that local optimization isn’t going to cut it. This calls for new disruption.

The next big thing in big data

Given the background above, I believe that in the same way that demand for big data led to the birth of the current generation of data-management systems, the drive for fast data will also lead to new kinds of data management systems. Unlike the current set of databases, I believe that the next generation databases will be written natively to flash and will use direct access to flash rather than the regular disk-based access. In addition to that, those databases will include high performance event streaming capabilities as part of their core API to allow processing of the data as it comes in, and thus allowing real-time data processing.

 

Fast data in the cloud

Many of the existing databases weren’t designed to run in cloud as first-class citizens and quite often require fairly complex setup to run well in a cloud environment.

As cloud infrastructure matures, we now have more options to run big data workload on the cloud. The next generation databases need to be designed to run as a service from the get-go.

To avoid the latency that is often associated with such a setup, the next generation databases will need to run as close as possible to the application. Assuming that many of the applications will run in one cloud or another, it means that those databases will need to have built-in support for different cloud environments. In addition, they will need to leverage dynamic code shipping to pass code with the data, and in this way allow complex processing with minimum network hops.

In-memory databases and data grid are the closest candidates to drive next-gen flash databases

RAM and flash-based devices have much more in common than flash and hard drive. In both RAM and flash, access time is fairly low and wasn’t really the bottleneck; in-memory databases use direct access to the RAM device through key value interface to store and index data.

Those factors makes in-memory based databases more likely to fit into the next generation flash databases.

The combination of in-memory databases and data grid on top of flash devices also allows the system to overcome some of the key capacity and cost per GB limitations of an in-memory based solution. The integration of the two will allow an increase in the capacity per single node to the limit of the underlying flash device rather than to the limit of the size of RAM.

The overall architecture of the integrated solution looks as follows (Source: Gartner):

 

 

As can be seen in the diagram above, the IMC (in-memory computing) layer acts as the front end to the flash device and handles the transactional data access and stream processing, while the flash device is used as an extension of the RAM device from the application perspective.

There are basically two modes of integration between the RAM and flash device.

LRU Mode – In this mode, we use the RAM as a caching layer to the flash device. The RAM device holds the “Hot,” i.e. the most recently used, and the flash device holds the entire set.

Pros: Optimized for maximum capacity.

Cons: Limited query to simple key/value access

Fast Index Mode – In this mode, all the indices in RAM and the data in flash holds the entire set.

Pros: Supports complex querying, including range queries and aggregated functions.

Cons: Capacity is limited to the size of indexes that can be held in RAM.

In both cases, the access to flash is done directly using key/value interface and not through disk drive interface. As both RAM and flash drives are fairly fast, it is simpler to write data to the flash drive synchronously and in this way avoid potential complexity due to inconsistency.

It is also quite common to sync the data from the IMC and flash devices into an external data store that runs on a traditional hard drive. This process is done asynchronously and in batches to minimize the performance overhead.

Direct flash access in real numbers

To put some real numbers behind these statements, I wanted to refer to one of the recent benchmarks that was done using an in-memory data grid based on GigaSpaces XAP and direct flash access using a key/value software API that allows direct access to various flash devices. The benchmark was conducted on several devices as well as on private and public cloud-based services on AWS. The benchmark shows a ten times increase in managed data without compromising on performance.

 

 

 

For full details of the benchmark, see the following white paper:

XAP MemoryXtend -Massive Application Storage Capacity for Real-Time Applications

 

 

 

Network Automation and Orchestration with Openstack and Cloudify

In my previous post I discussed the evolution of networking in the cloud - from commodity hardware to the cloud, and the need for the virtualization of network functions, and the different drivers for change.  In this post I will focus on orchestrating your networking through OpenStack’s Neutron, its challenges and potential.

Bringing the Network and Applications Together

So, as I mentioned in my previous post, when OpenStack introduced its networking API, known as Neutron, this provided a new way to set up networking services and configurations such as routers, subnets, floating IPs, security groups, and more through a simple API.  Until now we used to think of network configuration and application deployments as two separate things. This also goes back to the way most of our IT is structured.  We have different groups for networking operations and for applications’ operations and in most cases they have a fairly loose interface between one another.

As with DevOps once we open our infrastructure layers through APIs we can break those traditional boundaries between networking and apps. In a networking context, this means being able to create and configure our network as part of the application deployment.  To understand what this really means, let’s examine what the typical steps that we need to follow to set up networking for our application.

Typical Network Configuration:

1. Set up admin network for our application monitoring and control services
2. Set up application network to allow internet access to the relevant machines
3. Set up the security group per application tier (open only the ports and IPs relevant for that tier)
4. Set up floating IPs where needed
5. Set up a load balancer and configure it to route traffic to our web containers

Adding Continuous Availability

There are a set of additional network configuration steps that needs to be considered when we think of high availability.

Such steps often include:

1. Partitioning our services into separate availability zones and regions
2. Setting up connectivity between the availability zones and regions (which often run on separate networks)
3. Switching back and forth between each availability zone in case of a failure

Orchestrating and Automating  Network Configurations as a Part of Application Deployment

Today most of the steps required to set up our network are often done manually and externally to our applications. Now that we have API access, and our networking is basically software-driven, we can automate this entire process.  As many of those steps tie in more directly to our application services, we need to include them with our application deployment.

Network Orchestration Through a Simple Blueprint

With an orchestrator, users can define their own gateways, routers, and security groups through simple software configuration files, or blueprints, that are written in a language that the orchestrator (software operator) can understand.

With the latest Cloudify 2.7 release we added the ability for users to set up their network configuration as part of their application deployment blueprint. This allows users to set up security groups, separate networks, isolate the management network from the application’s network, define whether a certain piece of your application will run in the same availability zone or not, control if the storage will be close to the application geographically (i.e. the affinity rule).  Read about Network Automation to see how such a blueprint should look.

Final words

Bringing the application and networking together provides an easy way for users to capture the value of newly available network capabilities without having to become a network operator themselves. This significantly simplifies the complexities involved with exposing networking elements to the user by abstracting the networking piece of our deployment into an application blueprint.

In addition, the fact that we can setup our network configuration as part of our application deployment allows us to harden our security to match our application topology exactly . For example, we can choose to open only the necessary nodes to internet access, setup security groups dynamically to match the ports and IP addresses that are allowed to access our service and other such customizations. In this way, setting up an application and its network configuration is infinitely simplified for an application developer.

What is NFV?

I first heard the term NFV a year ago as part of the writeup that I was doing with Alcatel Lucent on CloudBand titled Carrier Grade PaaS covering our collaboration. Ever since that time it looks like the term has risen in popularity with every network provider or infrastructure provider now launching a new NFV initiative.

 

As with any new hype, the term NFV quickly became overloaded and confusing, so I thought it would be beneficial to try and clarify it a bit.

 

What is NFV (technical definition)?

NFV stands for Network Function Virtualization - it basically means that your routers, load balancers and firewalls that are currently shipped in boxes will become a virtual entity rather than a physical box. As in the case of compute or storage virtualization, the process of virtualizing network functions is quite the same; we take each function (box) and turn it into a software function that can run on any given box. The decision of which box is going to serve each function can change at runtime based on the desired SLA and would not involve changing the software piece.

What does that really mean?

For most people, network functions do not mean a lot. In addition, the definition that I found over the internet for NFV for the most part comes from people who often tend to speak in a language that only a small community can really understand.

 

To understand what NFV really means we need to take a step back and look at the overall change that the IT industry as a whole is undergoing.

The IT industry is going through a big industrialization change that is very similar to the change that the car industry underwent when Ford came out with Model-T. The change with the Model-T car wasn’t in the making a faster engine, quite the contrary, the Model-T was inferior to other cars that were produced in those days. The industrial revolution with the Model-T was in the ability to produce cars in mass production. This revolution changed the entire automotive industry that propagated to other industries, as well, from consumer electronics through the food industry, which followed the same principles and adopted the same mass production manufacturing principles.

 

The IT industry is going through a fairly similar change - until now we used to build our IT manually, and we’re now shifting into mass production of IT, as well.  In this analogy cloud is the manufacturing floor, and DevOps is the process for optimizing and automating the production pipeline.

 

How IT industrialization is related to NFV?

The carrier industry today relies on a highly customized infrastructure, no one carrier infrastructure runs in the same way as another carrier infrastructure.  Once upon a time carriers saw this as an advantage and ended up with layers and layers of fairly proprietary and costly setups for running their business.

 

As the competition in this industry became tighter this operational model becomes not just less economical, but a huge threat, as it limits the speed in which carriers can adapt to new economic and market changes.  This already has resulted in their loss of ground quite rapidly to competition from internet shops such as Google, Amazon, Microsoft, and others.

 

Carriers realized that in order to survive in this new world they needed to reduce their operational costs and increase the speed in which they can introduce new services, as well as scale their business.

 

This is where NFV comes into play.  NFV is basically a better operational model for running the carrier backbone.  Instead of a highly customized and costly backbone, we’ll use a commodity based infrastructure, and use opensource frameworks and with that we can innovate faster, scale and all this at a much lower cost.

 

What NFV is comprised of?

NFV isn’t a standard (yet) or a product. Even though there are various bodies such as ETSI that try to define standards for NFV, it is very unlikely that we’ll see any real standard emerging from this sort of initiative.  This is simply due to the fact that standards often emerge when an industry reaches a certain level of maturity and we’re just not there yet.

 

At this point it is a set of “de-facto” standards bred from best practices mostly from the cloud providers that proved to be able to deliver a more efficient and agile operational model for running their infrastructure.

 

Quite often that includes the following core components:

 

1. Cloud-based infrastructure - OpenStack is currently the most popular choice for this purpose.

2. Software-defined network functions - This is a mix of the existing network functions provided as software packages, as well as new purely open source players that are making a new entrance into this world.

3. Orchestration engine - Responsible for provisioning of the network functions on a cloud-based infrastructure. TOSCA is a standard orchestration language defined by the OASIS organisation that provides a standard modeling language. Projects such as OpenStack Heat, as infrastructure orchestration in combination with Cloudify as an application orchestration tool are a good reference for this.

4. Analytics engine -  Analytics engines are basically the feedback loop. They are an essential piece to measure which part to measure whether our services meet their desired SLAs and also as a means to analyze and optimize workloads. In the context of NFV, where many of the insights and decisions need to happen in real-time, the analytics engine will be heavily real-time based. A real time analytics engine that is more specific to the operational monitoring domain is Reimann.

What makes NFV different than any other cloud-based infrastructure?

For the most part network functions are not different than a database or any other software function that isn’t a defined as a network function. Having said that there are number of characteristics that are more specific to network functions such as:

• Highly distributed deployment - Carrier data centers tend to span across multi-site deployments

• Deterministic latency and performance - Network functions are more sensitive to latency and non-deterministic behavior that can be quite common in virtual data centers.

• Support for third-party virtual appliances - Many of the network functions are packed as VMs. Those third-party VMs are mostly treated as a blackbox and can be accessed only through custom interfaces that are specific to the network function. Managing those functions can be fairly different than that of the other software services, as a result. The main difference, though, is that most management systems install an agent to control each managed service. In the context of network functions, these assumptions are no longer valid, and therefore managing virtual appliances can be done only remotely and not through a local agent.

• Support for legacy network services - Many of the existing network functions were written in a pre-cloud world, as such they were built with specific assumptions that the high availability and scalability model, as well as the configuration model that are usually very much human-driven. Changing all that in one day isn’t going to be realistic, and therefore, there needs to be a more gradual path to transition those services into the new world or even replacing them with a more modern infrastructure.

• High Degree of Security - Carrier networks assume a high degree of isolation on the network level. This often maps to a daily sophisticated network setup of VLANs and a separate network hierarchy that is not yet supported by most of the existing cloud infrastructures. 

The Role of Orchestration in NFV

In a pre-NFV world setting, provisioning the carrier infrastructure was fairly human driven.  In an NFV world those human operations need to be automated through software. In that context the orchestrator is a software defined operator and it’s the piece that manages the deployment of the network function in the right location and hardware. It is also responsible for orchestrating the network firewalls, routers and such, to fit a specific service deployment and security constraints. Like the human operator it is also responsible for continuously monitoring the deployed services and ensuring that they meet their desired SLA.

Final words

NFV is part of a broader shift toward industrialization of IT. In this particular case NFV is simply a way to make the operational model of carriers more efficient by adopting similar best practices and tools that have already proven themselves at scale by the major cloud providers such a Amazon, Google, Microsoft and others.

On the business side it has disrupted the entire networking industry that used to sell proprietary boxes. In the new virtualized world, boxes have become commoditized and the software on top becomes the main play. This opens up the door to new players that will have a pure software solution that will be specifically designed for a virtualized setup.

As with any disruptive force we will see big players that find it difficult to adapt to this change phased out to make room for new players who will take over, in a similar fashion to the way the internet wave brought Google, Amazon, PayPal and other players who were completely new commoners at the time and became the new market leaders.

On the technical side of things virtualizing the network function is only the first step. In order to experience the full potential of a virtualized carriers, we need to have the entire deployment and management completely automated. What we are missing in this picture is the software equivalent of the human operator, which is also known as the orchestrator. The orchestrator will become the major piece in putting all of this together.   

In the next post I’ll describe in more depth the role of an orchestrator in the context of NFV.