Nati Shalom's Blog

Cloudify Momentum Accelerates, Exceeding Second Quarter Projections

Last quarter we launched Cloudify 4, a major milestone transforming Cloudify from a DevOps tool to a complete Application and Network Management Platform. This is highlighted by the addition of an enhanced customizable portal, based on React, with an emphasis on multi-tenancy and security in addition to our continued support for container orchestration through integrations with the Kubernetes, Docker Swarm and Mesos platforms.

Version 4 was part of a more significant move to introduce a whole new thinking of how next generation cloud application management should be built. As part of that play, we also launched a new website (cloudify.co) and branding which reflects this change.

Cloudify 4 played a big part in both the previous and current quarters’ wins, the current exceeding even last quarter’s results and target revenue by 60%.

In addition, there was a GlobalData report which puts Cloudify as the leading open MANO orchestration framework with the largest number of deployments amongst 13 vendors who were evaluated in this report. This comes on top of other mentions in this Light Reading report as well as an OPNFV eBook covering their vCPE and vIMS (Clearwater) use case powered by Cloudify orchestration, which puts Cloudify as the leading reference implementation for open NFV orchestration.

In this post, I wanted to highlight the key achievements from Cloudify second quarter results of 2017 and how they fit into our overall strategy.

Key Achievements

Open vCPE Wins

Last quarter we launched a new Open vCPE solution which introduced an orchestration-first approach to NFV. The orchestration-first approach aims to address one of the biggest challenges for providers with the current infrastructure-first approach to NFV - namely, the ability to integrate with existing networks. According to a recent IHS survey this has been a concern of 52% of the carriers.

Since we launched this innovative solution, Partner Communications (previously Orange Israel) went to production and launched their new V-NET product using Cloudify as orchestrator and VMware as the NFVI.

SD-WAN Screenshot from Partner Communication’s new VNet offering

More details on this specific use case can be found in this post: How Cloudify Enabled a Telco to Create a Fully Automated, Managed CPE Service - A Use Case

This quarter we added another major European carrier into this program, a major win over both established providers and other open source alternatives. This deal also included a partnership with one of the leading system integrators who are responsible for the end-to-end delivery of the solution.

The combination of Cloudify as the Open vCPE framework and system integrator, responsible for the end-to-end solution, is a winning combination that could help carriers get the best of both worlds. On one hand this solution could be easily tailored to the carrier’s infrastructure and needs, and on the other the carrier receives a full turnkey-like solution similar to others, such as Cisco, but without the lock-in and significantly more economical.

Cloud Native VNF Win

One of the challenges that carriers face today when launching a new NFV project is that the VNFs (switches, routers, firewalls, etc) were not designed to run on a virtual network.

In December of last year, we launched a new Cloud Native VNF program followed by a new business model offering targeted specifically for VNF providers.

I’m happy to say that this year we signed an OEM partnership agreement with MetaSwitch, one of the first movers in the Cloud-Native VNF space and established partners of ours for the last couple of years. Metaswitch was also selected by Swisscom as their PoC winner in the NFV/SDN “Call For Innovation” for their new product Cloud Native Network Evolution in December 2016.

A great example of this partnership can be seen in this MetaSwitch demo.

Cloud Native ClearWater IMS using Cloudify

DevOps Automation / Multi-Cloud Win

During a joint event with Atos and Société Générale in Paris, the two companies presented their journey toward a DevOps and Multi-Cloud strategy and why they chose to base their strategy on TOSCA, using Alien4Cloud and Cloudify.

I think the following slides from the event speak for themselves.

Why Atos chose Cloudify

How Cloudify fits with Atos Aliend4Cloud

This win adds to other enterprises that are using Cloudify for multi-cloud orchestration.

More about the event can be found in this post.

Edge Computing Win

One of the most promising opportunities that we closed this quarter happens to be in the area of Edge Computing. Edge computing is the movement from the central clouds we all work with today, to a highly distributed cloud in which airplanes, ships, or cars and virtually any other IoT device becomes a moving data center of its own.

Edge computing pushes cloud management as we know it to the limit with requirements such as low footprint, dealing with unreliable networks, and extreme scale in a highly distributed network environment. In this specific use case, Cloudify is utilized as a central manager as well as on the edge where the customer is running Kubernetes on bare-metal and controlled by Cloudify.

I’m super excited about this use case as it fits perfectly with the flexibility of Cloudify to allow nested and distributed management all while simultaneously leveraging low-footprint orchestration through the use of ARIA on the edge device.

This allows a federated management model where we can take advantage of the power at the edge to have more autonomous management capabilities.

NFV Lab Win

We launched the first NFV Lab last year, and since then, many participants have joined the lab.

In this quarter we extended the lab to a full on-demand PoC environment and it is currently being successfully utilized by one of the Top 5 global carriers.

This was a great example showing how we could take a complex use case such as NFV IMS and turn it into a simple user experience by allowing access to a fully operational lab in a matter of minutes.

It was also a great example of how an open, bottom up strategy could become applicable in a fairly conservative industry. The move to a more open approach is also changing the way carriers adopt new technologies. Developers are becoming more influential in the decision-making process and they are influenced mostly by adoption patterns in the open source community.

This is not just a technology shift, but a huge cultural shift, as I outlined in an earlier post called “What Developers Want From Their Technology (But Mostly Cloud).”

We’re now working on a new version of our lab offering to allow an even simpler experience by turning it into a complete self-service experience.

ONAP Wins

Traditionally new technology adoption takes years to succeed, and requires close collaboration among those producing network technology and those consuming it. In the past, standards development organizations (SDOs) have played a critical role in offering a forum for discussion and debate, and well-established processes for systematically standardizing and verifying new technologies.

In a software driven world, innovation can’t wait for SDOs to deliver hard requirements that can’t fit an ever changing reality. As a result, industry has been increasingly turning to open source communities for implementation expertise and feedback.

ONAP brings together Open ECOMP and Open-O projects as a comprehensive platform for real-time, policy-driven orchestration and automation of physical and virtual network functions that will enable software, network, IT and cloud providers and developers to rapidly create new services. By consolidating member resources, ONAP will deliver a unified architecture and implementation, with an open standards focus, faster than any one project could on its own.

On June 8th ONAP announced that Accenture, CertusNet, Coriant, Juniper Networks, Mavenir, Mirantis, PCCW Global, Red Hat, VEON and Windstream joined as new members to contribute to the open source framework for network automation. These are added to the current members, which include a broad mix of major vendors, service providers, system integrators and consultancies: Amdocs, ARM, AT&T, Bell Canada, BOCO Inter-Telecom, Canonical, China Mobile, China Telecom, China Unicom, Ciena, Cisco, Cloudbase Solutions, Ericsson, Huawei, IBM, Intel, Metaswitch, Microsoft, H3C Technologies, Nokia, Open Networking Foundation, Orange, Raisecom, Reliance Jio, Tech Mahindra, VMware, Wind River and ZTE.

Cloudify is a founding member of OPEN-O, and ARIA is the common TOSCA parser in OPEN-O. In addition there are several models within ECOMP which are using the community version of Cloudify. Cloudify is a platinum founding member of ONAP, committed to open-source and continuing our work on networks powered by open code, standards and orchestration.

ONAP is committed to model-driven orchestration based on TOSCA that crosses industry borders - vendors and service provides. When ONAP wins, the industry WINS and Cloudify WINS!

Our participation in ONAP and our continues and consistent commitment to open-source and open-standard has played a major factor in winning the large carriers deals. Cloudify also organized an ONAP Launch Event at the Open Networking Summit in April 2017.

The industry shift towards open models such as ONAP is becoming more evident and recent RFx are starting to indicate that the ETSI MANO Standards and Specs for Service Orchestration is not considered as a viable way forward, It was also indicated the vendors in this space are encouraged to look at alternative approaches such as ONAP, OSM or Open-O that overcome the ETSI MANO shortcoming.

Final Notes

One of my biggest frustrations with NFV is that, while it was clear that Openness Is the True Path of NFV, most of the big carriers got cold feet as soon as they had to make a final and risky decision regarding their NFV solution and often went back to their comfort zone, of working with traditional vendors. Not surprisingly many of those projects failed, sometimes in a big way. One of the more notable example of this is how Telefónica Ditched HPE as Virtualization Lead.

What makes me happy now is to see that more carriers are starting to realize that the path to openness is through partnering with players who have it already in their DNA, throughout the products they build, and business strategy they execute. The win with Partner Communications and the fact that they were able to build a vCPE and SD-WAN solution that is not just open, but far more innovative than most of the regular NFV projects at a fraction of the time and cost, has paved the way for others to follow. I’m now starting see the beginnings of a trend in which more of those players start to adopt true openness as part of their NFV strategy.

I’m also starting to see that, with the growing maturity of network virtualization, there are more use cases and demand in other areas such as Open vCPE, Cloud-Native VNFs, and network virtualization, in which that plays a key role, as I outlined in a new whitepaper on the subject.

As more enterprises adopt a multi-cloud strategy, it becomes clear that network virtualization is going to play a key role in how we manage and secure our multi-cloud networking in enterprises as well, not just in Telcos. This will be realized as part of the regular DevOps process which means that we'll see a more application-driven networking patterns similar to the way other infrastructure resources are being managed.

The key drivers and use cases for edge computing

Application-driven networking also has the potential of completely changing the way we think about network security. Rather than having static VPC connections or firewall rules, we can turn them on only when they are needed and remove them when the application no longer needs them. We can also think of taking proactive measures such as closing the network for a specific tier of the application or even moving the entire application to a completely different datacenter when it’s under attack.

Edge computing definitely stretches this vision out in multidimensional paths as we turn every edge into a network of many miniature clouds that need to be coordinated to deliver a specific service. Edge computing also has the potential to bring the cloud strategies of enterprises and carriers much closer together both in the architecture and in the kind of application and services that would run on that environment. So, while this is still very early days for edge computing, I feel that the move to the edge will happen faster than anything else that we have experienced so far.

Building Large-Scale Services with Micro-Services and TOSCA

A decade ago, large-scale services were an edge-case scenario, often designed as one-off systems.

Then, in the mid 2000s, the financial services industry began using algorithmic trading and automated trading systems, which required scalability under extreme low-latency in a stateful transactional system. It quickly became clear that the classic tier-based architecture which was popular at the time was insufficient for this use case. That led me in 2007 to write about one of the first alternative architecture patterns for building large-scale systems, also known as Space-Based Architecture (SBA).

Today, applications like social networks, networking services and IoT have made large-scale systems the norm rather than the exception. As a consequence, we need a more systematic model for designing and implementing large-scale, SBA-type systems. Microservices architecture, in conjunction with the TOSCA modeling language, offers an excellent solution.

Let’s explore how this solution works.

Model-driven microservices

The key concept behind a microservices architecture is to break large systems into a set of loosely coupled autonomous services. The TOSCA language lets us describe the domain model behind each individual service, whether it is based on containerized or non-containerized components.

Model-driven design is an architectural concept in which the application or service domain model is defined first, and the rest of the interaction with the service is driven from that domain model. Take a look at this graphical illustration:

The building blocks provided by TOSCA to model this service include...

Node: Analogous to class in other object-oriented languages, a node can define specific interfaces and inherit others. TOSCA comes with a set of built-in nodes, and open source orchestration tools (i.e., Cloudify and ARIA) add a set of built-in node libraries that cover popular cloud resources like AWS, OpenStack, VMware, GCP and Azure.

Relationship: This defines the relationships among nodes.

Group: This feature packages nodes in “pods” and is used to manage the lifecycle of the group as a single unit.

Adding Custom Resources as Node Types: This capability adds custom nodes through imports of other libraries, as is done with similar languages.

Remote Execution Through Workflow: Workflow defines business logic, and TOSCA defines implicit and explicit workflows. Implicit workflows (i.e., install) execute the nodes’ lifecycle operations according to their dependencies. Explicit workflows provide a way to interact with the service nodes in an abstracted manner (i.e., upgrade).

Discovery Through Relationship: Relationships let one node discover other, related nodes.

Service composition pattern

So far we discussed how we can use the TOSCA language to describe the domain model of a particular microservice. Open source orchestration platforms that use TOSCA introduce a new pattern known as the service-proxy in which we can describe a service as a single node type to the other related service. The service-proxy pattern uses the service input/output as a generic way in which a service interfaces with the outside world.

The example below defines a composite service comprising a set of node types with two special service-proxy node types. The service-proxy node type is responsible for representing the output properties of external services as local properties of that specific node type. It is also responsible for either discovering or launching the related services (Microservice 1 and Microservice 2):

This lets us leverage existing TOSCA features for modeling the internal components of a specific service and also the relationship with external services.

Deployment-update is a feature in the open source orchestration approaches noted above that lets us update a service topology after it has been deployed. We can update existing services after deployment with new nodes, and we can change existing nodes.

In the example below, we use the service-proxy pattern in conjunction with deployment-update to add a new service to an existing composite service. We use deployment-update to add a new service-proxy node and point it to the new microservice.

Container support with cluster management options

TOSCA can be extended to support integrations with popular cluster management tools like Docker Swarm, Docker Compose, Kubernetes and Apache Mesos. TOSCA can manage container infrastructure and orchestrate services running on container platforms.

The need for generic service orchestration

Most large-scale systems never fit into a single platform and quite often come as a set of composite services that are linked together. To automate the processes of managing and delivering those applications, we need a generic service orchestration that is not bound to any of the underlying platforms but instead can integrate with many of them in order to fulfill cross-domain automation tasks.

This generic services orchestration doesn’t have to be an alternative to the more purpose-built orchestration (a.k.a. domain orchestration). It could actually use the domain-specific orchestration as an abstraction layer that hides the complexity of managing a particular service on a given target platform. The built-in neutrality and openness of TOSCA makes it an ideal service orchestration modeling language.

In summary

The explosive growth of IoT and distributed networks—and the degree to which they are now woven into the fabric of our lives—has created a dire need for large-scale computing systems that can scale, offer low-latency and high-availability, and are highly automated and manageable. A microservices architecture built with TOSCA is an excellent way to meet these objectives. This approach not only offers all the benefits afforded by microservices (scalability, fault isolation, flexibility, etc.) but also enables integrations with cluster management tools, container platforms and popular cloud resources like AWS, OpenStack, VMware, GCP and Azure. Best of all, the capability to build large-scale systems with microservices and TOSCA is readily available through the open source orchestration tools.