What does it take for network automation to succeed?

Previously published on RCR Wireless

Author: Jo Hicks, Marketing Specialist, EXFO

October 24th 2019

 

Over the last couple of years, the drive toward network automation has gained momentum. Today it’s widely acknowledged that automation is essential to the success and effectiveness of future network operations—and it’s easy to see why. Communication service providers (CSPs) have reached a tipping point. The modernization of network infrastructure needed to support the ever-increasing demands placed upon the network—from streaming, applications’ usage, social media and more—mean that CSPs will have to scale-up their efforts to create and maintain networks that are more agile and efficient than ever.

In the face of this greater complexity, service providers will be increasingly reliant upon automation to support both service agility and quality and minimize operational costs. A detailed look at the benefits of network automation are compelling. Improved efficiency by the removal of human labour to perform time-consuming, routine tasks, reduced probability of errors by the elimination of manual tasks that can lead to human error and the over-reliance on individual expertise rather than established standard operating procedures, can both improve the integrity and stability of the network. Network stability is critical for many reasons. When it takes on average 6x more time to resolve vs. prevent issues, it pays to be proactive in finding ways to guarantee stability—to the extent that’s actually possible ^(1). Plus, automation means greater speed and agility for companies which translates into lower operational expenses and increased revenues attained from greater overall efficiency.

As discussed above, automation promises to deliver many advantages. So what’s the holdup? Maybe it’s not the promise, but the process involved with how to get automation done right that’s causing operators to hit the brakes on fully-fledged network automation. For telecom automation to take flight, it must be implemented the right way—by leveraging all available and up-to-date information contained within multiple OSS and BSS databases. What’s needed is an end-to-end view of network assets that delivers 100% visibility, making it possible to pinpoint areas of weakness in the network that might result in sub-optimal customer experience. And since only 15% of CSPs have a fully integrated view of network and service topology, they clearly need a solution to help regain control of their network and bring end-to-end dependencies into focus ⁽²⁾.

Automation aided by reliably-correlated topology

For a number of years, EXFO Ontology’s engagements in the telecom sector have involved the improvement of infrastructure data integrity and the determination of network dependency mapping for the advancement of an important (and ambitious) subset of the overall E2E set of automation projects. Projects like: automated cross-domain provisioning, service impact analysis, change impact analysis, and of course, automated troubleshooting.

If customer-to-infrastructure dependency analysis is readily available and correct then you can achieve proactive assurance more easily. This is where real-time, active topology comes into play. The presence of a real-time, cross-domain network and a live service topology model covering both legacy and next generation networks and services is the key to both a successful SOC and automation stack. Successfully implementing large-scale automation is tied directly to an accurate understanding and description of the network, service and customer infrastructures. It’s the glue that binds the customer’s services to the ever-changing shared infrastructure. And as a result, service impact analysis, common cause analysis and change impact analysis become viable and are easily deployed. Machine learning algorithms can become a lot more accurate if populated with richer data; real-time topology enriches and contextualises the mountains of network data ingested (faults, test data and logs) in relation to services and customers. Real-time topology also greatly improves fault deduplication, and machine learning algorithms can help to facilitate automation.

Several EXFO Ontology projects currently in deployment are helping our customers lay a solid foundation for the safe and effective adoption of automation. For instance, EXFO is currently working with a Northern European Tier 1 network operator in rolling-out the application of real-time topology for automating the identification of common causes behind network outages. Large-scale outages and the protracted diagnosis of what’s gone wrong faced by the Tier 1 network operator (and other operators) can take days, or even weeks. This is extremely disruptive. The traditional manual method of diagnosing problems always involved gathering the same group of over-worked SMEs in a war room environment to identify and resolve issues. Fundamentally, the only option available was to determine which individuals using the telco’s services were experiencing a degradation and manually trace back from this starting point to locate the offending network equipment. Using human logic to address the outage translates into slow response times, and from there, increased operational costs. Instead, utilizing this data and applying it to a real-time topology engine changes the game!

The automated common cause analysis solution is designed to consume performance and real-time data across a range of networks: enterprise, broadband, mobile backhaul and now 5G. Real-time topology allows us to track and understand the configuration of the entire network across all sources. From the real-time telemetry data, a ‘symptom set’ is created from which it is possible to see common and shared performance issues that appear across a large set of end-point entities within the infrastructure. Common, repeated and simultaneous symptoms imply a common and singular cause, this means that the symptom set can be used to automate a topological analysis algorithm called “common cause analysis”. After a symptom set is created and fed to the EXFO topology engine, the system uses it’s cross domain topological model to find the unique set of network nodes upon which all entities in the symptom set share a dependency.

In another recent example, EXFO worked with a customer to facilitate inventory federation. A leading Tier 1 North American operator needed EXFO to help them make sense of the topology held in numerous inventory systems and networks it had inherited following several major acquisitions, and to automate provisioning across its infrastructure successfully, all whilst maintaining excellent customer experience.

The Tier 1 operator suffered from the perennial problem faced by all major telecom networks; following several very large acquisitions, they needed to tackle the enormous challenge of managing and consolidating six networks and eight inventory systems, as well as automating provisioning across all these new systems.

Early attempts did not achieve accurate cross-domain linking and resulted in a very high fallout rate for automated provisioning attempts. The lack of a complete and well-linked model meant that the process would periodically route services over external (and expensive) networks when internal resources could have been used. Due to the lack of comprehensive automation and the magnitude of the challenges faced, staff were still required for the task of manual provisioning.

Accordingly, despite the fact that the company was investing in building an automation capability, the much-needed initiative was in danger of being abandoned due to lack of tangible progress. The company called on EXFO to address their business process architecture, ensure successful provisioning over their incredibly complex network and continue to deliver the excellent customer experience that the CSP was known for.

EXFO’s solution was to create and maintain an inventory federation (a cross-domain topology) by dynamically linking silo networks. A workflow orchestrator places a service design request to the EXFO Ontology based Federated Inventory, the system responds immediately with an ordered list of complete and suitable service design alternatives, starting with the design that is the best fit for the required service parameters. The service path designs are then communicated back to the orchestrator.

This deployment showed that by removing barriers between silo network data and creating a singular, logical reference model to link them together it was possible to quickly increase accuracy, reduce hardware costs, and perhaps most importantly, lower fallout rates thereby creating the optimum conditions to accomplish meaningful automation.

Right now, EXFO is putting automated operations at the centre everything we do. This is because we recognize that automation will be vital to keeping up with industry innovation and customer expectations. Benedict Enweani, EXFO’s Director of Applications and Analytics and one of the original architects of EXFO Ontology having built the solution from the ground up, is excited about the bright future for both EXFO and automation. “Because customer services ultimately depend on the capability of shared, multidomain end-to-end network connectivity the ability to understanding real-time topology has become a key automation prerequisite,” said Enweani. “CSPs who invest in their infrastructure now—with a view to gaining visibility into their networks and an in-depth understanding of the data, as well as dependency relationships—will reap the rewards in the near-to-long term as the benefits of automation come into focus. Operators need technology that can achieve these goals if they want to offer fully-assured services in the highly-automated future”.

Footnotes 1 and 2 : Statistics taken from Heavy Reading Custom Survey for EXFO, 2019-07 Auto-Assurance Survey Results Summary