Are You Still Testing to RFC 2544? Really?

Have you heard about RFC 6815? It is an application statement for RFC 2544. And I can already hear you saying: “An RFC that talks about another RFC, how exciting!”

Well, all I can add to that is that sarcasm gets you nowhere, at least from an Ethernet service activation perspective...

Now, do not touch that mouse! Let’s start with a bit of background on what an RFC is, followed by some information on both RFC 2544 and RFC 6815. RFC stands for Request for Comments, and consists of a recommendation produced by the Internet Engineering Task Force (IETF). The IETF website indicates that the IETF standardizes all the protocol layers in between the transmission hardware and the application layer protocols, from IP itself up to general applications like e-mail and HTTP. The IETF is a forum where individuals ranging from service providers to vendors and scholars exchange on future protocols and standards to advance communication technologies.

Now that we have defined RFC and put it into context, let's start at the beginning, and then go back to address my initial question.

RFC 6815 is a very interesting publication, as it informs RFC 2544 users that RFC 2544 was not created with the prospect of turning up services, but to benchmark network elements, as stated in its title, “Benchmarking Methodology for Network Interconnect Devices.”

Before going further, let’s just put in perspective why RFC 2544 started to be used in the field. With the advent of Ethernet-based service, there was no standardized service activation methodology available to service providers. As service providers started to evaluate Ethernet network elements with RFC 2544 in their labs, they decided to use it in the field as well. That’s where the concerns started…

RFC 6815 makes an interesting point about service providers mentioning RFC 2544 testing even when their methods and procedures are only somewhat based on RFC 2544, but in actual fact are far from it. To clarify this point, I will only use the latency test as an example. As described in RFC 2544, a latency measurement is made at the mid-point of the time duration of the test iteration. This test must be repeated 20 times, and for each of the seven RFC 2544-specified frame sizes. Because each iteration lasts two minutes, you end up with a total latency test time of 4.66 hours (2 minutes * 20 iterations * 7 frame sizes). This test duration only applies to the latency test; you would still need to perform the throughput, back-to-back and frame loss tests. Because service providers need to turn up services rapidly, their methods and procedures have adapted RFC 2544, but cannot be called RFC 2544…

Because RFC 2544 was designed for benchmarking of network elements, and not services, it also lacks service attribute testing. RFC 6815 has issued some positioning statement, since RFC 2544 cannot be used for: 

1. Validation of telecommunication service configuration, such as the committed information rate (CIR).
2. Validation of performance metrics in a telecommunication service-level agreement (SLA), such as frame loss and latency.
3. Telecommunication service activation testing, where traffic that shares network resources with the test might be adversely affected.

There are two others major points that are missing from this list, as follows:

1. RFC 2544 is a performance-based measurement, which means that the methodology will take the necessary amount of time required to find the absolute performance limit of a device/service but  not if it is meeting the service level agreement (SLA) negotiated with the end-user.
2. Inter-frame delay variation (IFDV) is not part of the RFC 2544 methodology.

Insofar as EXFO’s thoughts on RFC 6815 are concerned, for the most part, we agree with it. However, although some of the points made in RFC 6815 are valid, they do not always reflect the service provider reality, which revolves around methods and procedures that are based on RFC 2544, but behave differently. If a service provider would like to use a standardized methodology as part of its methods and procedures, as written in RFC 6815, we would suggest using ITU-T Y.1564.

Y.1564 was created within the context of Ethernet service activation based on the service attributes used by service providers to define their SLAs. There are numerous publications on the values of Y.1564, but what makes it a valid replacement to RFC 2544-like testing is its simplicity, accuracy and reduced test time. Service providers worldwide have changed their methods and procedures to include Y.1564. We understand that some service providers are still using an RFC 2544-like methodology as part of their methods and procedures, but we feel that having them switch to an Y.1564-based one is worth the effort.

So, if your methods and procedures are based on RFC 2544, why not make the switch to a methodology adapted to turning up Ethernet-based services, in other words, Y.1564!