EANTC Test Report: ADVA FSP 150 ProVMe

EANTC Test Report: ADVA FSP 150 ProVMe Page 1 of 8
ADVA FSP 150 ProVMe
Performance and Functionality Test Report
Introduction
EANTC was commissioned by Intel under the Intel®
Network Builders program to perform independent
tests of the ADVA FSP 150 ProVMe (F2.6) solution.
ADVA’s FSP 150 ProVMe combines an open platform
for hosting of virtual network functions with multilayer
business service demarcation in a single network
element. We were engaged to verify ADVA’s claims
regarding integration of fiber-based business access
solutions with open VNF-hosting platforms for Network
Function Virtualization (NFV) of fiber-based business
access solutions and open-source virtual infrastructure
management (VIM) systems for Network Function Virtu-
alization (NFV). ADVA claims that FSP 150 ProVMe
can be configured with OpenFlow, OpenStack and
Netconf/YANG-supporting Software Defined
Networks (SDN) and NFV-centric networks, creating
significant advantages over legacy proprietary solu-
tions.
In our series of tests we reviewed FSP 150 ProVMe’s
performance and functionality. EANTC confirmed that
FSP 150 ProVMe can replace legacy customer prem-
ises routers with virtualized, integrated solutions,
without sacrificing any of the functionality or perfor-
mance.
Specifically, we covered three key capabilities:
• The VNF lifecycle management process was
demonstrated with standard OpenStack Juno
release showing the ability to openly integrate into
typical NFVI architectures.
• The performance improvement promises of Single
Root I/O Virtualization (SR-IOV) versus Open
vSwitch (OVS) were evaluated and confirmed with
a range of tests evaluating the forwarding
performance.
• FSP 150 ProVMe can provide a wide range of
hardware-based support functions improving
performance without consuming compute
resources. Our test cases verified Access Control
Lists (ACLs), port mirroring, network clock
synchronization and VNF-performance assurance
functions running independently from the server.
As our evaluation showed, ADVA successfully
combined its vast experience in manufacturing
classic Ethernet-based CPEs with a solid NFV infra-
structure based on the compact architecture of the
Intel® Xeon® processor D family to construct a
unique edge-based NFV solution with value-added
hardware features.
VNF Lifecycle Management
ADVA successfully demonstrated extensive lifecycle
management with Brocade vRouter VNF including
On-Boarding, VNF requirements allocation,
Network service creation and Instantiation by using
a standard OpenStack tool.
We started the test session by on-boarding a
Brocade vRouter image by using both the
command-line tool glance as well as via Horizon
GUI and verified its presence in the image list, as
shown in figure 1. ADVA used a Heat template to
simplify the network service creation which covered
Test Highlights
 Support for OVS and SR-IOV oper-
ation, SR-IOV halving latency in
comparison
 Full line rate performance (1Gbit/s)
in SR-IOV mode with IPv4 traffic
for all packet sizes
 Full line rate performance (1Gbit/s)
in OVS mode with IPv4/IPv6 traffic
mix for packet sizes above 128
bytes
 VNF lifecycle management with
OpenStack Juno Heat templates
 Automated connectivity manage-
ment with Modular Layer 2 (ML2)
plugin for OpenStack's Neutron
 Hardware-based performance
assurance functions for network
connectivity and VNF hosting do
not consume compute resources
 Hardware-based precision time
synchronization with slave clock
independent of the traffic load

all steps needed for the procedure. ADVA claimed
that cloud-init and config drive are supported for
VNF configuration. By using cloud-init, all configu-
rations could be loaded during the VNF on-
boarding process. In the config drive method, the
configuration would be saved on an additional disk
and loaded after the on-boarding process. Limita-
tions in configuring the Brocade vRouter VNF with
cloud-init could be solved by directly accessing the
VM`s console.
Service chaining requires connectivity to be estab-
lished among VMs and from VMs to virtual and
physical network interfaces. Connectivity is
managed by the OpenStack's Neutron. As the FSP
150 ProVMe combines virtual switching capability
with physical switches, ADVA provides a mecha-
nism driver interfacing with the ML2 plugin into
Neutron for configuring end-to-end connectivity
through virtual and physical switches in an auto-
mated way. Our tests revealed some differences in
VLAN handling between SR-IOV drivers and the
VM, which were solved by a script provided by
ADVA.
We validated that the newly deployed Brocade
vRouter functioned correctly by sending test traffic
through the instance.Through these procedures
ADVA demonstrated the simplicity of Lifecycle
management by using standard OpenStack tools.
Forwarding Performance
The forwarding performance can be verified for partic-
ular interfaces or for a whole device by measuring the
throughput and delay performance metric when the
data traffic is transmitted into the interface or device at
high load. The goal of our test was to find the perfor-
mance of customer premises equipment (FSP 150
ProVMe) virtual switch coupled with a Network Func-
tion Virtualization Interface Platform based on Intel
Xeon processor D family.
The test was performed with a series of standard
packet sizes ranging from 64 to 1518 octets based on
RFC 2544 section 26.1 and 26.2, as well as a mix of
packet sizes realistically resembling typical Internet
traffic (IMIX). The IMIX used in the test is defined as a
proportional mix of several frame sizes as specified in
Table 1 below.
Table 1: IMIX Composition
Figure 1: OpenStack VNF Image List
Frame Size [bytes] Weight Percentage
IMIX for IPv4
64 3 4.8%
100 26 41.3%
256 6 9.5%
570 5 7.9%
1300 6 9.5%
1518 17 27.0%
IMIX for IPv6
128 29 46.0%
256 6 9.5%
570 5 8.0%
1300 6 9.5%
1518 17 27.0%

In this series of tests we measured the throughput
and latency metric when the data traffic is trans-
mitted into the FSP 150 ProVMe at high load. We
wanted to show what base line performance can be
expected from OVS and SR-IOV in NFV setup. For
the performance verification we tested with a series
of simple services, with one and two VNFs in a
chain as described in figure 2. As a result, the total
forwarding performance of the platform depends
on how many services had to be operated in order
to create the desired service chain. As in figure 2
the test traffic was transmitted through a single VNF
as well as a service chain with two VNFs. For this
purpose ADVA used Brocade vRouter as a VNF to
create a service chain and to forward the IP traffic
between two interfaces. In this test case we did not
analyze the performance of Brocade vRouter.
ADVA’s FSP 150 ProVMe platform offers two options
to connect the VMs, namely Open vSwitch (OVS) and
SR-IOV (Single-Root I/O Virtualization). OVS provides
a fully functional switch implementation in software
and is capable of stripping the VLAN tag, therefore
the VNF did not require specific VLAN configuration.
On the contrary, SR-IOV bypasses the OVS and
connects the VNFs directly to the physical ports and
enables multiple VNFs to share PCI hardware
resources. SR-IOV is designed to have the perfor-
mance similar to PCI pass-through, but allows multiple
VMs to be connected via same network interface,
separated by VLAN. In the SR-IOV setup, the traffic
arrives with VLAN tag at the VM. Our SR-IOV testing
focused on IPv4 traffic while ADVA worked on
resolving the problem with VLAN tagged IPv6 traffic
on the Brocade vRouter. However, we performed the
OVS test with IPv4 and IPv6 traffic. The test scenarios
are categorized as follows.
Table 2: Test Scenarios – Forwarding Performance
The test setup consisted of a FSP 150 ProVMe and
an Ixia traffic generator. We connected two
Ethernet ports to the physical ports and simulated
bidirectional IP test traffic at the target traffic trans-
mission rate of 1 Gbit/s between two ports of FSP
150 ProVMe using IxNetwork software. By using
RFC2544 test methodology, we found the
throughput value for each of the frame sizes by
locating the maximum traffic rate where no loss
occurs through binary search. At the same time, we
measured the latency achieved at the maximum
bandwidth. The goal of the test was to find the
performance metric of OVS and SR-IOV solutions in
ADVA’s FSP 150 ProVMe platform.
We achieved full line rate performance (1 Gbit/s)
for all scenarios and packet sizes with the sole
exception of the 2-VNF setup with OVS and 82-byte
packets. This behavior is within expectations, as
traffic consisting of small packets results in higher
packet rate and relative processing overhead.
Parallel to the throughput, we measured the latency
for each setup. As expected, the latency increases
with the number of VNFs, as the packets are
passing more segments in the service chain (see
figure 4 and 5). Also as expected, SR-IOV achieves
Figure 2: VNF Chaining – Forwarding Performance
Router
vSwitch vSwitch
Router
Traffic Generator
FSP150 ProVMevSwitchBrocade vRouter VNF
Virtual networksGigabit Ethernet
Router
Physical port Virtual port
Router
Test VNFs vSwitch Traffic
1 1x VNF OVS IPv4+IPv6
2 2x VNF OVS IPv4+IPv6
3 1x VNF SR-IOV IPv4
4 2x VNF SR-IOV IPv4

slightly lower latency due to bypassing the OVS.
The increased number of interconnections leads to
increased processing time for the switch process.
OVS with two VNFs has the highest average
latency for all the packet sizes; notably it was 224
microseconds (μs) at 1024 bytes packet size (see
figure 4). The lowest average latency (46 μs) was
measured for SR-IOV with one VNF at 128 bytes
packet size.
Figure 5 indicates that OVS with two VNFs has the
highest maximum latency measurement at 1024
bytes packet size, which is 868 μs. The lowest
maximum latency is measured for SR-IOV with one
VNF at 82 bytes and 128 bytes packet size, which
is 192 μs.
From figure 4 and 5 we can conclude that OVS has
twice the latency of SR-IOV; with SR-IOV it can be
clearly seen that latency scales linearly with the
number of VNFs.
Hardware-Based Support Functions
In addition to the high performance server, the FSP
150 ProVMe provides a wide range of hardware-
assisted support functions such as synchronization
and service assurance. Those functions can be acti-
vated without requiring compute resources, hence,
not creating any negative impact on the perfor-
mance of revenue-generating VNFs. This advantage
was shown with a series of tests demonstrating inde-
pendence of firewalling with ACLs, port mirroring
and synchronization from service performance. The
test cases looked at the impact of service activation
on VNF latency and analyzed any correlation
between hardware functions and software appli-
ances.
Port Mirroring
ADVA’s FSP 150 ProVMe provides traffic mirroring
functionality, where the entire traffic through a
specific interface can be copied and sent to another
physical or virtual interface. For this test, we set up
an additional VM running plain Ubuntu and PCI
Pass-through connection to Ubuntu VM for port
mirroring. This test setup consists of one Brocade
vRouter, SR-IOV and one Ubuntu VM. Using stan-
dard interface statistics and packet capture in
Ubuntu VM we verified that FSP 150 ProVMe
mirrors the traffic during the test. At the same time
we compared the test results with standard SR-IOV
with a single Brocade vRouter test setup.
Data transmission was not significantly affected by
using port mirroring at any frame size (see figure
6). We confirmed that FSP 150 ProVMe reaches
100% throughput while using the port mirroring
feature
Figure 3: Throughput Performance
Figure 4: Average Latency
Figure 5: Maximum Latency

In figure 7, the highest latency for port mirroring
was observed at 1024 byte packet size, which was
increased by 97 μs and also the value was
increased by 40 μs at 1280 byte packet size.
Access Control List (ACL)
The Network Interface Device (NID) component of
the FSP 150 ProVMe platform can apply ACL rules
to the incoming and outgoing traffic. This can
provide a simple but efficient alternative to a full-
featured firewall, or help reduce the traffic load by
filtering out undesirable traffic before it reaches the
NFVI. The filtering rules are executed in the order
defined by their priority value and can allow or
deny ranges of MAC addresses, VLANs or IPv4/
IPv6 addresses.
In our test, we defined a total of 20 rules, 10 for
each access and network-side interfaces. These
included 3 MAC-based, 2 VLAN-based and 5 IP-
based rules. The rules were defined to allow our test
traffic to pass, but not before the entire list was eval-
uated. We manually confirmed that the rules were
in fact blocking matching addresses and then ran a
regular throughput test. This test setup consists of
one Brocade vRouter, SR-IOV and the above
mentioned ACL rules. We compared the test results
with standard SR-IOV with a single Brocade vRouter
test setup.
In figure 8 we can see that the average latency is
increased by nearly 10 μs only at 82 bytes, 1280
bytes and 1518 bytes while using NID ACLs and at
most frame sizes data transmission was not
affected.
In figure 9, the maximum latency for NID ACLs was
observed at 1518 byte packet size, which was
increased by 100μs and also the value of NID ACLs
was increased by 133μs at 1280 byte packet size.
We confirmed that FSP 150 ProVMe reaches 100%
throughput while using the ACL feature.
Clock Synchronization
FSP 150 ProVMe platform provides synchronization
delivery and assurance functions using Synchronous
Ethernet (SyncE) and IEEE1588 Precision Time
Protocol (PTP) to address timing requirements in
mobile networks. It can act as a slave, boundary
and master clock. In this test setup we measured the
Figure 6: Port Mirroring - Average Latency
Figure 7: Port Mirroring - Maximum Latency
Figure 8: ACL - Average Latency
Figure 9: ACL - Maximum Latency

accuracy of the slave clock synchronization with an
external boundary clock and verified that the slave
clock synchronization is not affected by the traffic
load. The device provides pulse-per-second (PPS)
signal and 10 MHz outputs which we used to
measure both phase and frequency quality of the
PTP synchronization.
Our test bed consisted of two interconnected FSP
150 ProVMe devices and a Calnex Paragon-X test
equipment. One of the ADVA devices acted as a
slave clock and the other as a boundary clock.
Paragon-X has a GPS-fed grandmaster clock device
and a Phase/ Frequency analyzer. All three devices
use PTP and the grandmaster clock provides
synchronization to the boundary clock, which it
then relays to the slave clock. The PTP signal
between the boundary and the slave clock ran in a
separate VLAN over the same physical link as the
test traffic. The quality of the frequency and phase
synchronization was measured by the Phase/
Frequency analyzer. The analyzer received PPS and
10-MHz signals from both grandmaster and the
slave clock. We used the G.823 E1 SEC and
G.8271.1 masks as a requirement for frequency
and phase synchronization and verified the long-
term stability of the synchronization by running the
measurement for 6 hours. Figure 10 depicts the test
setup.
We started the test by allowing the boundary clock
to acquire a lock via the GPS-fed grandmaster
clock. Once it was locked, the slave started to lock
to the Boundary Clock. PPS and 10 MHz signals of
the locked slave clock were measured by the
Phase/Frequency analyzer with grandmaster
signals. We performed the test once with traffic
load to verify that PTP synchronization was not
affected by traffic running in parallel. To define the
network load we used a network load profile from
ITU-T G.8261, test case 13.
The measured average absolute time error of PPS
signal was only -11 ns against the mask G8271
accuracy level 4 (±1.5μs) requirements of ±1.1μs.
The measured maximum Maximum Time Interval
Error (MTIE) value of the PPS signal was 14 ns and
when traffic was added, it increased only by 44 ns
and was still well below the mask.
As well as the measured maximum MTIE value of
frequency was nearly 19 ns and during the traffic it
increased only by 43 ns. All the measurements
show that phase and frequency successfully
synchronized with slave clock and the synchroniza-
tion was not affected by the traffic load.
VNF Performance Assurance Functions
The FSP 150 ProVMe provides a set of advanced
assurance functions for VNF in-service monitoring.
EECPA (Enhanced Ethernet Connection Performance
Analysis) provides an integrated traffic generator/
analyzer for synthetic performance measurements.
EECPA performs measurements between the internal
virtual ports, therefore using different reference
points than the external Ixia analyzer. Due to a
shorter data path which excludes the hardware NID
Figure 10: Test Setup – Clock Synchronization
Traffic Generator
FSP150 ProVMevSwitchBrocade vRouter VNF PTP Daemon
Virtual networksGigabit Ethernet
Router PTP
Physical port Virtual port
Router Boundary
Clock
Router Slave
Clock
10 MHz signal PPS signal PTP GPS signal
Phase/Frequency
Analyzer
Paragon X
Grandmaster
Clock
12:50:00
GPS

component, the measured latency is lower, as
expected.
This case is applicable for one way testing of VM(s)
in the embedded or external service while EECPA
generator and monitor are implemented in the
internal hardware as shown in the figure below.
Since EECPA generator and monitor are located in
the same Node test operation and delay measure-
ment is tightly coordinated: same timestamp gener-
ator is used and test start and test end scheduling is
simple.
In this test we used only one stream to generate
1Gbit/s IPv4 traffic with 100 bytes frame size. For
the analysis we used the OVS and SR-IOV setup
and measure the latency of EECPA measurement.
We confirmed that FSP 150 ProVMe is able to
generate test traffic at 100% throughput and
measure the performance of the VNFs or Service
Chain it is applied to.
Figures 12 and 13 show examples of the test results
measured by EECPA, showing how FSP 150
ProVMe is able to measure the latency achieved
through the service chain of VNFs provisioned on
the unit.
About EANTC
EANTC (European Advanced
Networking Test Center) is
internationally recognized as
one of the world's leading
independent test centers for
telecommunication technolo-
gies. Based in Berlin, the
company offers vendor-
neutral consultancy and realistic, reproducible high-
quality testing services since 1991. Customers
include leading network equipment manufacturers,
tier-1 service providers, large enterprises and
governments worldwide. EANTC's proof of
concept, acceptance tests and network audits cover
established and next-generation fixed and mobile
network technologies.
EANTC AG
Salzufer 14, 10587 Berlin, Germany
info@eantc.com, http://www.eantc.com/
Figure 11: Test Setup - EECPA
Figure 12: EECPA Latency Test Results
Figure 13: EECPA Test Configuration and Results

Intel and Xeon are trademarks of Intel Corporation
or its subsidiaries in the U.S. and/or other coun-
tries.

EANTC Test Report: ADVA FSP 150 ProVMe

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (18)

Similar to EANTC Test Report: ADVA FSP 150 ProVMe

Similar to EANTC Test Report: ADVA FSP 150 ProVMe (20)

More from ADVA

More from ADVA (20)

Recently uploaded

Recently uploaded (20)

EANTC Test Report: ADVA FSP 150 ProVMe