total number of messages transmitted to the controller
If this test is repeated with a varying number of nodes with the same topology, the results SHOULD be reported in the form of a graph. The X coordinate SHOULD be the number of nodes (N), and the Y coordinate SHOULD be the Average Asynchronous Message Processing Time.5.1.3. Asynchronous Message Processing Rate
Objective:
Measure the number of responses to asynchronous messages (a new flow arrival notification message, link down, etc.) for which the controller(s) performed processing and replied with a valid and productive (non-trivial) response message.
Using a single procedure, this test will measure the following two benchmarks on the Asynchronous Message Processing Rate (see Section 2.3.1.3 of [RFC8455]):
1. Maximum Asynchronous Message Processing Rate
2. Loss-Free Asynchronous Message Processing RateBhuvaneswaran, et al. Informational [Page 14] RFC 8456 SDN Controller Benchmarking Methodology October 2018
Here, two benchmarks are determined through a series of trials where the number of messages sent to the controller(s) and the responses received from the controller(s) are counted over the Trial Duration. The message response rate and the Message Loss Ratio are calculated for each trial.
Reference Test Setup:
This test SHOULD use one of the test setups illustrated in Section 3.1 or Section 3.2 of this document.
Prerequisites:
1. The controller(s) MUST have successfully completed the network topology discovery for the connected Network Devices.
2. Choose and record the Trial Duration (Td), the sending rate STEP size, the tolerance on equality for two consecutive trials (P%), and the maximum possible message-sending rate (Ntx1/Td).
Procedure:
1. Generate asynchronous messages continuously at the maximum possible rate on the established connections from all the emulated/simulated Network Devices for the given Trial Duration (Td).
2. Record the total number of responses received (Nrx1) from the controller as well as the number of messages sent (Ntx1) to the controller within the Trial Duration (Td).
3. Calculate the Asynchronous Message Processing Rate (APR1) and the Message Loss Ratio (Lr1). Ensure that the controller(s) has returned to normal operation.
4. Repeat the trial by reducing the asynchronous message-sending rate used in the last trial by the STEP size.
5. Continue repeating the trials and reducing the sending rate until both the maximum value of Nrxn (number of responses received from the controller) and the Nrxn corresponding to a Loss Ratio of zero have been found.Bhuvaneswaran, et al. Informational [Page 15] RFC 8456 SDN Controller Benchmarking Methodology October 2018
6. The trials corresponding to the benchmark levels MUST be repeated using the same asynchronous message rates until the responses received from the controller are equal (+/-P%) for two consecutive trials.
7. Record the number of responses received (Nrxn) from the controller as well as the number of messages sent (Ntxn) to the controller in the last trial.
Measurements:
Nrxn Asynchronous Message Processing Rate (APRn) = ----- Td
Maximum Asynchronous Message Processing Rate = MAX(APRn) for all n
Nrxn Asynchronous Message Loss Ratio (Lrn) = 1 - ----- Ntxn
Loss-Free Asynchronous Message Processing Rate = MAX(APRn) given Lrn = 0
Reporting Format:
The Asynchronous Message Processing Rate results MUST be reported in tabular format, with a row for each trial.
The table should report the following information, in addition to the configuration parameters captured per Section 4.8, with columns:
- Offered rate (Ntxn/Td)
- Asynchronous Message Processing Rate (APRn)
- Loss Ratio (Lr)
- Benchmark at this iteration (blank for none, Maximum
Message Processing Rate)
The results MAY be presented in the form of a graph. The X axis SHOULD be the offered rate, and dual Y axes would represent the Asynchronous Message Processing Rate and the Loss Ratio, respectively.Bhuvaneswaran, et al. Informational [Page 16] RFC 8456 SDN Controller Benchmarking Methodology October 2018
If this test is repeated with a varying number of nodes over the same topology, the results SHOULD be reported in the form of a graph. The X axis SHOULD be the number of nodes (N), and the Y axis SHOULD be the Asynchronous Message Processing Rate. Both the Maximum Asynchronous Message Processing Rate and the Loss-Free Asynchronous Message Processing Rate should be plotted for each N.5.1.4. Reactive Path Provisioning Time
Objective:
Measure the time taken by the controller to set up a path reactively between source and destination nodes, defined as the interval starting with the first flow provisioning request message received by the controller(s) at its southbound interface and ending with the last flow provisioning response message sent from the controller(s) at its southbound interface.
Reference Test Setup:
This test SHOULD use one of the test setups illustrated in Section 3.1 or Section 3.2 of this document. The number of Network Devices in the path is a parameter of the test that may be varied from two to the maximum discovery size in repetitions of this test.
Prerequisites:
1. The controller MUST contain the network topology information for the deployed network topology.
2. The controller should know the location of the destination endpoint for which the path has to be provisioned. This can be achieved through dynamic learning or static provisioning.
3. Ensure that the default action for "flow miss" in the Network Device is configured to "send to controller".
4. Ensure that each Network Device in a path requires the controller to make the forwarding decision while paving the entire path.Bhuvaneswaran, et al. Informational [Page 17] RFC 8456 SDN Controller Benchmarking Methodology October 2018
Procedure:
1. Send a single traffic stream from test traffic generator TP1 to test traffic generator TP2.
2. Record the time of the first flow provisioning request message sent to the controller (Tsf1) from the Network Device at the forwarding-plane test emulator interface (I1).
3. Wait for the arrival of the first traffic frame at the endpoint (i.e., test traffic generator TP2) or the expiry of the Trial Duration (Td).
4. Record the time of the last flow provisioning response message received from the controller (Tdf1) to the Network Device at the forwarding-plane test emulator interface (I1).
Measurements:
Reactive Path Provisioning Time (RPT1) = Tdf1 - Tsf1
Average Reactive Path Provisioning Time = RPT1 + RPT2 + RPT3 .. RPTn -------------------------- Total Trials
Reactive Path Provisioning Time Variance (TRPv) = SUM[SQUAREOF(RPTi - TRPm)] -------------------------- Total Trials - 1
Where TRPm is the Average Reactive Path Provisioning Time.
Reporting Format:
The Reactive Path Provisioning Time results MUST be reported in tabular format, with a row for each iteration. The last row of the table indicates the Reactive Path Provisioning Time variance, and the previous row indicates the Average Reactive Path Provisioning Time.
The report should capture the following information, in addition to the configuration parameters captured per Section 4.8:
- Number of Network Devices in the path
Objective:
Measure the time taken by the controller to set up a path proactively between source and destination nodes, defined as the interval starting with the first proactive flow provisioned in the controller(s) at its northbound interface and ending with the last flow provisioning response message sent from the controller(s) at its southbound interface.
Reference Test Setup:
This test SHOULD use one of the test setups illustrated in Section 3.1 or Section 3.2 of this document.
Prerequisites:
1. The controller MUST contain the network topology information for the deployed network topology.
2. The controller should know the location of the destination endpoint for which the path has to be provisioned. This can be achieved through dynamic learning or static provisioning.
3. Ensure that the default action for "flow miss" in the Network Device is "drop".
Procedure:
1. Send a single traffic stream from test traffic generator TP1 to test traffic generator TP2.
2. Install the flow entries so that the traffic travels from test traffic generator TP1 until it reaches test traffic generator TP2 through the controller's northbound interface or management interface.
3. Wait for the arrival of the first traffic frame at test traffic generator TP2 or the expiry of the Trial Duration (Td).
4. Record the time when the proactive flow is provisioned in the controller (Tsf1) at the management-plane test emulator interface (I2).
5. Record the time of the last flow provisioning message received from the controller (Tdf1) at the forwarding-plane test emulator interface (I1).Bhuvaneswaran, et al. Informational [Page 19] RFC 8456 SDN Controller Benchmarking Methodology October 2018
Measurements:
Proactive Flow Provisioning Time (PPT1) = Tdf1 - Tsf1
Average Proactive Path Provisioning Time = PPT1 + PPT2 + PPT3 .. PPTn -------------------------- Total Trials
Proactive Path Provisioning Time Variance (TPPv) = SUM[SQUAREOF(PPTi - TPPm)] -------------------------- Total Trials - 1
Where TPPm is the Average Proactive Path Provisioning Time.
Reporting Format:
The Proactive Path Provisioning Time results MUST be reported in tabular format, with a row for each iteration. The last row of the table indicates the Proactive Path Provisioning Time variance, and the previous row indicates the Average Proactive Path Provisioning Time.
The report should capture the following information, in addition to the configuration parameters captured per Section 4.8:
- Number of Network Devices in the path
Objective:
Measure the maximum number of independent paths a controller can concurrently establish per second between source and destination nodes reactively, defined as the number of paths provisioned per second by the controller(s) at its southbound interface for the flow provisioning requests received for path provisioning at its southbound interface between the start of the test and the expiry of the given Trial Duration.
Reference Test Setup:
This test SHOULD use one of the test setups illustrated in Section 3.1 or Section 3.2 of this document.
Prerequisites:
1. The controller MUST contain the network topology information for the deployed network topology.
2. The controller should know the location of destination addresses for which the paths have to be provisioned. This can be achieved through dynamic learning or static provisioning.
3. Ensure that the default action for "flow miss" in the Network Device is configured to "send to controller".
4. Ensure that each Network Device in a path requires the controller to make the forwarding decision while provisioning the entire path.
Procedure:
1. Send traffic with unique source and destination addresses from test traffic generator TP1.
2. Record the total number of unique traffic frames (Ndf) received at test traffic generator TP2 within the Trial Duration (Td).Bhuvaneswaran, et al. Informational [Page 21] RFC 8456 SDN Controller Benchmarking Methodology October 2018
Measurements:
Ndf Reactive Path Provisioning Rate (RPR1) = ------ Td
Average Reactive Path Provisioning Rate = RPR1 + RPR2 + RPR3 .. RPRn -------------------------- Total Trials
Reactive Path Provisioning Rate Variance (RPPv) = SUM[SQUAREOF(RPRi - RPPm)] -------------------------- Total Trials - 1
Where RPPm is the Average Reactive Path Provisioning Rate.
Reporting Format:
The Reactive Path Provisioning Rate results MUST be reported in tabular format, with a row for each iteration. The last row of the table indicates the Reactive Path Provisioning Rate variance, and the previous row indicates the Average Reactive Path Provisioning Rate.
The report should capture the following information, in addition to the configuration parameters captured per Section 4.8:
- Number of Network Devices in the path
- Offered rate
Objective:
Measure the maximum number of independent paths a controller can concurrently establish per second between source and destination nodes proactively, defined as the number of paths provisioned per second by the controller(s) at its southbound interface for the paths requested in its northbound interface between the start of the test and the expiry of the given Trial Duration. The measurement is based on data-plane observations of successful path activation.
Reference Test Setup:
This test SHOULD use one of the test setups illustrated in Section 3.1 or Section 3.2 of this document.
Prerequisites:
1. The controller MUST contain the network topology information for the deployed network topology.
2. The controller should know the location of destination addresses for which the paths have to be provisioned. This can be achieved through dynamic learning or static provisioning.
3. Ensure that the default action for "flow miss" in the Network Device is "drop".
Procedure:
1. Send traffic continuously with unique source and destination addresses from test traffic generator TP1.
2. Install corresponding flow entries so that the traffic travels from simulated sources at test traffic generator TP1 until it reaches the simulated destinations at test traffic generator TP2 through the controller's northbound interface or management interface.
3. Record the total number of unique traffic frames (Ndf) received at test traffic generator TP2 within the Trial Duration (Td).Bhuvaneswaran, et al. Informational [Page 23] RFC 8456 SDN Controller Benchmarking Methodology October 2018
Measurements:
Ndf Proactive Path Provisioning Rate (PPR1) = ------ Td
Average Proactive Path Provisioning Rate = PPR1 + PPR2 + PPR3 .. PPRn -------------------------- Total Trials
Proactive Path Provisioning Rate Variance (PPPv) = SUM[SQUAREOF(PPRi - PPPm)] ------------------------- Total Trials - 1
Where PPPm is the Average Proactive Path Provisioning Rate.
Reporting Format:
The Proactive Path Provisioning Rate results MUST be reported in tabular format, with a row for each iteration. The last row of the table indicates the Proactive Path Provisioning Rate variance, and the previous row indicates the Average Proactive Path Provisioning Rate.
The report should capture the following information, in addition to the configuration parameters captured per Section 4.8:
- Number of Network Devices in the path
- Offered rate
Objective:
Measure the amount of time taken by the controller to detect any changes in the network topology, defined as the interval starting with the notification message received by the controller(s) at its southbound interface and ending with the first topology rediscovery message sent from the controller(s) at its southbound interface.
Reference Test Setup:
This test SHOULD use one of the test setups illustrated in Section 3.1 or Section 3.2 of this document.
Prerequisites:
1. The controller MUST have successfully discovered the network topology information for the deployed network topology.
2. The periodic network discovery operation should be configured to twice the Trial Duration (Td) value.
Procedure:
1. Trigger a topology change event by bringing down an active Network Device in the topology.
2. Record the time when the first topology change notification is sent to the controller (Tcn) at the forwarding-plane test emulator interface (I1).
3. Stop the trial when the controller sends the first topology rediscovery message to the Network Device or the expiry of the Trial Duration (Td).
4. Record the time when the first topology rediscovery message is received from the controller (Tcd) at the forwarding-plane test emulator interface (I1).Bhuvaneswaran, et al. Informational [Page 25] RFC 8456 SDN Controller Benchmarking Methodology October 2018
Measurements:
Network Topology Change Detection Time (TDT1) = Tcd - Tcn
Average Network Topology Change Detection Time = TDT1 + TDT2 + TDT3 .. TDTn -------------------------- Total Trials
Network Topology Change Detection Time Variance (NTDv) = SUM[SQUAREOF(TDTi - NTDm)] -------------------------- Total Trials - 1
Where NTDm is the Average Network Topology Change Detection Time.
Reporting Format:
The Network Topology Change Detection Time results MUST be reported in tabular format, with a row for each iteration. The last row of the table indicates the Network Topology Change Detection Time variance, and the previous row indicates the Average Network Topology Change Detection Time.5.2. Scalability 5.2.1. Control Sessions Capacity
Objective:
Measure the maximum number of control sessions the controller can maintain, defined as the number of sessions that the controller can accept from Network Devices, starting with the first control session and ending with the last control session that the controller(s) accepts at its southbound interface.
Reference Test Setup:
This test SHOULD use one of the test setups illustrated in Section 3.1 or Section 3.2 of this document.
Prerequisites:
NoneBhuvaneswaran, et al. Informational [Page 26] RFC 8456 SDN Controller Benchmarking Methodology October 2018
Procedure:
1. Establish control connections with the controller from every Network Device emulated in the forwarding-plane test emulator.
2. Stop the trial when the controller starts dropping the control connections.
3. Record the number of successful connections established (CCn) with the controller at the forwarding-plane test emulator.
Measurement:
Control Sessions Capacity = CCn
Reporting Format:
The Control Sessions Capacity results MUST be reported in addition to the configuration parameters captured per Section 4.8.5.2.2. Network Discovery Size
Objective:
Measure the network size (number of nodes, links, and hosts) that a controller can discover, defined as the size of a network that the controller(s) can discover, starting with a network topology provided by the user for discovery and ending with the number of nodes, links, and hosts that the controller(s) were able to successfully discover.
Reference Test Setup:
This test SHOULD use one of the test setups illustrated in Section 3.1 or Section 3.2 of this document.
Prerequisites:
1. The controller MUST support automatic network discovery.
2. The tester should be able to retrieve the discovered topology information through either the controller's management interface or northbound interface.Bhuvaneswaran, et al. Informational [Page 27] RFC 8456 SDN Controller Benchmarking Methodology October 2018
Procedure:
1. Establish the network connections between the controller and the network nodes.
2. Query the controller every t seconds (the RECOMMENDED value for t is 30) to obtain the discovered network topology information through the northbound interface or the management interface.
3. Stop the trial when the discovered network topology information remains the same as that of the last two query responses.
4. Compare the obtained network topology information with the deployed network topology information.
5. If the comparison is successful, increase the number of nodes by 1 and repeat the trial. If the comparison is unsuccessful, decrease the number of nodes by 1 and repeat the trial.
6. Continue the trial until the comparison (step 5) is successful.
7. Record the number of nodes for the last trial run (Ns) where the topology comparison was successful.
Measurement:
Network Discovery Size = Ns
Reporting Format:
The Network Discovery Size results MUST be reported in addition to the configuration parameters captured per Section 4.8.Bhuvaneswaran, et al. Informational [Page 28] RFC 8456 SDN Controller Benchmarking Methodology October 2018 5.2.3. Forwarding Table Capacity
Objective:
Measure the maximum number of flow entries a controller can manage in its Forwarding Table.
Reference Test Setup:
This test SHOULD use one of the test setups illustrated in Section 3.1 or Section 3.2 of this document.
Prerequisites:
1. The controller's Forwarding Table should be empty.
2. "Flow idle time" MUST be set to a higher or infinite value.
3. The controller MUST have successfully completed network topology discovery.
4. The tester should be able to retrieve the Forwarding Table information through either the controller's management interface or northbound interface.
Procedures:
o Reactive Flow Provisioning Mode:
1. Send bidirectional traffic continuously with unique source and destination addresses from test traffic generators TP1 and TP2 at the Asynchronous Message Processing Rate of the controller.
2. Query the controller at a regular interval (e.g., every 5 seconds) for the number of learned flow entries from its northbound interface.
3. Stop the trial when the retrieved value is constant for three consecutive iterations, and record the value received from the last query (Nrp).Bhuvaneswaran, et al. Informational [Page 29] RFC 8456 SDN Controller Benchmarking Methodology October 2018
o Proactive Flow Provisioning Mode:
1. Install unique flows continuously through the controller's northbound interface or management interface until a failure response is received from the controller.
2. Record the total number of successful responses (Nrp).
Note:
Some controller designs for Proactive Flow Provisioning mode may require the switch to send flow setup requests in order to generate flow setup responses. In such cases, it is recommended to generate bidirectional traffic for the provisioned flows.
Measurements:
Proactive Flow Provisioning Mode:
Max Flow Entries = Total number of flows provisioned (Nrp)
Reactive Flow Provisioning Mode:
Max Flow Entries = Total number of learned flow entries (Nrp)
Forwarding Table Capacity = Max Flow Entries
Reporting Format:
The Forwarding Table Capacity results MUST be tabulated with the following information, in addition to the configuration parameters captured per Section 4.8:
- Provisioning Type (Proactive/Reactive)
Objective:
Determine the effects of handling error packets and notifications on performance tests. The impact MUST be measured for the following performance tests:
1. Path Provisioning Rate
2. Path Provisioning Time
3. Network Topology Change Detection Time
Reference Test Setup:
This test SHOULD use one of the test setups illustrated in Section 3.1 or Section 3.2 of this document.
Prerequisites:
1. This test MUST be performed after obtaining the baseline measurement results for the performance tests listed above.
2. Ensure that the invalid messages are not dropped by the intermediate devices connecting the controller and Network Devices.
Procedure:
1. Perform the above-listed performance tests, and send 1% of the messages from the Asynchronous Message Processing Rate test (Section 5.1.3) as invalid messages from the connected Network Devices emulated at the forwarding-plane test emulator.
2. Perform the above-listed performance tests, and send 2% of the messages from the Asynchronous Message Processing Rate test (Section 5.1.3) as invalid messages from the connected Network Devices emulated at the forwarding-plane test emulator.
Note:
Invalid messages can be frames with incorrect protocol fields or any form of failure notifications sent towards the controller.Bhuvaneswaran, et al. Informational [Page 31] RFC 8456 SDN Controller Benchmarking Methodology October 2018
Measurements:
Measurements MUST be done as per the equation defined in the "Measurements" section of the corresponding test listed under "Objective".
Reporting Format:
The Exception Handling results MUST be reported in tabular format, with a column for each of the below parameters and row for each of the above-listed performance tests:
- Without Exceptions
- With 1% Exceptions
- With 2% Exceptions
Objective:
Determine the effects of handling DoS attacks on performance and scalability tests. The impact MUST be measured for the following tests:
1. Path Provisioning Rate
2. Path Provisioning Time
3. Network Topology Change Detection Time
4. Network Discovery Size
Reference Test Setup:
This test SHOULD use one of the test setups illustrated in Section 3.1 or Section 3.2 of this document.
Prerequisite:
This test MUST be performed after obtaining the baseline measurement results for the performance tests listed above.Bhuvaneswaran, et al. Informational [Page 32] RFC 8456 SDN Controller Benchmarking Methodology October 2018
Procedure:
Perform the above-listed tests, and launch a DoS attack towards the controller while the trial is running.
Note: DoS attacks can be launched on one of the following interfaces:
1. Northbound (e.g., query for flow entries continuously on the northbound interface)
2. Management (e.g., Ping requests to the controller's management interface)
3. Southbound (e.g., TCP SYN messages on the southbound interface)
Measurements:
Measurements MUST be done as per the equation defined in the "Measurements" section of the corresponding test listed under "Objective".
Reporting Format:
The results regarding the handling of DoS attacks MUST be reported in tabular format, with a column for each of the below parameters and a row for each of the above-listed tests.
- Without any attacks
- With attacks
The report should also specify the nature of the attack and the interface in question.Bhuvaneswaran, et al. Informational [Page 33] RFC 8456 SDN Controller Benchmarking Methodology October 2018 5.4. Reliability 5.4.1. Controller Failover Time
Objective:
Measure the time taken to switch from an active controller to the backup controller when the controllers work in redundancy mode and the active controller fails, defined as the interval starting when the active controller is brought down and ending with the first rediscovery message received from the new controller at its southbound interface.
Reference Test Setup:
This test SHOULD use the test setup illustrated in Section 3.2 of this document.
Prerequisites:
1. Master controller election MUST be completed.
2. Nodes are connected to the controller cluster per the implemented redundancy mode (e.g., active-standby).
3. The controller cluster should have successfully completed the network topology discovery.
4. The Network Device MUST send all new flows to the controller when it receives them from the test traffic generator.
5. The controller should have learned the location of the destination (D1) at test traffic generator TP2.Bhuvaneswaran, et al. Informational [Page 34] RFC 8456 SDN Controller Benchmarking Methodology October 2018
Procedure:
1. Send unidirectional traffic continuously with incremental sequence numbers and source addresses from test traffic generator TP1 at the rate at which the controller can process the traffic without any drops.
2. Ensure that there are no packet drops observed at test traffic generator TP2.
3. Bring down the active controller.
4. Stop the trial when the first frame after the failover operation is received on test traffic generator TP2.
5. Record the time at which the last valid frame was received (T1) at test traffic generator TP2 before the sequence error and the time at which the first valid frame was received (T2) after the sequence error at test traffic generator TP2.
Measurements:
Controller Failover Time = (T2 - T1)
Packet Loss = Number of missing packet sequences
Reporting Format:
The Controller Failover Time results MUST be tabulated with the following information:
- Number of cluster nodes
- Redundancy mode
- Controller Failover Time
- Packet Loss
- Cluster keep-alive interval
Objective:
Measure the time taken by the controller to reroute traffic when there is a failure in existing traffic paths, defined as the interval starting with the first failure notification message received by the controller and ending with the last flow re-provisioning message sent by the controller at its southbound interface.
Reference Test Setup:
This test SHOULD use one of the test setups illustrated in Section 3.1 or Section 3.2 of this document.
Prerequisites:
1. A network with a specified number of nodes and redundant paths MUST be deployed.
2. The controller MUST know the location of test traffic generators TP1 and TP2.
3. Ensure that the controller does not pre-provision the alternate path in the emulated Network Devices at the forwarding-plane test emulator.
Procedure:
1. Send bidirectional traffic continuously with a unique sequence number from test traffic generators TP1 and TP2.
2. Bring down a link or switch in the traffic path.
3. Stop the trial after receiving the first frame after network reconvergence.
4. Record the time of the last received frame prior to the frame loss at test traffic generator TP2 (TP2-Tlfr) and the time of the first frame received after the frame loss at test traffic generator TP2 (TP2-Tffr). There must be a gap in sequence numbers of these frames.
5. Record the time of the last received frame prior to the frame loss at test traffic generator TP1 (TP1-Tlfr) and the time of the first frame received after the frame loss at test traffic generator TP1 (TP1-Tffr).Bhuvaneswaran, et al. Informational [Page 36] RFC 8456 SDN Controller Benchmarking Methodology October 2018
Measurements:
Forward Direction Path Re-provisioning Time (FDRT) = (TP2-Tffr - TP2-Tlfr)
Reverse Direction Path Re-provisioning Time (RDRT) = (TP1-Tffr - TP1-Tlfr)
Network Re-provisioning Time = (FDRT + RDRT)/2
Forward Direction Packet Loss = Number of missing sequence frames at test traffic generator TP1
Reverse Direction Packet Loss = Number of missing sequence frames at test traffic generator TP2
Reporting Format:
The Network Re-provisioning Time results MUST be tabulated with the following information:
- Number of nodes in the primary path
- Number of nodes in the alternate path
- Network Re-provisioning Time
- Forward Direction Packet Loss
- Reverse Direction Packet Loss
This document has no IANA actions.Bhuvaneswaran, et al. Informational [Page 37] RFC 8456 SDN Controller Benchmarking Methodology October 2018 7. Security Considerations
The benchmarking tests described in this document are limited to the performance characterization of controllers in a lab environment with isolated networks.
The benchmarking network topology will be an independent test setup and MUST NOT be connected to devices that may forward the test traffic into a production network or misroute traffic to the test management network.
Further, benchmarking is performed on a "black-box" basis, relying solely on measurements observable external to the controller.
Special capabilities SHOULD NOT exist in the controller specifically for benchmarking purposes. Any implications for network security arising from the controller SHOULD be identical in the lab and in production networks.8. References 8.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, <https://www.rfc-editor.org/info/rfc2119>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[RFC8455] Bhuvaneswaran, V., Basil, A., Tassinari, M., Manral, V., and S. Banks, "Terminology for Benchmarking Software-Defined Networking (SDN) Controller Performance", RFC 8455, DOI 10.17487/RFC8455, October 2018, <https://www.rfc-editor.org/info/rfc8455>.8.2. Informative References
[OpenFlow-Spec] ONF, "OpenFlow Switch Specification" Version 1.4.0 (Wire Protocol 0x05), October 2013, <https://www.opennetworking.org/wp-content/ uploads/2014/10/openflow-spec-v1.4.0.pdf>.Bhuvaneswaran, et al. Informational [Page 38] RFC 8456 SDN Controller Benchmarking Methodology October 2018 Appendix A. Benchmarking Methodology Using OpenFlow Controllers
This section gives an overview of the OpenFlow protocol
[OpenFlow-Spec] and provides a test methodology for benchmarking SDN
Controllers supporting the OpenFlow southbound protocol. The
OpenFlow protocol is used as an example to illustrate the
methodologies defined in this document.
A.1. Protocol Overview
OpenFlow [OpenFlow-Spec] is an open standard protocol defined by the Open Networking Foundation (ONF) and used for programming the forwarding plane of network switches or routers via a centralized controller.A.2. Messages Overview
The OpenFlow protocol supports three message types -- namely, controller-to-switch, asynchronous, and symmetric.
Controller-to-switch messages are initiated by the controller and used to directly manage or inspect the state of the switch. These messages allow controllers to query/configure the switch ("features" messages, configuration messages), collect information from a switch (Read-State messages), send packets on a specified port of a switch (OFPT_PACKET_OUT messages), and modify the switch forwarding plane and state (Modify-State messages, Role-Request messages, etc.).
Asynchronous messages are generated by the switch without a controller soliciting them. These messages allow switches to update controllers to denote an arrival of a new flow (OFPT_PACKET_IN messages), switch state changes ("flow-removed" messages, port-status messages), and errors (Error messages).
Symmetric messages are generated in either direction without solicitation. These messages allow switches and controllers to set up a connection (Hello messages), verify liveness (Echo messages), and offer additional functionalities (Experimenter messages).A.3. Connection Overview
The OpenFlow channel is used to exchange OpenFlow messages between an OpenFlow switch and an OpenFlow controller. The OpenFlow channel connection can be set up using plain TCP or TLS. By default, a switch establishes a single connection with the SDN Controller. A switch may establish multiple parallel connections to a single controller (auxiliary connection) or multiple controllers to handle controller failures and load balancing.Bhuvaneswaran, et al. Informational [Page 39] RFC 8456 SDN Controller Benchmarking Methodology October 2018 A.4. Performance Benchmarking Tests A.4.1. Network Topology Discovery Time
Procedure:
Network Devices OpenFlow SDN Controller Application | | | | |<Initialize controller | | |app., NB and SB interfaces>| | | | |<Deploy network with | | | given no. of OF switches> | | | | | | OFPT_HELLO Exchange | | |<-------------------------->| | | | | | OFPT_PACKET_OUT with LLDP| | | to all switches| | (Tm1)|<---------------------------| | | | | | OFPT_PACKET_IN with LLDP| | | rcvd from Switch 1| | |--------------------------->| | | | | | OFPT_PACKET_IN with LLDP| | | rcvd from Switch 2| | |--------------------------->| | | . | | | . | | | | | | OFPT_PACKET_IN with LLDP| | | rcvd from Switch n| | (Tmn)|--------------------------->| | | | | | | <Wait for the expiry of| | | the Trial Duration (Td)>| | | | | | Query the controller for| | | discovered n/w topo. (Di)| | |<--------------------------| | | | | | <Compare the discovered| | | n/w topology and the| | | offered n/w topology>| | | |Bhuvaneswaran, et al. Informational [Page 40] RFC 8456 SDN Controller Benchmarking Methodology October 2018
Legend:
NB: Northbound SB: Southbound OF: OpenFlow OFP: OpenFlow Protocol LLDP: Link-Layer Discovery Protocol Tm1: Time of reception of first LLDP message from controller Tmn: Time of last LLDP message sent to controller
Discussion:
The Network Topology Discovery Time can be obtained by calculating the time difference between the first OFPT_PACKET_OUT with an LLDP message received from the controller (Tm1) and the last OFPT_PACKET_IN with an LLDP message sent to the controller (Tmn) when the comparison is successful.Bhuvaneswaran, et al. Informational [Page 41] RFC 8456 SDN Controller Benchmarking Methodology October 2018 A.4.2. Asynchronous Message Processing Time
Procedure:
Network Devices OpenFlow SDN Controller Application | | | |OFPT_PACKET_IN with single | | |OFP match header | | (T0)|--------------------------->| | | | | |OFPT_PACKET_OUT with single | | | OFP action header | | (R0)|<---------------------------| | | . | | | . | | | . | | | | | |OFPT_PACKET_IN with single | | |OFP match header | | (Tn)|--------------------------->| | | | | |OFPT_PACKET_OUT with single | | | OFP action header | | (Rn)|<---------------------------| | | | | |<Wait for the expiry of the | | |Trial Duration> | | | | | |<Record the number of | | |OFPT_PACKET_INs/ | | |OFPT_PACKET_OUTs | | |exchanged (Nrx)> | | | | |
Legend:
T0,T1, ..Tn: transmit timestamps of OFPT_PACKET_IN messages R0,R1, ..Rn: receive timestamps of OFPT_PACKET_OUT messages Nrx: Number of successful OFPT_PACKET_IN/OFPT_PACKET_OUT message exchanges
Discussion:
The Asynchronous Message Processing Time will be obtained by calculating the sum of ((R0 - T0),(R1 - T1)..(Rn - Tn))/Nrx.Bhuvaneswaran, et al. Informational [Page 42] RFC 8456 SDN Controller Benchmarking Methodology October 2018 A.4.3. Asynchronous Message Processing Rate
Procedure:
Network Devices OpenFlow SDN Controller Application | | | |OFPT_PACKET_IN with single | | |OFP match header | | |--------------------------->| | | | | |OFPT_PACKET_OUT with single | | | OFP action header | | |<---------------------------| | | | | | . | | | . | | | . | | | | | |OFPT_PACKET_IN with single | | |OFP match header | | |--------------------------->| | | | | |OFPT_PACKET_OUT with single | | | OFP action header | | |<---------------------------| | | | | |<Repeat the steps until | | |the expiry of the | | |Trial Duration> | | | | | |<Record the number of OFP | | (Ntx1)|match headers sent> | | | | | |<Record the number of OFP | | (Nrx1)|action headers rcvd> | | | | |
Note: The Ntx1 on initial trials should be greater than Nrx1. Repeat the trials until the Nrxn for two consecutive trials equals (+/-P%).Bhuvaneswaran, et al. Informational [Page 43] RFC 8456 SDN Controller Benchmarking Methodology October 2018
Discussion:
Using a single procedure, this test will measure two benchmarks:
1. The Maximum Asynchronous Message Processing Rate will be obtained by calculating the maximum OFPT_PACKET_OUTs (Nrxn) received from the controller(s) across n trials.
2. The Loss-Free Asynchronous Message Processing Rate will be obtained by calculating the maximum OFPT_PACKET_OUTs received from the controller(s) when the Loss Ratio equals zero. The Loss Ratio is obtained by calculating 1 - Nrxn/Ntxn.A.4.4. Reactive Path Provisioning Time
Procedure:
Test Traffic Test Traffic Network Devices OpenFlow Generator TP1 Generator TP2 Controller | | | | | |G-ARP (D1) | | | |--------------------->| | | | | | | | |OFPT_PACKET_IN(D1) | | | |-------------------->| | | | | |Traffic (S1,D1) | | (Tsf1)|----------------------------------->| | | | | | | | | | | | | | | | |OFPT_PACKET_IN(S1,D1)| | | |-------------------->| | | | | | | | FLOW_MOD(D1) | | | |<--------------------| | | | | | |Traffic (S1,D1) | | | (Tdf1)|<---------------------| | | | | |Bhuvaneswaran, et al. Informational [Page 44] RFC 8456 SDN Controller Benchmarking Methodology October 2018
Legend:
G-ARP: Gratuitous ARP message Tsf1: Time of first frame sent from TP1 Tdf1: Time of first frame received from TP2
Discussion:
The Reactive Path Provisioning Time can be obtained by finding the time difference between the transmit and receive times of the traffic (Tsf1 - Tdf1).Bhuvaneswaran, et al. Informational [Page 45] RFC 8456 SDN Controller Benchmarking Methodology October 2018 A.4.5. Proactive Path Provisioning Time
Procedure:
Test Traffic Test Traffic Network Devices OpenFlow SDN Generator TP1 Generator TP2 Controller Application | | | | | | | | | | | | | |<Install flow| | | | | for S1,D1> | | |G-ARP (D1) | | | | |-------------->| | | | | | | | | | |OFPT_PACKET_IN(D1)| | | | |----------------->| | | | | | | |Traffic (S1,D1) | | | (Tsf1)|--------------------------->| | | | | | | | | | | FLOW_MOD(D1) | | | | |<-----------------| | | | | | | | |Traffic (S1,D1)| | | | (Tdf1)|<--------------| | | | | | | |
Legend:
G-ARP: Gratuitous ARP message Tsf1: Time of first frame sent from TP1 Tdf1: Time of first frame received from TP2
Discussion:
The Proactive Path Provisioning Time can be obtained by finding the time difference between the transmit and receive times of the traffic (Tsf1 - Tdf1).Bhuvaneswaran, et al. Informational [Page 46] RFC 8456 SDN Controller Benchmarking Methodology October 2018 A.4.6. Reactive Path Provisioning Rate
Procedure:
Test Traffic Test Traffic Network Devices OpenFlow Generator TP1 Generator TP2 Controller | | | | | | | | | | | | | |G-ARP (D1..Dn) | | | |--------------------| | | | | | | | |OFPT_PACKET_IN(D1..Dn)| | | |--------------------->| | | | | |Traffic (S1..Sn,D1..Dn) | | |--------------------------------->| | | | | | | | |OFPT_PACKET_IN(S1..Sn,| | | | D1..Dn)| | | |--------------------->| | | | | | | | FLOW_MOD(S1) | | | |<---------------------| | | | | | | | FLOW_MOD(D1) | | | |<---------------------| | | | | | | | FLOW_MOD(S2) | | | |<---------------------| | | | | | | | FLOW_MOD(D2) | | | |<---------------------| | | | . | | | | . | | | | | | | | FLOW_MOD(Sn) | | | |<---------------------| | | | | | | | FLOW_MOD(Dn) | | | |<---------------------| | | | | | | Traffic (S1..Sn, | | | | D1..Dn)| | | |<-------------------| | | | | | | | | |Bhuvaneswaran, et al. Informational [Page 47] RFC 8456 SDN Controller Benchmarking Methodology October 2018
Legend:
G-ARP: Gratuitous ARP message D1..Dn: Destination Endpoint 1, Destination Endpoint 2 ..., Destination Endpoint n S1..Sn: Source Endpoint 1, Source Endpoint 2 ..., Source Endpoint n
Discussion:
The Reactive Path Provisioning Rate can be obtained by finding the total number of frames received at test traffic generator TP2 after the Trial Duration.Bhuvaneswaran, et al. Informational [Page 48] RFC 8456 SDN Controller Benchmarking Methodology October 2018 A.4.7. Proactive Path Provisioning Rate
Procedure:
Test Traffic Test Traffic Network Devices OpenFlow SDN Generator TP1 Generator TP2 Controller Application | | | | | | |G-ARP (D1..Dn) | | | | |--------------->| | | | | | | | | | |OFPT_PACKET_IN | | | | | (D1..Dn)| | | | |---------------->| | | | | | | |Traffic (S1..Sn,D1..Dn) | | | (Tsf1)|---------------------------->| | | | | | | | | | | |<Install flow| | | | | for S1,D1> | | | | | | | | | | . | | | | |<Install flow| | | | | for Sn,Dn> | | | | | | | | | FLOW_MOD(S1) | | | | |<----------------| | | | | | | | | | FLOW_MOD(D1) | | | | |<----------------| | | | | | | | | | . | | | | | FLOW_MOD(Sn) | | | | |<----------------| | | | | | | | | | FLOW_MOD(Dn) | | | | |<----------------| | | | | | | | |Traffic (S1..Sn,| | | | | D1..Dn)| | | | (Tdf1)|<---------------| | | | | | | |Bhuvaneswaran, et al. Informational [Page 49] RFC 8456 SDN Controller Benchmarking Methodology October 2018
Legend:
G-ARP: Gratuitous ARP message D1..Dn: Destination Endpoint 1, Destination Endpoint 2 ..., Destination Endpoint n S1..Sn: Source Endpoint 1, Source Endpoint 2 ..., Source Endpoint n
Discussion:
The Proactive Path Provisioning Rate can be obtained by finding the total number of frames received at test traffic generator TP2 after the Trial Duration.A.4.8. Network Topology Change Detection Time
Procedure:
Network Devices OpenFlow SDN Controller Application | | | | | <Bring down a link in | | | Switch S1>| | | | T0 |PORT_STATUS with link down | | | from S1 | | |--------------------------->| | | | | |First OFPT_PACKET_OUT with | | |LLDP to OF switch | | T1 |<---------------------------| | | | | | | <Record time of first| | | OFPT_PACKET_OUT with| | | LLDP T1>| | | |
Discussion:
The Network Topology Change Detection Time can be obtained by finding the difference between the time that OpenFlow Switch S1 sends the PORT_STATUS message (T0) and the time that the OpenFlow controller sends the first topology rediscovery message (T1) to OpenFlow switches.Bhuvaneswaran, et al. Informational [Page 50] RFC 8456 SDN Controller Benchmarking Methodology October 2018 A.5. Scalability A.5.1. Control Sessions Capacity
Procedure:
Network Devices OpenFlow Controller | | | OFPT_HELLO Exchange for Switch 1 | |<------------------------------------->| | | | OFPT_HELLO Exchange for Switch 2 | |<------------------------------------->| | . | | . | | . | | OFPT_HELLO Exchange for Switch n | |X<----------------------------------->X| | |
Discussion:
The value of Switch (n - 1) will provide the Control Sessions Capacity.Bhuvaneswaran, et al. Informational [Page 51] RFC 8456 SDN Controller Benchmarking Methodology October 2018 A.5.2. Network Discovery Size
Procedure:
Network Devices OpenFlow SDN Controller Application | | | | | <Deploy network with | | |given no. of OF switches N>| | | | | OFPT_HELLO Exchange | | |<-------------------------->| | | | | | OFPT_PACKET_OUT with LLDP| | | to all switches | | |<---------------------------| | | | | | OFPT_PACKET_IN with LLDP| | | rcvd from Switch 1| | |--------------------------->| | | | | | OFPT_PACKET_IN with LLDP| | | rcvd from Switch 2| | |--------------------------->| | | . | | | . | | | | | | OFPT_PACKET_IN with LLDP| | | rcvd from Switch n| | |--------------------------->| | | | | | | <Wait for the expiry of| | | the Trial Duration (Td)>| | | | | | Query the controller for| | | discovered n/w topo. (N1)| | |<--------------------------| | | | | | <If N1==N, repeat Step 1| | | with N + 1 nodes| | | until N1<N >| | | | | | <If N1<N, repeat Step 1 | | | with N=N1 nodes once and | | | exit> | | | |Bhuvaneswaran, et al. Informational [Page 52] RFC 8456 SDN Controller Benchmarking Methodology October 2018
Legend:
n/w topo: Network topology OF: OpenFlow
Discussion:
The value of N1 provides the Network Discovery Size value. The
Trial Duration can be set to the stipulated time within which the
user expects the controller to complete the discovery process.
Bhuvaneswaran, et al. Informational [Page 53]
RFC 8456 SDN Controller Benchmarking Methodology October 2018
A.5.3. Forwarding Table Capacity
Procedure:
Test Traffic Network Devices OpenFlow SDN Generator TP1 Controller Application | | | | | | | | |G-ARP (H1..Hn) | | | |---------------->| | | | | | | | |OFPT_PACKET_IN(D1..Dn)| | | |--------------------->| | | | | | | | |<Wait for 5 secs>| | | | | | | | <Query for FWD | | | | entry> |(F1) | | | | | | |<Wait for 5 secs>| | | | | | | | <Query for FWD | | | | entry> |(F2) | | | | | | |<Wait for 5 secs>| | | | | | | | <Query for FWD | | | | entry> |(F3) | | | | | | | <Repeat Step 2 | | | |until F1==F2==F3>| | | | |
Legend:
G-ARP: Gratuitous ARP message H1..Hn: Host 1 .. Host n FWD: Forwarding Table
Discussion:
Query the controller's Forwarding Table entries multiple times, until three consecutive queries return the same value. The last value retrieved from the controller will provide the Forwarding Table Capacity value. The query interval is user configurable. The interval of 5 seconds shown in this example is for representational purposes.Bhuvaneswaran, et al. Informational [Page 54] RFC 8456 SDN Controller Benchmarking Methodology October 2018 A.6. Security A.6.1. Exception Handling Procedure: Test Traffic Test Traffic Network Devices OpenFlow SDN Generator TP1 Generator TP2 Controller Application
| | | | | | |G-ARP (D1..Dn) | | | | |--------------->| | | | | | | | | | |OFPT_PACKET_IN(D1..Dn)| | | | |--------------------->| | | | | | | |Traffic (S1..Sn,D1..Dn) | | | |-------------------------->| | | | | | | | | | |OFPT_PACKET_IN(S1..Sa,| | | | | D1..Da)| | | | |--------------------->| | | | | | | | | |OFPT_PACKET_IN | | | | | (Sa+1..Sn,| | | | | Da+1..Dn)| | | | | (1% incorrect OFP| | | | | match header)| | | | |--------------------->| | | | | | | | | | FLOW_MOD(D1..Dn)| | | | |<---------------------| | | | | | | | | | FLOW_MOD(S1..Sa)| | | | | OFP headers| | | | |<---------------------| | | | | | | | |Traffic (S1..Sa,| | | | | D1..Da)| | | | |<---------------| | | | | | | | | | | | <Wait for the| | | | | expiry of the| | | | | Trial| | | | | Duration>| | | | | | | | | | <Record Rx| | | | | frames at| | | | | TP2 (Rn1)>|Bhuvaneswaran, et al. Informational [Page 55] RFC 8456 SDN Controller Benchmarking Methodology October 2018
| | | | | | | | | <Repeat | | | | | Step 1 with| | | | | 2% incorrect| | | | |OFPT_PACKET_INs>| | | | | | | | | | <Record Rx| | | | | frames at| | | | | TP2 (Rn2)>|
Legend:
G-ARP: Gratuitous ARP message OFPT_PACKET_IN(Sa+1..Sn,Da+1..Dn): OFPT_PACKET_IN with wrong version number Rn1: Total number of frames received at Test Port 2 with 1% incorrect frames Rn2: Total number of frames received at Test Port 2 with 2% incorrect frames
Discussion:
The traffic rate sent towards the OpenFlow switch from Test Port 1 should be 1% higher than the Path Programming Rate. Rn1 will provide the Path Provisioning Rate of the controller when 1% of incorrect frames are received, and Rn2 will provide the Path Provisioning Rate of the controller when 2% of incorrect frames are received.
The procedure defined above provides test steps to determine the
effects of handling error packets on the Path Programming Rate.
The same procedure can be adapted to determine the effects on
other performance tests listed in this benchmarking test.
Bhuvaneswaran, et al. Informational [Page 56]
RFC 8456 SDN Controller Benchmarking Methodology October 2018
A.6.2. Handling Denial-of-Service Attacks
Procedure:
Test Traffic Test Traffic Network Device OpenFlow SDN
Generator TP1 Generator TP2 Controller Application| | | | | | |G-ARP (D1..Dn) | | | | |---------------->| | | | | | | | | | |OFPT_PACKET_IN(D1..Dn)| | | | |--------------------->| | | | | | | |Traffic (S1..Sn,D1..Dn) | | | |--------------------------->| | | | | | | | | | |OFPT_PACKET_IN(S1..Sn,| | | | | D1..Dn)| | | | |--------------------->| | | | | | | | | |TCP SYN attack | | | | |from a switch | | | | |--------------------->| | | | | | | | | |FLOW_MOD(D1..Dn) | | | | |<---------------------| | | | | | | | | | FLOW_MOD(S1..Sn) | | | | | OFP headers | | | | |<---------------------| | | | | | | | |Traffic (S1..Sn, | | | | | D1..Dn) | | | | |<----------------| | | | | | | | | | | |<Wait for the| | | | |expiry of the| | | | | Trial| | | | | Duration>| | | | | | | | | | <Record Rx| | | | | frames at| | | | | TP2 (Rn1)>| | | | | |Bhuvaneswaran, et al. Informational [Page 57] RFC 8456 SDN Controller Benchmarking Methodology October 2018
Legend:
G-ARP: Gratuitous ARP message
Discussion:
A TCP SYN attack should be launched from one of the
emulated/simulated OpenFlow switches. Rn1 provides the Path
Programming Rate of the controller upon handling a denial-of-
service attack.
The procedure defined above provides test steps to determine the
effects of handling denial of service on the Path Programming
Rate. The same procedure can be adapted to determine the effects
on other performance tests listed in this benchmarking test.
Bhuvaneswaran, et al. Informational [Page 58]
RFC 8456 SDN Controller Benchmarking Methodology October 2018
A.7. Reliability
A.7.1. Controller Failover Time
Procedure:
Test Traffic Test Traffic Network Device OpenFlow SDN
Generator TP1 Generator TP2 Controller Application| | | | | | |G-ARP (D1) | | | | |-------------->| | | | | | | | | | |OFPT_PACKET_IN(D1) | | | | |---------------------->| | | | | | | |Traffic (S1..Sn,D1) | | | |--------------------------->| | | | | | | | | | | | | | | |OFPT_PACKET_IN(S1,D1) | | | | |---------------------->| | | | | | | | | |FLOW_MOD(D1) | | | | |<----------------------| | | | |FLOW_MOD(S1) | | | | |<----------------------| | | | | | | | |Traffic (S1,D1)| | | | |<--------------| | | | | | | | | | |OFPT_PACKET_IN(S2,D1) | | | | |---------------------->| | | | | | | | | |FLOW_MOD(S2) | | | | |<----------------------| | | | | | | | | |OFPT_PACKET_IN | | | | | (Sn-1,D1) | | | | |---------------------->| | | | | | | | | |OFPT_PACKET_IN(Sn,D1) | | | | |---------------------->| | | | | . | | | | | . |<Bring down | | | | . | the active | | | | | controller> | | | | FLOW_MOD(Sn-1) | | | | | X<-----------------| |Bhuvaneswaran, et al. Informational [Page 59] RFC 8456 SDN Controller Benchmarking Methodology October 2018
| | | | | | | |FLOW_MOD(Sn) | | | | |<----------------------| | | | | | | | |Traffic (Sn,D1)| | | | |<--------------| | | | | | | | | | | |<Stop the | | | | |test after | | | | |recv. traffic| | | | |upon | | | | |failure> |
Legend:
G-ARP: Gratuitous ARP message
Discussion:
The time difference between the last valid frame received before the traffic loss and the first frame received after the traffic loss will provide the Controller Failover Time.
If there is no frame loss during the Controller Failover Time, the Controller Failover Time can be deemed negligible.Bhuvaneswaran, et al. Informational [Page 60] RFC 8456 SDN Controller Benchmarking Methodology October 2018 A.7.2. Network Re-provisioning Time Procedure: Test Traffic Test Traffic Network Devices OpenFlow SDN Generator TP1 Generator TP2 Controller Application
| | | | | | |G-ARP (D1) | | | | |--------------->| | | | | | | | | | |OFPT_PACKET_IN(D1) | | | | |--------------------->| | | |G-ARP (S1) | | | |----------------------------->| | | | | | | | | | |OFPT_PACKET_IN(S1) | | | | |--------------------->| | | | | | | |Traffic (S1,D1,Seq. no (1..n))| | | |----------------------------->| | | | | | | | | | |OFPT_PACKET_IN(S1,D1) | | | | |--------------------->| | | | | | | | | Traffic (D1,S1,| | | | | Seq. no (1..n))| | | | |--------------->| | | | | | | | | | |OFPT_PACKET_IN(D1,S1) | | | | |--------------------->| | | | | | | | | |FLOW_MOD(D1) | | | | |<---------------------| | | | | | | | | |FLOW_MOD(S1) | | | | |<---------------------| | | | | | | | | Traffic (S1,D1,| | | | | Seq. no(1))| | | | |<---------------| | | | | | | | | | Traffic (S1,D1,| | | | | Seq. no(2))| | | | |<---------------| | | | | | | |Bhuvaneswaran, et al. Informational [Page 61] RFC 8456 SDN Controller Benchmarking Methodology October 2018
| | | | | | Traffic (D1,S1,Seq. no(1))| | | |<-----------------------------| | | | | | | | | Traffic (D1,S1,Seq. no(2))| | | |<-----------------------------| | | | | | | | | Traffic (D1,S1,Seq. no(x))| | | |<-----------------------------| | | | | | | | | | Traffic (S1,D1,| | | | | Seq. no(x))| | | | |<---------------| | | | | | | | | | | | | | | | | <Bring down | | | | | the switch in| | | | | the active| | | | | traffic path>| | | | | | | | |PORT_STATUS(Sa) | | | | |--------------------->| | | | | | | | | Traffic (S1,D1,| | | | | Seq. no(n - 1))| | | | | X<------------| | | | | | | | |Traffic (D1,S1,Seq. no(n - 1))| | | | X<------------------------| | | | | | | | | | | | | | | |FLOW_MOD(D1) | | | | |<---------------------| | | | | | | | | |FLOW_MOD(S1) | | | | |<---------------------| | | | | | | | Traffic (D1,S1,Seq. no(n))| | | |<-----------------------------| | | | | | | | | | Traffic (S1,D1,| | | | | Seq. no(n))| | | | |<---------------| | | | | | | | | | | |<Stop the test| | | | | after recv. | | | | | traffic upon| | | | | failover> |Bhuvaneswaran, et al. Informational [Page 62] RFC 8456 SDN Controller Benchmarking Methodology October 2018
Legend:
G-ARP: Gratuitous ARP message Seq. no: Sequence number Sa: Neighbor switch of the switch that was brought down
Discussion:
The time difference between the last valid frame received before the traffic loss (packet with sequence number x) and the first frame received after the traffic loss (packet with sequence number n) will provide the Network Re-provisioning Time.
Note that the trial is valid only when the controller provisions the alternate path upon network failure.Acknowledgments
The authors would like to thank the following individuals for providing their valuable comments regarding the earlier draft versions of this document: Al Morton (AT&T), Sandeep Gangadharan (HP), M. Georgescu (NAIST), Andrew McGregor (Google), Scott Bradner, Jay Karthik (Cisco), Ramki Krishnan (VMware), Boris Khasanov (Huawei), and Brian Castelli (Spirent).Bhuvaneswaran, et al. Informational [Page 63] RFC 8456 SDN Controller Benchmarking Methodology October 2018 Authors' Addresses
Bhuvaneswaran Vengainathan Veryx Technologies Inc. 1 International Plaza, Suite 550 Philadelphia, PA 19113 United States of America
Email: bhuvaneswaran.vengainathan@veryxtech.com
Anton Basil Veryx Technologies Inc. 1 International Plaza, Suite 550 Philadelphia, PA 19113 United States of America
Email: anton.basil@veryxtech.com
Mark Tassinari Hewlett Packard Enterprise 8000 Foothills Blvd. Roseville, CA 95747 United States of America
Email: mark.tassinari@hpe.com
Vishwas Manral NanoSec Co 3350 Thomas Rd. Santa Clara, CA 95054 United States of America
Email: vishwas.manral@gmail.com
Sarah Banks VSS Monitoring 930 De Guigne Drive Sunnyvale, CA 94085 United States of America
Email: sbanks@encrypted.netBhuvaneswaran, et al. Informational [Page 64]