GENWiki

Premier IT Outsourcing and Support Services within the UK

User Tools

Site Tools


rfc:rfc6645

Internet Engineering Task Force (IETF) J. Novak Request for Comments: 6645 Cisco Systems, Inc. Category: Informational July 2012 ISSN: 2070-1721

                 IP Flow Information Accounting and
                  Export Benchmarking Methodology

Abstract

 This document provides a methodology and framework for quantifying
 the performance impact of the monitoring of IP flows on a network
 device and the export of this information to a Collector.  It
 identifies the rate at which the IP flows are created, expired, and
 successfully exported as a new performance metric in combination with
 traditional throughput.  The metric is only applicable to the devices
 compliant with RFC 5470, "Architecture for IP Flow Information
 Export".  The methodology quantifies the impact of the IP flow
 monitoring process on the network equipment.

Status of This Memo

 This document is not an Internet Standards Track specification; it is
 published for informational purposes.
 This document is a product of the Internet Engineering Task Force
 (IETF).  It represents the consensus of the IETF community.  It has
 received public review and has been approved for publication by the
 Internet Engineering Steering Group (IESG).  Not all documents
 approved by the IESG are a candidate for any level of Internet
 Standard; see Section 2 of RFC 5741.
 Information about the current status of this document, any
 errata, and how to provide feedback on it may be obtained at
 http://www.rfc-editor.org/info/rfc6645.

Novak Informational [Page 1] RFC 6645 Flow Monitoring Benchmarking July 2012

Copyright Notice

 Copyright (c) 2012 IETF Trust and the persons identified as the
 document authors.  All rights reserved.
 This document is subject to BCP 78 and the IETF Trust's Legal
 Provisions Relating to IETF Documents
 (http://trustee.ietf.org/license-info) in effect on the date of
 publication of this document.  Please review these documents
 carefully, as they describe your rights and restrictions with respect
 to this document.  Code Components extracted from this document must
 include Simplified BSD License text as described in Section 4.e of
 the Trust Legal Provisions and are provided without warranty as
 described in the Simplified BSD License.

Novak Informational [Page 2] RFC 6645 Flow Monitoring Benchmarking July 2012

Table of Contents

 1. Introduction ....................................................4
 2. Terminology .....................................................5
    2.1. Existing Terminology .......................................5
    2.2. New Terminology ............................................6
 3. Flow Monitoring Performance Benchmark ...........................8
    3.1. Definition .................................................8
    3.2. Device Applicability .......................................8
    3.3. Measurement Concept ........................................8
    3.4. The Measurement Procedure Overview .........................9
 4. Measurement Setup ..............................................11
    4.1. Measurement Topology ......................................11
    4.2. Baseline DUT Setup ........................................13
    4.3. Flow Monitoring Configuration .............................13
    4.4. Collector .................................................19
    4.5. Sampling ..................................................19
    4.6. Frame Formats .............................................19
    4.7. Frame Sizes ...............................................20
    4.8. Flow Export Data Packet Sizes .............................20
    4.9. Illustrative Test Setup Examples ..........................20
 5. Flow Monitoring Throughput Measurement Methodology .............22
    5.1. Flow Monitoring Configuration .............................23
    5.2. Traffic Configuration .....................................24
    5.3. Cache Population ..........................................25
    5.4. Measurement Time Interval .................................25
    5.5. Flow Export Rate Measurement ..............................26
    5.6. The Measurement Procedure .................................27
 6. RFC 2544 Measurements ..........................................28
    6.1. Flow Monitoring Configuration..............................28
    6.2. Measurements with the Flow Monitoring Throughput Setup ....29
    6.3. Measurements with Fixed Flow Export Rate...................29
 7. Flow Monitoring Accuracy .......................................30
 8. Evaluating Flow Monitoring Applicability .......................31
 9. Acknowledgements ...............................................32
 10. Security Considerations .......................................32
 11. References ....................................................33
    11.1. Normative References .....................................33
    11.2. Informative References ...................................33
 Appendix A. Recommended Report Format .............................35
 Appendix B. Miscellaneous Tests ...................................36
     B.1. DUT Under Traffic Load ...................................36
     B.2. In-Band Flow Export ......................................36
     B.3. Variable Packet Rate .....................................37
     B.4. Bursty Traffic ...........................................37
     B.5. Various Flow Monitoring Configurations ...................38
     B.6. Tests with Bidirectional Traffic .........................38
     B.7. Instantaneous Flow Export Rate ...........................39

Novak Informational [Page 3] RFC 6645 Flow Monitoring Benchmarking July 2012

1. Introduction

 Monitoring IP flows (Flow monitoring) is defined in the "Architecture
 for IP Flow Information Export" [RFC5470] and related IPFIX documents
 specified in Section 1.2 of [RFC5470].  It analyzes the traffic using
 predefined fields from the packet header as keys and stores the
 traffic and other internal information in the DUT (Device Under Test)
 memory.  This cached flow information is then formatted into records
 (see Section 2.1 for term definitions) and exported from the DUT to
 an external data collector for analysis.  More details on the
 measurement architecture are provided in Section 3.3.
 Flow monitoring on network devices is widely deployed and has
 numerous uses in both service-provider and enterprise segments as
 detailed in the "Requirements for IP Flow Information Export (IPFIX)"
 [RFC3917].  This document provides a methodology for measuring Flow
 monitoring performance so that network operators have a framework to
 measure the impact on the network and network equipment.
 This document's goal is to provide a series of methodology
 specifications for the measurement of Flow monitoring performance in
 a way that is comparable amongst various implementations, platforms,
 and vendor devices.
 Flow monitoring is, in most cases, run on network devices that also
 forward packets.  Therefore, this document also provides the
 methodology for [RFC2544] measurements in the presence of Flow
 monitoring.  It is applicable to IPv6 and MPLS traffic with their
 specifics defined in [RFC5180] and [RFC5695], respectively.
 This document specifies a methodology to measure the maximum IP Flow
 Export Rate that a network device can sustain without impacting the
 Forwarding Plane, without losing any IP flow information and without
 compromising IP flow accuracy (see Section 7 for details).
 [RFC2544], [RFC5180], and [RFC5695] specify benchmarking of network
 devices forwarding IPv4, IPv6, and MPLS [RFC3031] traffic,
 respectively.  The methodology specified in this document stays the
 same for any traffic type.  The only restriction may be the DUT's
 lack of support for Flow monitoring of a particular traffic type.
 A variety of different DUT architectures exist that are capable of
 Flow monitoring and export.  As such, this document does not attempt
 to list the various white-box variables (e.g., CPU load, memory
 utilization, hardware resources utilization, etc.) that could be
 gathered as they always help in comparison evaluations.  A more
 complete understanding of the stress points of a particular device

Novak Informational [Page 4] RFC 6645 Flow Monitoring Benchmarking July 2012

 can be attained using this internal information, and the tester MAY
 choose to gather this information during the measurement iterations.

2. Terminology

 The terminology used in this document is based on that defined in
 [RFC5470], [RFC2285], and [RFC1242], as summarized in Section 2.1.
 The only new terms needed for this methodology are defined in Section
 2.2.
 Additionally, the key words "MUST", "MUST NOT", "REQUIRED", "SHALL",
 "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and
 "OPTIONAL" in this document are to be interpreted as described  in
 RFC 2119 [RFC2119].

2.1. Existing Terminology

  Device Under Test (DUT)   [RFC2285, Section 3.1.1]
  Flow                      [RFC5101, Section 2]
  Flow Key                  [RFC5101, Section 2]
  Flow Record               [RFC5101, Section 2]
  Template Record           [RFC5101, Section 2]
  Observation Point         [RFC5470, Section 2]
  Metering Process          [RFC5470, Section 2]
  Exporting Process         [RFC5470, Section 2]
  Exporter                  [RFC5470, Section 2]
  Collector                 [RFC5470, Section 2]
  Control Information       [RFC5470, Section 2]
  Data Stream               [RFC5470, Section 2]
  Flow Expiration           [RFC5470, Section 5.1.1]
  Flow Export               [RFC5470, Section 5.1.2]
  Throughput                [RFC1242, Section 3.17]

Novak Informational [Page 5] RFC 6645 Flow Monitoring Benchmarking July 2012

2.2. New Terminology

2.2.1. Cache

 Definition:
    Memory area held and dedicated by the DUT to store Flow
    information prior to the Flow Expiration.

2.2.2. Cache Size

 Definition:
    The size of the Cache in terms of how many entries the Cache can
    hold.
 Discussion:
    This term is typically represented as a configurable option in the
    particular Flow monitoring implementation.  Its highest value will
    depend on the memory available in the network device.
 Measurement units:
    Number of Cache entries

2.2.3. Active Timeout

 Definition:
    For long-running Flows, the time interval after which the Metering
    Process expires a Cache entry to ensure Flow data is regularly
    updated.
 Discussion:
    This term is typically presented as a configurable option in the
    particular Flow monitoring implementation.  See Section 5.1.1 of
    [RFC5470] for a more detailed discussion.
    Flows are considered long running when they last longer than
    several multiples of the Active Timeout.  If the Active Timeout is
    zero, then Flows are considered long running if they contain many
    more packets (tens of packets) than usually observed in a single
    transaction.
 Measurement units:
    Seconds

Novak Informational [Page 6] RFC 6645 Flow Monitoring Benchmarking July 2012

2.2.4. Idle Timeout

 Definition:
    The time interval used by the Metering Process to expire an entry
    from the Cache when no more packets belonging to that specific
    Cache entry have been observed during the interval.
 Discussion:
    Idle Timeout is typically represented as a configurable option in
    the particular Flow monitoring implementation.  See Section 5.1.1
    of [RFC5470] for more detailed discussion.  Note that some
    documents in the industry refer to "Idle Timeout" as "inactive
    timeout".
 Measurement units:
    Seconds

2.2.5. Flow Export Rate

 Definition:
    The number of Cache entries that expire from the Cache (as defined
    by the Flow Expiration term) and are exported to the Collector
    within a measurement time interval.  There SHOULD NOT be any
    export filtering, so that all the expired Cache entries are
    exported.  If there is export filtering and it can't be disabled,
    this MUST be indicated in the measurement report.
    The measured Flow Export Rate MUST include both the Data Stream
    and the Control Information, as defined in Section 2 of [RFC5470].
 Discussion:
    The Flow Export Rate is measured using Flow Export data observed
    at the Collector by counting the exported Flow Records during the
    measurement time interval (see Section 5.4).  The value obtained
    is an average of the instantaneous export rates observed during
    the measurement time interval.  The smallest possible measurement
    interval (if attempting to measure a nearly instantaneous export
    rate rather than average export rate on the DUT) is limited by the
    export capabilities of the particular Flow monitoring
    implementation (when physical-layer issues between the DUT and the
    Collector are excluded).
 Measurement units:
    Number of Flow Records per second

Novak Informational [Page 7] RFC 6645 Flow Monitoring Benchmarking July 2012

3. Flow Monitoring Performance Benchmark

3.1. Definition

 Flow Monitoring Throughput
 Definition:
    The maximum Flow Export Rate the DUT can sustain without losing a
    single Cache entry.  Additionally, for packet forwarding devices,
    the maximum Flow Export Rate the DUT can sustain without dropping
    packets in the Forwarding Plane (see Figure 1).
 Measurement units:
    Number of Flow Records per second
 Discussion:
    The losses of Cache entries, or forwarded packets per this
    definition are assumed to happen due to the lack of DUT resources
    to process any additional traffic information or lack of resources
    to process Flow Export data.  The physical-layer issues, like
    insufficient bandwidth from the DUT to the Collector or lack of
    Collector resources, MUST be excluded as detailed in Section 4.

3.2. Device Applicability

 The Flow monitoring performance metric is applicable to network
 devices that deploy the architecture described in [RFC5470].  These
 devices can be network packet forwarding devices or appliances that
 analyze traffic but do not forward traffic (e.g., probes, sniffers,
 replicators).
 This document does not intend to measure Collector performance, it
 only requires sufficient Collector resources (as specified in Section
 4.4) in order to measure the DUT characteristics.

3.3. Measurement Concept

 Figure 1 presents the functional block diagram of the DUT.  The
 traffic in the figure represents test traffic sent to the DUT and
 forwarded by the DUT, if possible.  When testing devices that do not
 act as network packet forwarding devices (such as probes, sniffers,
 and replicators), the Forwarding Plane is simply an Observation Point
 as defined in Section 2 of [RFC5470].  The Throughput of such devices
 will always be zero, and the only applicable performance metric is
 the Flow Monitoring Throughput.  Netflow is specified by [RFC3954].

Novak Informational [Page 8] RFC 6645 Flow Monitoring Benchmarking July 2012

            +------------------------- +
            | IPFIX | NetFlow | Others |
            +------------------------- +
            |            ^             |
            |       Flow Export        |
            |            ^             |
            |     +-------------+      |
            |     | Monitoring  |      |
            |     |   Plane     |      |
            |     +-------------+      |
            |            ^             |
            |     traffic information  |
            |            ^             |
            |     +-------------+      |
            |     |             |      |
 traffic ---|---->| Forwarding  |------|---->
            |     |    Plane    |      |
            |     +-------------+      |
            |                          |
            |           DUT            |
            +------------------------- +
 Figure 1.  The Functional Block Diagram of the DUT
 Flow monitoring is represented in Figure 1 by the Monitoring Plane;
 it is enabled as specified in Section 4.3.  It uses the traffic
 information provided by the Forwarding Plane and configured Flow Keys
 to create Cache entries representing the traffic forwarded (or
 observed) by the DUT in the DUT Cache.  The Cache entries are expired
 from the Cache depending on the Cache configuration (e.g., the Active
 and Idle Timeouts, the Cache Size), number of Cache entries, and the
 traffic pattern.  The Cache entries are used by the Exporting Process
 to format the Flow Records, which are then exported from the DUT to
 the Collector (see Figure 2 in Section 4).
 The Forwarding Plane and Monitoring Plane represent two separate
 functional blocks, each with its own performance capability.  The
 Forwarding Plane handles user data packets and is fully characterized
 by the metrics defined by [RFC1242].
 The Monitoring Plane handles Flows that reflect the analyzed traffic.
 The metric for Monitoring Plane performance is the Flow Export Rate,
 and the benchmark is the Flow Monitoring Throughput.

3.4. The Measurement Procedure Overview

 The measurement procedure is fully specified in Sections 4, 5, and 6.
 This section provides an overview of principles for the measurements.

Novak Informational [Page 9] RFC 6645 Flow Monitoring Benchmarking July 2012

 The basic measurement procedure of the performance characteristics of
 a DUT with Flow monitoring enabled is a conventional Throughput
 measurement using a search algorithm to determine the maximum packet
 rate at which none of the offered packets and corresponding Flow
 Records are dropped by the DUT as described in [RFC1242] and Section
 26.1 of [RFC2544].
 The DUT with Flow monitoring enabled contains two functional blocks
 that need to be measured using characteristics applicable to one or
 both blocks (see Figure 1).  See Sections 3.4.1 and 3.4.2 for further
 discussion.
 On one hand, the Monitoring Plane and Forwarding Plane (see Figure 1)
 need to be looked at as two independent blocks, and the performance
 of each measured independently.  On the other hand, when measuring
 the performance of one, the status and performance of the other MUST
 be known and benchmarked when both are present.

3.4.1. Monitoring Plane Performance Measurement

 The Flow Monitoring Throughput MUST be (and can only be) measured
 with one packet per Flow as specified in Section 5.  This traffic
 type represents the most demanding traffic from the Flow monitoring
 point of view and will exercise the Monitoring Plane (see Figure 1)
 of the DUT most.  In this scenario, every packet seen by the DUT
 creates a new Cache entry and forces the DUT to fill the Cache
 instead of just updating the packet and byte counters of an already
 existing Cache entry.
 The exit criteria for the Flow Monitoring Throughput measurement are
 one of the following (e.g., if any of the conditions are reached):
 a. The Flow Export Rate at which the DUT starts to lose Flow
    Information or the Flow Information gets corrupted.
 b. The Flow Export Rate at which the Forwarding Plane starts to drop
    or corrupt packets (if the Forwarding Plane is present).
 A corrupted packet here means packet header corruption (resulting in
 the cyclic redundancy check failure on the transmission level and
 consequent packet drop) or packet payload corruption, which leads to
 lost application-level data.

3.4.2. Forwarding Plane Performance Measurement

 The Forwarding Plane (see Figure 1) performance metrics are fully
 specified by [RFC1242] and MUST be measured accordingly.  A detailed
 traffic analysis (see below) with relation to Flow monitoring MUST be

Novak Informational [Page 10] RFC 6645 Flow Monitoring Benchmarking July 2012

 performed prior of any [RFC2544] measurements.  Most importantly, the
 Flow Export Rate caused by the test traffic during an [RFC2544]
 measurement MUST be known and reported.
 The required test traffic analysis mainly involves the following:
 a. Which packet header parameters are incremented or changed during
    traffic generation.
 b. Which Flow Keys the Flow monitoring configuration uses to generate
    Flow Records.
 The performance metrics described in RFC 1242 can be measured in one
 of the three modes:
 a. As a baseline of forwarding performance without Flow monitoring.
 b. At a certain level of Flow monitoring activity specified by a Flow
    Export Rate lower than the Flow Monitoring Throughput.
 c. At the maximum level of Flow monitoring performance, e.g., using
    traffic conditions representing a measurement of Flow Monitoring
    Throughput.
 The above mentioned measurement mode in point a.  represents an
 ordinary Throughput measurement specified in RFC 2544.  The details
 of how to set up the measurements in points b. and c. are given in
 Section 6.

4. Measurement Setup

 This section concentrates on the setup of all components necessary to
 perform Flow monitoring performance measurement.  The recommended
 reporting format can be found in Appendix A.

4.1. Measurement Topology

 The measurement topology described in this section is applicable only
 to the measurements with packet forwarding network devices.  The
 possible architectures and implementation of the traffic monitoring
 appliances (see Section 3.2) are too various to be covered in this
 document.  Instead of the Forwarding Plane, these appliances
 generally have some kind of feed (e.g., an optical splitter, an
 interface sniffing traffic on a shared media, or an internal channel
 on the DUT providing a copy of the traffic) providing the information
 about the traffic necessary for Flow monitoring analysis.  The
 measurement topology then needs to be adjusted to the appliance
 architecture and MUST be part of the measurement report.

Novak Informational [Page 11] RFC 6645 Flow Monitoring Benchmarking July 2012

 The measurement setup is identical to that used by [RFC2544], with
 the addition of a Collector to analyze the Flow Export (see Figure
 2).
 In the measurement topology with unidirectional traffic, the traffic
 is transmitted from the sender to the receiver through the DUT.  The
 received traffic is analyzed to check that it is identical to the
 generated traffic.
 The ideal way to implement the measurement is by using a single
 device to provide the sender and receiver capabilities with one
 sending port and one receiving port.  This allows for an easy check
 as to whether all the traffic sent by the sender was re-transmitted
 by the DUT and received at the receiver.
                     +-----------+
                     |           |
                     | Collector |
                     |           |
                     |Flow Record|
                     | analysis  |
                     |           |
                     +-----------+
                           ^
                           | Flow Export
                           |
                           | Export Interface
 +--------+         +-------------+          +----------+
 |        |         |             |          | traffic  |
 | traffic|      (*)|             |          | receiver |
 | sender |-------->|     DUT     |--------->|          |
 |        |         |             |          | traffic  |
 |        |         |             |          | analysis |
 +--------+         +-------------+          +----------+
 Figure 2.  Measurement Topology with Unidirectional Traffic
 The DUT's export interface (connecting the Collector) MUST NOT be
 used for forwarding test traffic but only for the Flow Export data
 containing the Flow Records.  In all measurements, the export
 interface MUST have enough bandwidth to transmit Flow Export data
 without congestion.  In other words, the export interface MUST NOT be
 a bottleneck during the measurement.
 The traffic receiver MUST have sufficient resources to measure all
 test traffic transferred successfully by the DUT.  This may be
 checked through measurements with and without the DUT.

Novak Informational [Page 12] RFC 6645 Flow Monitoring Benchmarking July 2012

 Note that more complex topologies might be required.  For example, if
 the effects of enabling Flow monitoring on several interfaces is of
 concern, or the maximum speed of media transmission is less than the
 DUT Throughput, the topology can be expanded with several input and
 output ports.  However, the topology MUST be clearly written in the
 measurement report.

4.2. Baseline DUT Setup

 The baseline DUT setup and the way the setup is reported in the
 measurement results is fully specified in Section 7 of [RFC2544].
 The baseline DUT configuration might include other features, like
 packet filters or quality of service on the input and/or output
 interfaces, if there is the need to study Flow monitoring in the
 presence of those features.  The Flow monitoring measurement
 procedures do not change in this case.  Consideration needs to be
 made when evaluating measurement results to take into account the
 possible change of packet rates offered to the DUT and Flow
 monitoring after application of the features to the configuration.
 Any such feature configuration MUST be part of the measurement
 report.
 The DUT export interface (see Figure 2) SHOULD be configured with
 sufficient output buffers to avoid dropping the Flow Export data due
 to a simple lack of resources in the interface hardware.  The applied
 configuration MUST be part of the measurement report.
 The test designer has the freedom to run tests in multiple
 configurations.  It is therefore possible to run both non-production
 and real deployment configurations in the laboratory, according to
 the needs of the tester.  All configurations MUST be part of the
 measurement report.

4.3. Flow Monitoring Configuration

 This section covers all of the aspects of the Flow monitoring
 configuration necessary on the DUT in order to perform the Flow
 monitoring performance measurement.  The necessary configuration has
 a number of components (see [RFC5470]), namely Observation Points,
 Metering Process, and Exporting Process as detailed below.
 The DUT MUST support the Flow monitoring architecture as specified by
 [RFC5470].  The DUT SHOULD support IPFIX [RFC5101] to allow a
 meaningful results comparison due to the standardized export
 protocol.

Novak Informational [Page 13] RFC 6645 Flow Monitoring Benchmarking July 2012

 The DUT configuration, any existing Cache, and Cache entries MUST be
 erased before the application of any new configuration for the
 currently executed measurement.

4.3.1. Observation Points

 The Observation Points specify the interfaces and direction in which
 the Flow monitoring traffic analysis is to be performed.
 The (*) in Figure 2 designates the Observation Points in the default
 configuration.  Other DUT Observation Points might be configured
 depending on the specific measurement needs as follows:
 a. ingress port/ports only
 b. egress port/ports only
 c. both ingress and egress
 This test topology corresponds to unidirectional traffic only with
 traffic analysis performed on the input and/or output interface.
 Testing with bidirectional traffic is discussed in Appendix B.
 Generally, the placement of Observation Points depends upon the
 position of the DUT in the deployed network and the purpose of Flow
 monitoring.  See [RFC3917] for detailed discussion.  The measurement
 procedures are otherwise the same for all these possible
 configurations.
 In the case of both ingress and egress Flow monitoring being enabled
 on one DUT, the resulting analysis should consider that each Flow
 will be represented in the DUT Cache by two Flow Records (one for
 each direction).  Therefore, the Flow Export will also contain those
 two Flow Records.
 If more than one Observation Point for one direction is defined on
 the DUT, the traffic passing through each of the Observation Points
 MUST be configured in such a way that it creates Flows and Flow
 Records that do not overlap.  Each packet (or set of packets if
 measuring more than one packet per Flow - see Section 6.3.1) sent to
 the DUT on different ports still creates one unique Flow Record.
 The specific Observation Points and associated monitoring direction
 MUST be included as part of the measurement report.

4.3.2. Metering Process

 The Metering Process MUST be enabled in order to create the Cache in
 the DUT and configure the Cache related parameters.

Novak Informational [Page 14] RFC 6645 Flow Monitoring Benchmarking July 2012

 The Cache Size available to the DUT MUST be known and taken into
 account when designing the measurement as specified in Section 5.
 Typically, the Cache Size will be present in the "show" commands of
 the Flow monitoring process, in either the actual configuration or
 the product documentation from the DUT vendor.  The Cache Size MUST
 have a fixed value for the entire duration of the measurement.  This
 method is not applicable to benchmarking any Flow monitoring
 applications that dynamically change their Cache Size.
 The configuration of the Metering Process MUST be included as part of
 the measurement report.  For example, when a Flow monitoring
 implementation uses timeouts to expire entries from the Cache, the
 Cache's Idle and Active Timeouts MUST be known and taken into account
 when designing the measurement as specified in Section 5.  If the
 Flow monitoring implementation allows only timeouts equal to zero
 (e.g., immediate timeout or non-existent Cache), then the measurement
 conditions in Section 5 are fulfilled inherently without any
 additional configuration.  The DUT simply exports information about
 every packet immediately, subject to the Flow Export Rate definition
 in Section 2.2.5.
 If the Flow monitoring implementation allows configuration of
 multiple Metering Processes on a single DUT, the exact configuration
 of each process MUST be included in the measurement report.  Only
 measurements with the same number of Metering Processes can be
 compared.
 The Cache Size and the Idle and Active Timeouts MUST be included in
 the measurement report.

4.3.3. Exporting Process

 The Exporting Process MUST be configured in order to export the Flow
 Record data to the Collector.
 The Exporting Process MUST be configured in such a way that all Flow
 Records from all configured Observation Points are exported towards
 the Collector, after the expiration policy, which is composed of the
 Idle and Active Timeouts and Cache Size.
 The Exporting Process SHOULD be configured with IPFIX [RFC5101] as
 the protocol used to format the Flow Export data.  If the Flow
 monitoring implementation does not support IPFIX, proprietary
 protocols MAY be used.  Only measurements with the same export
 protocol SHOULD be compared since the protocols may differ in their
 export efficiency.  The export efficiency might also be influenced by
 the Template Record used and the ordering of the individual export
 fields within the template.

Novak Informational [Page 15] RFC 6645 Flow Monitoring Benchmarking July 2012

 The Template Records used by the tested implementations SHOULD be
 analyzed and documented as part of the measurement report.  Ideally,
 only tests with same Template Records should be compared.
 Various Flow monitoring implementations might use different default
 values regarding the export of Control Information [RFC5470];
 therefore, the Flow Export corresponding to Control Information
 SHOULD be analyzed and reported as a separate item on the measurement
 report.  The export of Control Information SHOULD always be
 configured consistently across all testing and configured to the
 minimal possible value.  Ideally, just one set of Control Information
 should be exported during each measurement.  Note that Control
 Information includes options and Template Records [RFC5470].
 Section 10 of [RFC5101] and Section 8.1 of [RFC5470] discuss the
 possibility of deploying various transport-layer protocols to deliver
 Flow Export data from the DUT to the Collector.  The selected
 protocol MUST be included in the measurement report.  Only benchmarks
 with the same transport-layer protocol SHOULD be compared.  If the
 Flow monitoring implementation allows the use of multiple transport-
 layer protocols, each of the protocols SHOULD be measured in a
 separate measurement run and the results reported independently in
 the measurement report.
 If a reliable transport protocol is used for the transmission of the
 Flow Export data from the DUT, the configuration of the Transport
 session MUST allow for non-blocking data transmission.  An example of
 parameters to look at would be the TCP window size and maximum
 segment size (MSS).  The most substantial transport-layer parameters
 should be included in the measurement report.

4.3.4. Flow Records

 A Flow Record contains information about a specific Flow observed at
 an Observation Point.  A Flow Record contains measured properties of
 the Flow (e.g., the total number of bytes for all the Flow packets)
 and usually characteristic properties of the Flow (e.g., source IP
 address).
 The Flow Record definition is implementation specific.  A Flow
 monitoring implementation might allow for only a fixed Flow Record
 definition, based on the most common IP parameters in the IPv4 or
 IPv6 headers -- for example, source and destination IP addresses, IP
 protocol numbers, or transport-level port numbers.  Another
 implementation might allow the user to define their own arbitrary
 Flow Record to monitor the traffic.  The only requirement for the
 measurements defined in this document is the need for a large

Novak Informational [Page 16] RFC 6645 Flow Monitoring Benchmarking July 2012

 number of Cache entries in the Cache.  The Flow Keys needed to
 achieve that will typically be source and destination IP addresses
 and transport-level port numbers.
 The recommended full IPv4, IPv6, or MPLS Flow Record is shown below.
 The IP address indicates either IPv4 or IPv6, depending on the
 traffic type being tested.  The Flow Record configuration is Flow
 monitoring implementation-specific; therefore, the examples below
 cannot provide an exact specification of individual entries in each
 Flow Record.  The best set of key fields to use is left to the test
 designer using the capabilities of the specific Flow monitoring
 implementation.
    Flow Keys:
       Source IP address
       Destination IP address
       MPLS label (for MPLS traffic type only)
       Transport-layer source port
       Transport-layer destination port
       IP protocol number (IPv6 next header)
       IP type of service (IPv6 traffic class)
    Other fields:
       Packet counter
       Byte counter
    Table 1: Recommended Configuration
 If the Flow monitoring allows for user-defined Flow Records, the
 minimal Flow Record configurations allowing large numbers of Cache
 entries are, for example:
    Flow Keys:
       Source IP address
       Destination IP address
    Other fields:
       Packet counter
 or:
    Flow Keys:
       Transport-layer source port
       Transport-layer destination port
    Other fields:
       Packet counter
    Table 2: User-Defined Configuration

Novak Informational [Page 17] RFC 6645 Flow Monitoring Benchmarking July 2012

 The Flow Record configuration MUST be clearly noted in the
 measurement report.  The Flow Monitoring Throughput measurements on
 different DUTs, or different Flow monitoring implementations, MUST be
 only compared for exactly the same Flow Record configuration.

4.3.5. Flow Monitoring with Multiple Configurations

 The Flow monitoring architecture as specified in [RFC5470] allows for
 more complicated configurations with multiple Metering and Exporting
 Processes on a single DUT.  Depending on the particular Flow
 monitoring implementation, it might affect the measured DUT
 performance.  Therefore, the measurement report should contain
 information about how many Metering and Exporting Processes were
 configured on the DUT for the selected Observation Points.
 The examples of such possible configurations are:
 a. Several Observation Points with a single Metering Process and a
    single Exporting Process.
 b. Several Observation Points, each with one Metering Process but all
    using just one instance of Exporting Process.
 c. Several Observation Points with per-Observation-Point Metering
    Process and Exporting Process.

4.3.6. MPLS Measurement Specifics

 The Flow Record configuration for measurements with MPLS encapsulated
 traffic SHOULD contain the MPLS label.  For this document's purposes,
 "MPLS Label" is the entire 4 byte MPLS header.  Typically, the label
 of the interest will be at the top of the label stack, but this
 depends on the details of the MPLS test setup.
 The tester SHOULD ensure that the data received by the Collector
 contains the expected MPLS labels.
 The MPLS forwarding performance document [RFC5695] specifies a number
 of possible MPLS label operations to test.  The Observation Points
 MUST be placed on all the DUT test interfaces where the particular
 MPLS label operation takes place.  The performance measurements
 SHOULD be performed with only one MPLS label operation at the time.
 The DUT MUST be configured in such a way that all the traffic is
 subject to the measured MPLS label operation.

Novak Informational [Page 18] RFC 6645 Flow Monitoring Benchmarking July 2012

4.4. Collector

 The Collector is needed in order to capture the Flow Export data,
 which allows the Flow Monitoring Throughput to be measured.
 The Collector can be used exclusively as a capture device, providing
 just hexadecimal format of the Flow Export data.  In such a case, it
 does not need to have any additional Flow Export decoding
 capabilities and all the decoding is done offline.
 However, if the Collector is also used to decode the Flow Export
 data, it SHOULD support IPFIX [RFC5101] for meaningful results
 analysis.  If proprietary Flow Export is deployed, the Collector MUST
 support it; otherwise, the Flow Export data analysis is not possible.
 The Collector MUST be capable of capturing the export packets sent
 from the DUT at the full rate without losing any of them.  When using
 reliable transport protocols (see also Section 4.3.3) to transmit
 Flow Export data, the Collector MUST have sufficient resources to
 guarantee non-blocking data transmission on the transport-layer
 session.
 During the analysis, the Flow Export data needs to be decoded and the
 received Flow Records counted.
 The capture buffer MUST be cleared at the beginning of each
 measurement.

4.5. Sampling

 Packet sampling and flow sampling is out of the scope of this
 document.  This document applies to situations without packet, flow,
 or export sampling.

4.6. Frame Formats

 Flow monitoring itself is not dependent in any way on the media used
 on the input and output ports.  Any media can be used as supported by
 the DUT and the test equipment.  This applies both to data forwarding
 interfaces and to the export interface (see Figure 2).
 At the time of this writing, the most common transmission media and
 corresponding frame formats (e.g., Ethernet, Packet over SONET) for
 IPv4, IPv6, and MPLS traffic are specified within [RFC2544],
 [RFC5180], and [RFC5695].
 The presented frame formats MUST be recorded in the measurement
 report.

Novak Informational [Page 19] RFC 6645 Flow Monitoring Benchmarking July 2012

4.7. Frame Sizes

 Frame sizes of the traffic to be analyzed by the DUT are specified in
 Section 9 of [RFC2544] for Ethernet type interfaces (64, 128, 256,
 1024, 1280, 1518 bytes) and in Section 5 of [RFC5180] for Packet over
 SONET interfaces (47, 64, 128, 256, 1024, 1280, 1518, 2048, 4096
 bytes).
 When measuring with large frame sizes, care needs to be taken to
 avoid any packet fragmentation on the DUT interfaces that could
 negatively affect measured performance values.
 The presented frame sizes MUST be recorded in the measurement report.

4.8. Flow Export Data Packet Sizes

 The Flow monitoring performance will be affected by the packet size
 that the particular implementation uses to transmit Flow Export data
 to the Collector.  The used packet size MUST be part of the
 measurement report and only measurements with same packet sizes
 SHOULD be compared.
 The DUT export interface (see Figure 2) maximum transmission unit
 (MTU) SHOULD be configured to the largest available value for the
 media.  The Flow Export MTU MUST be recorded in the measurement
 report.

4.9. Illustrative Test Setup Examples

 The examples below represent a hypothetical test setup to clarify the
 use of Flow monitoring parameters and configuration, together with
 traffic parameters to test Flow monitoring.  The actual benchmarking
 specifications are in Sections 5 and 6.

4.9.1. Example 1 - Idle Timeout Flow Expiration

 The traffic generator sends 1000 packets per second in 10000 defined
 streams, each stream identified by a unique destination IP address.
 Therefore, each stream has a packet rate of 0.1 packets per second.
 The packets are sent in a round-robin fashion (stream 1 to 10000)
 while incrementing the destination IP address for each sent packet.
 After a packet for stream 10000 is sent, the next packet destination
 IP address corresponds to stream 1's address again.
 The configured Cache Size is 20000 Flow Records.  The configured
 Active Timeout is 100 seconds, and the Idle Timeout is 5 seconds.

Novak Informational [Page 20] RFC 6645 Flow Monitoring Benchmarking July 2012

 Flow monitoring on the DUT uses the destination IP address as the
 Flow Key.
 A packet with the destination IP address equal to A is sent every 10
 seconds, so the Cache entry is refreshed in the Cache every 10
 seconds.  However, the Idle Timeout is 5 seconds, so the Cache
 entries will expire from the Cache due to the Idle Timeout, and when
 a new packet is sent with the same IP address A, it will create a new
 entry in the Cache.  This behavior depends upon the design and
 efficiency of the Cache ager, and incidences of multi-packet flows
 observed during this test should be noted.
 The measured Flow Export Rate in this case will be 1000 Flow Records
 per second since every single sent packet will always create a new
 Cache entry and 1000 packets per second are sent.
 The expected number of Cache entries in the Cache during the whole
 measurement is around 5000.  It corresponds to the Idle Timeout being
 5 seconds; during those five seconds, 5000 entries are created.  This
 expectation might change in real measurement setups with large Cache
 Sizes and a high packet rate where the DUT's actual export rate might
 be limited and lower than the Flow Expiration activity caused by the
 traffic offered to the DUT.  This behavior is entirely
 implementation-specific.

4.9.2. Example 2 - Active Timeout Flow Expiration

 The traffic generator sends 1000 packets per second in 100 defined
 streams, each stream identified by a unique destination IP address.
 Each stream has a packet rate of 10 packets per second.  The packets
 are sent in a round-robin fashion (stream 1 to 100) while
 incrementing the destination IP address for each sent packet.  After
 a packet for stream 100 is sent, the next packet destination IP
 address corresponds to stream 1's address again.
 The configured Cache Size is 1000 Flow Records.  The configured
 Active Timeout is 100 seconds.  The Idle Timeout is 10 seconds.
 Flow monitoring on the DUT uses the destination IP address as the
 Flow Key.
 After the first 100 packets are sent, 100 Cache entries will have
 been created in the Flow monitoring Cache.  The subsequent packets
 will be counted against the already created Cache entries since the
 destination IP address (Flow Key) has already been seen by the DUT
 (provided the Cache entries did not expire yet as described below).

Novak Informational [Page 21] RFC 6645 Flow Monitoring Benchmarking July 2012

 A packet with the destination IP address equal to A is sent every 0.1
 second, so the Cache entry is refreshed in the Cache every 0.1
 second, while the Idle Timeout is 10 seconds.  In this case, the
 Cache entries will not expire until the Active Timeout expires, e.g.,
 they will expire every 100 seconds and then the Cache entries will be
 created again.
 If the test measurement time is 50 seconds from the start of the
 traffic generator, then the measured Flow Export Rate is 0 since
 during this period nothing expired from the Cache.
 If the test measurement time is 100 seconds from the start of the
 traffic generator, then the measured Flow Export Rate is 1 Flow
 Record per second.
 If the test measurement time is 290 seconds from the start of the
 traffic generator, then the measured Flow Export Rate is 2/3 of a
 Flow Record per second since the Cache expired the same number of
 Flows twice (100) during the 290-seconds period.

5. Flow Monitoring Throughput Measurement Methodology

 Objective:
    To measure the Flow monitoring performance in a manner that is
    comparable between different Flow monitoring implementations.
 Metric definition:
    Flow Monitoring Throughput - see Section 3.
 Discussion:
    Different Flow monitoring implementations might choose to handle
    Flow Export from a partially empty Cache differently than in the
    case of the Cache being fully occupied.  Similarly, software- and
    hardware-based DUTs can handle the same situation as stated above
    differently.  The purpose of the benchmark measurement in this
    section is to define one measurement procedure covering all the
    possible behaviors.
    The only criteria is to measure as defined here until Flow Record
    or packet losses are seen.  The decision whether to dive deeper
    into the conditions under which the packet losses happen is left
    to the tester.

Novak Informational [Page 22] RFC 6645 Flow Monitoring Benchmarking July 2012

5.1. Flow Monitoring Configuration

 Cache Size
    Cache Size configuration is dictated by the expected position of
    the DUT in the network and by the chosen Flow Keys of the Flow
    Record.  The number of unique sets of Flow Keys that the traffic
    generator (sender) provides should be multiple times larger than
    the Cache Size.  This ensures that the existing Cache entries are
    never updated by a packet from the sender before the particular
    Flow Expiration and Flow Export.  This condition is simple to
    fulfill with linearly incremented Flow Keys (for example, IP
    addresses or transport-layer ports) where the range of values must
    be larger than the Cache Size.  When randomized traffic generation
    is in use, the generator must ensure that the same Flow Keys are
    not repeated within a range of randomly generated values.
    The Cache Size MUST be known in order to define the measurement
    circumstances properly.  Typically, the Cache Size will be found
    using the "show" commands of the Flow monitoring implementation in
    the actual configuration or in the product documentation from the
    vendor.
 Idle Timeout
    Idle Timeout is set (if configurable) to the minimum possible
    value on the DUT.  This ensures that the Cache entries are expired
    as soon as possible and exported out of the DUT Cache.  It MUST be
    known in order to define the measurement circumstances completely
    and equally across implementations.
 Active Timeout
    Active Timeout is set (if configurable) to a value equal to or
    higher than the Idle Timeout.  It MUST be known in order to define
    the measurement circumstances completely and equally across
    implementations.
 Flow Keys Definition:
    The test needs large numbers of unique Cache entries to be created
    by incrementing values of one or several Flow Keys.  The number of
    unique combinations of Flow Keys values SHOULD be several times
    larger than the DUT Cache Size.  This makes sure that any incoming
    packet will never refresh any already existing Cache entry.
 The availability of Cache Size, Idle Timeout, and Active Timeout as
 configuration parameters is implementation-specific.  If the Flow
 monitoring implementation does not support these parameters, the test
 possibilities, as specified by this document, are restricted.  Some

Novak Informational [Page 23] RFC 6645 Flow Monitoring Benchmarking July 2012

 testing might be viable if the implementation follows the guidance
 provided in the [IPFIX-CONFIG] document and is considered on a case-
 by-case basis.

5.2. Traffic Configuration

 Traffic Generation
    The traffic generator needs to increment the Flow Keys values with
    each sent packet.  This way, each packet represents one Cache
    entry in the DUT Cache.
    A particular Flow monitoring implementation might choose to deploy
    a hashing mechanism to match incoming data packets to a certain
    Flow.  In such a case, the combination of how the traffic is
    constructed and the hashing might influence the DUT Flow
    monitoring performance.  For example, if IP addresses are used as
    Flow Keys, this means there could be a performance difference for
    linearly incremented addresses (in ascending or descending order)
    as opposed to IP addresses randomized in a certain range.  If
    randomized IP address sequences are used, then the traffic
    generator needs to be able to reproduce the randomization (e.g.,
    the same set of IP addresses sent in the same order in different
    test runs) in order to compare various DUTs and Flow monitoring
    implementations.
    If the test traffic rate is below the maximum media rate for the
    particular packet size, the traffic generator MUST send the
    packets in equidistant time intervals.  Traffic generators that do
    not fulfill this condition MUST NOT and cannot be used for the
    Flow Monitoring Throughput measurement.  An example of this
    behavior is if the test traffic rate is one half of the media
    rate.  The traffic generator achieves this rate by sending packets
    each half of each second at the full media rate and sending
    nothing for the second half of each second.  In such conditions,
    it would be impossible to distinguish if the DUT failed to handle
    the Flows due to the shortage of input buffers during the burst or
    due to the limits in the Flow monitoring performance.
 Measurement Duration
    The measurement duration (e.g., how long the test traffic is sent
    to the DUT) MUST be at least two-times longer than the Idle
    Timeout; otherwise, no Flow Export would be seen.  The measurement
    duration SHOULD guarantee that the number of Cache entries created
    during the measurement exceeds the available Cache Size.

Novak Informational [Page 24] RFC 6645 Flow Monitoring Benchmarking July 2012

5.3. Cache Population

 The product of the Idle Timeout and the packet rate offered to the
 DUT (Cache population) during one measurement determines the total
 number of Cache entries in the DUT Cache during the measurement
 (while taking into account some margin for dynamic behavior during
 high DUT loads when processing the Flows).
 The Flow monitoring implementation might behave differently depending
 on the relation of the Cache population to the available Cache Size
 during the measurement.  This behavior is fully implementation-
 specific and will also be influenced if the DUT architecture is
 software based or hardware based.
 The Cache population (if it is lower or higher than the available
 Cache Size) during a particular benchmark measurement SHOULD be
 noted, and mainly only measurements with the same Cache population
 SHOULD be compared.

5.4. Measurement Time Interval

 The measurement time interval is the time value that is used to
 calculate the measured Flow Export Rate from the captured Flow Export
 data.  It is obtained as specified below.
 RFC 2544 specifies, with the precision of the packet beginning and
 ending, the time intervals to be used to measure the DUT time
 characteristics.  In the case of a Flow Monitoring Throughput
 measurement, the start and stop time needs to be clearly defined, but
 the granularity of this definition can be limited to just marking the
 start and stop time with the start and stop of the traffic generator.
 This assumes that the traffic generator and DUT are collocated and
 the variance in transmission delay from the generator to the DUT is
 negligible as compared to the total time of traffic generation.
 The measurement start time:
    the time when the traffic generator is started
 The measurement stop time: the time when the traffic generator is
    stopped
 The measurement time interval is then calculated as the difference
 (stop time) - (start time) - (Idle Timeout).
 This supposes that the Cache Size is large enough that the time
 needed to fill it with Cache entries is longer than the Idle Timeout.
 Otherwise, the time needed to fill the Cache needs to be used to
 calculate the measurement time interval in place of the Idle Timeout.

Novak Informational [Page 25] RFC 6645 Flow Monitoring Benchmarking July 2012

 Instead of measuring the absolute values of the stop and start times,
 it is possible to set up the traffic generator to send traffic for a
 certain predefined time interval, which is then used in the above
 definition instead of the difference (stop time) - (start time).
 The Collector MUST stop collecting the Flow Export data at the
 measurement stop time.
 The Idle Timeout (or the time needed to fill the Cache) causes delay
 of the Flow Export data behind the test traffic that is analyzed by
 the DUT.  For example, if the traffic starts at time point X, Flow
 Export will start only at the time point X + Idle Timeout (or X +
 time to fill the Cache).  Since Flow Export capture needs to stop
 with the traffic (because that's when the DUT stops processing the
 Flows at the given rate), the time interval during which the DUT kept
 exporting data is shorter by the Idle Timeout than the time interval
 when the test traffic was sent from the traffic generator to the DUT.

5.5. Flow Export Rate Measurement

 The Flow Export Rate needs to be measured in two consequent steps.
 The purpose of the first step (point a. below) is to gain the actual
 value for the rate; the second step (point b. below) needs to be done
 in order to verify that no Flow Record are dropped during the
 measurement:
 a. In the first step, the captured Flow Export data MUST be analyzed
    only for the capturing interval (measurement time interval) as
    specified in Section 5.4.  During this period, the DUT is forced
    to process Cache entries at the rate the packets are sent.  When
    traffic generation finishes, the behavior when emptying the Cache
    is completely implementation-specific; therefore, the Flow Export
    data from this period cannot be used for benchmarking.
 b. In the second step, all the Flow Export data from the DUT MUST be
    captured in order to determine the Flow Record losses.  It needs
    to be taken into account that especially when large Cache Sizes
    (in order of magnitude of hundreds of thousands of entries and
    higher) are in use, the Flow Export can take many multiples of
    Idle Timeout to empty the Cache after the measurement.  This
    behavior is completely implementation-specific.
 If the Collector has the capability to redirect the Flow Export data
 after the measurement time interval into a different capture buffer
 (or time stamp the received Flow Export data after that), this can be
 done in one step.  Otherwise, each Flow Monitoring Throughput
 measurement at a certain packet rate needs to be executed twice --
 once to capture the Flow Export data just for the measurement time

Novak Informational [Page 26] RFC 6645 Flow Monitoring Benchmarking July 2012

 interval (to determine the actual Flow Export Rate) and a second time
 to capture all Flow Export data in order to determine Flow Record
 losses at that packet rate.
 At the end of the measurement time interval, the DUT might still be
 processing Cache entries that belong to the Flows expired from the
 Cache before the end of the interval.  These Flow Records might
 appear in an export packet sent only after the end of the measurement
 interval.  This imprecision can be mitigated by use of large amounts
 of Flow Records during the measurement (so that the few Flow Records
 in one export packet can be ignored) or by use of timestamps exported
 with the Flow Records.

5.6. The Measurement Procedure

 The measurement procedure is the same as the Throughput measurement
 in Section 26.1 of [RFC2544] for the traffic sending side.  The DUT
 output analysis is done on the traffic generator receiving side for
 the test traffic, the same way as for RFC 2544 measurements.
 An additional analysis is performed using data captured by the
 Collector.  The purpose of this analysis is to establish the value of
 the Flow Export Rate during the current measurement step and to
 verify that no Flow Records were dropped during the measurement.  The
 procedure for measuring the Flow Export Rate is described in Section
 5.5.
 The Flow Export performance can be significantly affected by the way
 the Flow monitoring implementation formats the Flow Records into the
 Flow Export packets.  The ordering and frequency in which Control
 Information is exported and the number of Flow Records in one Flow
 Export packet are of interest.  In the worst case scenario, there is
 just one Flow Record in every Flow Export packet.
 Flow Export data should be sanity checked during the benchmark
 measurement for:
 a. the number of Flow Records per packet, by simply calculating the
    ratio of exported Flow Records to the number of Flow Export
    packets captured during the measurement (which should be available
    as a counter on the Collector capture buffer).
 b. the number of Flow Records corresponding to the export of Control
    Information per Flow Export packet (calculated as the ratio of the
    total number of such Flow Records in the Flow Export data and the
    number of Flow Export packets).

Novak Informational [Page 27] RFC 6645 Flow Monitoring Benchmarking July 2012

6. RFC 2544 Measurements

 RFC 2544 measurements can be performed under two Flow monitoring
 setups (see also Section 3.4.2).  This section details both and
 specifies ways to construct the test traffic so that RFC 2544
 measurements can be performed in a controlled environment from the
 Flow monitoring point of view.  A controlled Flow monitoring
 environment means that the tester always knows what Flow monitoring
 activity (Flow Export Rate) the traffic offered to the DUT causes.
 This section is applicable mainly for the Throughput (RFC 2544,
 Section 26.1) and latency (RFC 2544, Section 26.2 ) measurements.  It
 could also be used to measure frame loss rate (RFC 2544, Section
 26.3) and back-to-back frames (RFC 2544, Section 26.4).  Flow Export
 requires DUT resources to be generated and transmitted; therefore,
 the Throughput in most cases will be much lower when Flow monitoring
 is enabled on the DUT than when it is not.
 Objective:
    Provide RFC 2544 network device characteristics in the presence of
    Flow monitoring on the DUT.  RFC 2544 studies numerous
    characteristics of network devices.  The DUT forwarding and time
    characteristics without Flow monitoring present on the DUT can
    vary significantly when Flow monitoring is deployed on the network
    device.
 Metric definition:
    Metric as specified in [RFC2544].
 The measured Throughput MUST NOT include the packet rate
 corresponding to the Flow Export data, because it is not user traffic
 forwarded by the DUT.  It is generated by the DUT as a result of
 enabling Flow monitoring and does not contribute to the test traffic
 that the DUT can handle.  Flow Export requires DUT resources to be
 generated and transmitted; therefore, the Throughput in most cases
 will be much lower when Flow monitoring is enabled on the DUT than
 when it is not.

6.1. Flow Monitoring Configuration

 Flow monitoring configuration (as detailed in Section 4.3) needs to
 be applied the same way as discussed in Section 5 with the exception
 of the Active Timeout configuration.
 The Active Timeout SHOULD be configured to exceed several times the
 measurement time interval (see Section 5.4).  This ensures that if

Novak Informational [Page 28] RFC 6645 Flow Monitoring Benchmarking July 2012

 measurements with two traffic components are performed (see Section
 6.3.2), there is no Flow monitoring activity related to the second
 traffic component.
 The Flow monitoring configuration does not change in any other way
 for the measurement performed in this section.  What changes and
 makes the difference is the traffic configurations as specified in
 the sections below.

6.2. Measurements with the Flow Monitoring Throughput Setup

 To perform a measurement with Flow Monitoring Throughput setup, the
 major requirement is that the traffic and Flow monitoring be
 configured in such a way that each sent packet creates one entry in
 the DUT Cache.  This restricts the possible setups only to the
 measurement with two traffic components as specified in Section
 6.3.2.

6.3. Measurements with a Fixed Flow Export Rate

 This section covers the measurements where the RFC 2544 metrics need
 to be measured with Flow monitoring enabled, but at a certain Flow
 Export Rate that is lower than the Flow Monitoring Throughput.
 The tester here has both options as specified in Sections 6.3.1 and
 6.3.2.

6.3.1. Measurements with a Single Traffic Component

 Section 12 of [RFC2544] discusses the use of protocol source and
 destination addresses for defined measurements.  To perform all the
 RFC 2544 type measurements with Flow monitoring enabled, the defined
 Flow Keys SHOULD contain an IP source and destination address.  The
 RFC 2544 type measurements with Flow monitoring enabled then can be
 executed under these additional conditions:
 a. the test traffic is not limited to a single, unique pair of source
    and destination addresses.
 b. the traffic generator defines test traffic as follows: it allows
    for a parameter to send N (where N is an integer number starting
    at 1 and is incremented in small steps) packets with source IP
    address A and destination IP address B before changing both IP
    addresses to the next value.
 This test traffic definition allows execution of the Flow monitoring
 measurements with a fixed Flow Export Rate while measuring the DUT
 RFC 2544 characteristics.  This setup is the better option since it

Novak Informational [Page 29] RFC 6645 Flow Monitoring Benchmarking July 2012

 best simulates the live network traffic scenario with Flows
 containing more than just one packet.
 The initial packet rate at N equal to 1 defines the Flow Export Rate
 for the whole measurement procedure.  Subsequent increases of N will
 not change the Flow Export Rate as the time and Cache characteristics
 of the test traffic stay the same.  This setup is suitable for
 measurements with Flow Export Rates below the Flow Monitoring
 Throughput.

6.3.2 Measurements with Two Traffic Components

 The test traffic setup described in Section 6.3.1 might be difficult
 to achieve with commercial traffic generators or if the granularity
 of the traffic rates as defined by the initial packet rate at N equal
 to 1 are unsuitable for the required measurement.  An alternative
 mechanism is to define two traffic components in the test traffic:
 one to populate Flow monitoring Cache and the second to execute the
 RFC 2544 measurements.
 a. Flow monitoring test traffic component -- the exact traffic
    definition as specified in Section 5.2.
 b. RFC 2544 Test Traffic Component -- test traffic as specified by
    RFC 2544 MUST create just one entry in the DUT Cache.  In the
    particular setup discussed here, this would mean a traffic stream
    with just one pair of unique source and destination IP addresses
    (but could be avoided if Flow Keys were, for example, UDP/TCP
    source and destination ports and Flow Keys did not contain the
    addresses).
 The Flow monitoring traffic component will exercise the DUT in terms
 of Flow activity, while the second traffic component will measure the
 RFC 2544 characteristics.
 The measured Throughput is the sum of the packet rates of both
 traffic components.  The definition of other RFC 1242 metrics remains
 unchanged.

7. Flow Monitoring Accuracy

 The pure Flow Monitoring Throughput measurement described in Section
 5 provides the capability to verify the Flow monitoring accuracy in
 terms of the exported Flow Record data.  Since every Cache entry
 created in the Cache is populated by just one packet, the full set of
 captured data on the Collector can be parsed (e.g., providing the
 values of all Flow Keys and other Flow Record fields, not only the
 overall Flow Record count in the exported data), and each set of

Novak Informational [Page 30] RFC 6645 Flow Monitoring Benchmarking July 2012

 parameters from each Flow Record can be checked against the
 parameters as configured on the traffic generator and set in packets
 sent to the DUT.  The exported Flow Record is considered accurate if:
 a. all the Flow Record fields are present in each exported Flow
    Record.
 b. all the Flow Record fields' values match the value ranges set by
    the traffic generator (for example, an IP address falls within the
    range of the IP address increments on the traffic generator).
 c. all the possible Flow Record field values as defined at the
    traffic generator have been found in the captured export data on
    the Collector.  This check needs to be offset against detected
    packet losses at the DUT during the measurement.
 For a DUT with packet forwarding, the Flow monitoring accuracy also
 involves data checks on the received traffic, as already discussed in
 Section 4.

8. Evaluating Flow Monitoring Applicability

 The measurement results, as discussed in this document and obtained
 for certain DUTs, allow for a preliminary analysis of a Flow
 monitoring deployment based on the traffic analysis data from the
 providers' network.  An example of such traffic analysis in the
 Internet is provided by [CAIDA]; the way it can be used is discussed
 below.  The data needed to estimate if a certain network device can
 manage the particular amount of live traffic with Flow monitoring
 enabled is:
 Average packet size:            350 bytes
 Number of packets per IP flow:  20
 Expected data rate on the network device: 1 Gbit/s
 The average number of Flows created per second in the network device
 is needed and is determined as follows:
                     Expected packet rate
 Flows per second =  --------------------
                     Packet per flow
 When using the above example values, the network device is required
 to process 18000 Flows per second.  By executing the benchmarking as
 specified in this document, a platform capable of this processing can
 be determined for the deployment in that particular part of the user
 network.

Novak Informational [Page 31] RFC 6645 Flow Monitoring Benchmarking July 2012

 Keep in mind that the above is a very rough and averaged Flow
 activity estimate, which cannot account for traffic anomalies; for
 example, a large number of DNS request packets that are typically
 small packets coming from many different sources and represent mostly
 just one packet per Flow.

9. Acknowledgements

 This work was performed thanks to the patience and support of Cisco
 Systems NetFlow development team, namely Paul Aitken, Paul Atkins,
 and Andrew Johnson.  Thanks to Benoit Claise for numerous detailed
 reviews and presentations of the document, and to Aamer Akhter for
 initiating this work.  A special acknowledgment to the entire BMWG
 working group, especially to the chair, Al Morton, for the support
 and work on this document and Paul Aitken for a very detailed
 technical review.

10. Security Considerations

 Documents of this type do not directly affect the security of the
 Internet or corporate networks as long as benchmarking is not
 performed on devices or systems connected to operating networks.
 Benchmarking activities, as described in this memo, are limited to
 technology characterization using controlled stimuli in a laboratory
 environment, with dedicated address space and the constraints
 specified in sections above.
 The benchmarking network topology will be an independent test setup
 and MUST NOT be connected to devices that may forward the test
 traffic into a production network, or misroute traffic to the test
 management network.
 Further, benchmarking is performed on a "black-box" basis, relying
 solely on measurements observable external to the DUT.
 Special capabilities SHOULD NOT exist in the DUT specifically for
 benchmarking purposes.  Any implications for network security arising
 from the DUT SHOULD be identical in the lab and in production
 networks.

Novak Informational [Page 32] RFC 6645 Flow Monitoring Benchmarking July 2012

11. References

11.1. Normative References

 [RFC2119]   Bradner, S., "Key words for use in RFCs to Indicate
             Requirement Levels", BCP 14, RFC 2119, March 1997.
 [RFC2544]   Bradner, S. and J. McQuaid, "Benchmarking Methodology for
             Network Interconnect Devices", RFC 2544, March 1999.

11.2. Informative References

 [RFC1242]   Bradner, S., "Benchmarking Terminology for Network
             Interconnection Devices", RFC 1242, July 1991.
 [RFC2285]   Mandeville, R., "Benchmarking Terminology for LAN
             Switching Devices", RFC 2285, February 1998.
 [RFC3031]   Rosen, E., Viswanathan, A., and R. Callon, "Multiprotocol
             Label Switching Architecture", RFC 3031, January 2001.
 [RFC3917]   Quittek, J., Zseby, T., Claise, B., and S. Zander,
             "Requirements for IP Flow Information Export (IPFIX)",
             RFC 3917, October 2004.
 [RFC3954]   Claise, B., Ed., "Cisco Systems NetFlow Services Export
             Version 9", RFC 3954, October 2004.
 [RFC5101]   Claise, B., Ed., "Specification of the IP Flow
             Information Export (IPFIX) Protocol for the Exchange of
             IP Traffic Flow Information", RFC 5101, January 2008.
 [RFC5180]   Popoviciu, C., Hamza, A., Van de Velde, G., and D.
             Dugatkin, "IPv6 Benchmarking Methodology for Network
             Interconnect Devices", RFC 5180, May 2008.
 [RFC5470]   Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek,
             "Architecture for IP Flow Information Export", RFC 5470,
             March 2009.
 [RFC5695]   Akhter, A., Asati, R., and C. Pignataro, "MPLS Forwarding
             Benchmarking Methodology for IP Flows", RFC 5695,
             November 2009.
 [CAIDA]     Claffy, K., "The nature of the beast: recent traffic
             measurements from an Internet backbone",
             http://www.caida.org/publications/papers/1998/
             Inet98/Inet98.html

Novak Informational [Page 33] RFC 6645 Flow Monitoring Benchmarking July 2012

 [IPFIX-CONFIG]
             Muenz, G., Muenchen, TU, Claise, B., and P. Aitken,
             "Configuration Data Model for IPFIX and PSAMP", Work in
             Progress, July 2011.
 [PSAMP-MIB] Dietz, T., Claise, B., and J. Quittek, "Definitions of
             Managed Objects for Packet Sampling", Work in Progress,
             October 2011.
 [IPFIX-MIB] Dietz, T., Kobayashi, A., Claise, B., and G. Muenz,
             "Definitions of Managed Objects for IP Flow Information
             Export", Work in Progress, March 2012.

Novak Informational [Page 34] RFC 6645 Flow Monitoring Benchmarking July 2012

Appendix A. (Informative) Recommended Report Format

Parameter Units ———————————– ———————————— Test Case test case name (Sections 5 and 6) Test Topology Figure 2, other Traffic Type IPv4, IPv6, MPLS, other

Test Results

Flow Monitoring Throughput        Flow Records per second or Not
                                  Applicable
Flow Export Rate                  Flow Records per second or Not
                                  Applicable
Control Information Export Rate   Flow Records per second
Throughput                        packets per second
(Other RFC 1242 Metrics)          (as appropriate)

General Parameters

DUT Interface Type                Ethernet, POS, ATM, other
DUT Interface Bandwidth           MegaBits per second

Traffic Specifications

Number of Traffic Components      (see Sections 6.3.1 and 6.3.2)
For each traffic component:
Packet Size                       bytes
Traffic Packet Rate               packets per second
Traffic Bit Rate                  MegaBits per second
Number of Packets Sent            number of entries
Incremented Packet Header Fields  list of fields
Number of Unique Header Values    number of entries
Number of Packets per Flow        number of entries
Traffic Generation                linearly incremented or
                                  randomized

Flow monitoring Specifications

Direction                         ingress, egress, both
Observation Points                DUT interface names
Cache Size                        number of entries
Active Timeout                    seconds
Idle Timeout                      seconds
Flow Keys                         list of fields
Flow Record Fields                total number of fields
Number of Flows Created           number of entries
Flow Export Transport Protocol    UDP, TCP, SCTP, other
Flow Export Protocol              IPFIX, NetFlow, other
Flow Export data packet size      bytes
Flow Export MTU                   bytes

Novak Informational [Page 35] RFC 6645 Flow Monitoring Benchmarking July 2012

Parameter Units (continued) ———————————– ———————————— MPLS Specifications (for traffic type MPLS only)

Tested Label Operation            imposition, swap, disposition

The format of the report as documented in this appendix is informative, but the entries in the contents of it are required as specified in the corresponding sections of this document.

Many of the configuration parameters required by the measurement report can be retrieved from the [IPFIX-MIB] and [PSAMP-MIB] MIB modules, and from the [IPFIX-CONFIG] YANG module or other general MIBs. Therefore, querying those modules from the DUT would be beneficial: first of all, to help in populating the required entries of the measurement report, and also to document all the other configuration parameters from the DUT.

Appendix B. (Informative) Miscellaneous Tests

 This section lists tests that could be useful to asses a proper Flow
 monitoring operation under various operational or stress conditions.
 These tests are not deemed suitable for any benchmarking for various
 reasons.

B.1. DUT Under Traffic Load

 The Flow Monitoring Throughput should be measured under different
 levels of static traffic load through the DUT.  This can be achieved
 only by using two traffic components as discussed in Section 6.3.2.
 One traffic component exercises the Flow Monitoring Plane.  The
 second traffic component loads only the Forwarding Plane without
 affecting Flow monitoring (i.e., it creates just a certain amount of
 permanent Cache entries).
 The variance in Flow Monitoring Throughput as a function of the
 traffic load should be noted for comparison purposes between two DUTs
 of similar architecture and capability.

B.2. In-Band Flow Export

 The test topology in Section 4.1 mandates the use of a separate Flow
 Export interface to avoid the Flow Export data generated by the DUT
 to mix with the test traffic from the traffic generator.  This is
 necessary in order to create clear and reproducible test conditions
 for the benchmark measurement.
 The real network deployment of Flow monitoring might not allow for
 such a luxury -- for example, on a very geographically large network.

Novak Informational [Page 36] RFC 6645 Flow Monitoring Benchmarking July 2012

 In such a case, the Flow Export will use an ordinary traffic
 forwarding interface, e.g., in-band Flow Export.
 The Flow monitoring operation should be verified with in-band Flow
 Export configuration while following these test steps:
 a. Perform the benchmark test as specified in Section 5.  One of the
    results will be how much bandwidth Flow Export used on the
    dedicated Flow Export interface.
 b. Change Flow Export configuration to use the test interface.
 c. Repeat the benchmark test while the receiver filters out the Flow
    Export data from analysis.
 The expected result is that the Throughput achieved in step a. is
 same as the Throughput achieved in step c. provided that the
 bandwidth of the output DUT interface is not the bottleneck (in other
 words, it must have enough capacity to forward both test and Flow
 Export traffic).

B.3. Variable Packet Size

 The Flow monitoring measurements specified in this document would be
 interesting to repeat with variable packet sizes within one
 particular test (e.g., test traffic containing mixed packet sizes).
 The packet forwarding tests specified mainly in [RFC2544] do not
 recommend performing such tests.  Flow monitoring is not dependent on
 packet sizes, so such a test could be performed during the Flow
 Monitoring Throughput measurement, and verification of its value does
 not depend on the offered traffic packet sizes.  The tests must be
 carefully designed in order to avoid measurement errors due to the
 physical bandwidth limitations and changes of the base forwarding
 performance with packet size.

B.4. Bursty Traffic

 RFC 2544, Section 21 discusses and defines the use of bursty traffic.
 It can be used for Flow monitoring testing to gauge some short-term
 overload DUT capabilities in terms of Flow monitoring.  The test
 benchmark here would not be the Flow Export Rate the DUT can sustain,
 but the absolute number of Flow Records the DUT can process without
 dropping any single Flow Record.  The traffic setup to be used for
 this test is as follows:
 a. each sent packet creates a new Cache entry.
 b. the packet rate is set to the maximum transmission speed of the
    DUT interface used for the test.

Novak Informational [Page 37] RFC 6645 Flow Monitoring Benchmarking July 2012

B.5. Various Flow Monitoring Configurations

 This section translates the terminology used in the IPFIX documents
 ([RFC5470], [RFC5101], and others) into the terminology used in this
 document.  Section B.5.2 proposes another measurement that is
 impossible to verify in a black box test manner.

B.5.1. Throughput without the Metering Process

 If the Metering Process is not defined on the DUT it means no Flow
 monitoring Cache exists and no Flow analysis occurs.  The performance
 measurement of the DUT in such a case is just pure [RFC2544]
 measurement.

B.5.2. Throughput with the Metering Process

 If only the Metering Process is enabled, Flow analysis on the DUT is
 enabled and operational but no Flow Export happens.  The performance
 measurement of a DUT in such a configuration represents a useful test
 of the DUT's capabilities (this corresponds to the case when the
 network operator uses Flow monitoring, for example, for manual
 detection of denial-of-service attacks, and does not wish to use Flow
 Export).
 The performance testing on this DUT can be performed as discussed in
 this document, but it is not possible to verify the operation and
 results without interrogating the DUT.

B.5.3. Throughput with the Metering and Exporting Processes

 This test represents the performance testing as discussed in Section
 6.

B.6. Tests With Bidirectional Traffic

 Bidirectional traffic is not part of the normative benchmarking tests
 based on discussion with and recommendation of the Benchmarking
 working group.  The experienced participants stated that this kind of
 traffic did not provide reproducible results.
 The test topology in Figure 2 can be expanded to verify Flow
 monitoring functionality with bidirectional traffic using the
 interfaces in full duplex mode, e.g., sending and receiving
 simultaneously on each of them.
 The same rules should be applied for Flow creation in the DUT Cache
 (as per Sections 4.1 and 4.3.1) -- traffic passing through each
 Observation Point should always create a new Cache entry in the

Novak Informational [Page 38] RFC 6645 Flow Monitoring Benchmarking July 2012

 Cache, e.g., the same traffic should not be just looped back on the
 receiving interfaces to create the bidirectional traffic flow.

B.7. Instantaneous Flow Export Rate

 Additional useful information when analyzing the Flow Export data is
 the time distribution of the instantaneous Flow Export Rate.  It can
 be derived during the measurements in two ways:
 a. The Collector might provide the capability to decode Flow Export
    during capturing and at the same time count the Flow Records and
    provide the instantaneous (or simply, an average over shorter time
    interval than specified in Section 5.4) Flow Export Rate.
 b. The Flow Export protocol (like IPFIX [RFC5101]) can provide time
    stamps in the Flow Export packets that would allow time-based
    analysis and calculate the Flow Export Rate as an average over
    much shorter time interval than specified in Section 5.4.
 The accuracy and shortest time average will always be limited by the
 precision of the time stamps (1 second for IPFIX) or by the
 capabilities of the DUT and the Collector.

Author's Address

 Jan Novak (editor)
 Cisco Systems
 Edinburgh
 United Kingdom
 EMail: janovak@cisco.com

Novak Informational [Page 39]

/data/webs/external/dokuwiki/data/pages/rfc/rfc6645.txt · Last modified: 2012/07/10 17:59 by 127.0.0.1

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki