GENWiki

Network Working Group V. Manral Request for Comments: 4062 SiNett Corp. Category: Informational R. White

                                                         Cisco Systems
                                                             A. Shaikh
                                                  AT&T Labs (Research)
                                                            April 2005

             OSPF Benchmarking Terminology and Concepts

Status of This Memo

 This memo provides information for the Internet community.  It does
 not specify an Internet standard of any kind.  Distribution of this
 memo is unlimited.

Copyright Notice

 Copyright (C) The Internet Society (2005).

Abstract

 This document explains the terminology and concepts used in OSPF
 benchmarking.  Although some of these terms may be defined elsewhere
 (and we will refer the reader to those definitions in some cases) we
 include discussions concerning these terms, as they relate
 specifically to the tasks involved in benchmarking the OSPF protocol.

1. Introduction

 This document is a companion to [BENCHMARK], which describes basic
 Open Shortest Path First [OSPF] testing methods.  This document
 explains terminology and concepts used in OSPF Testing Framework
 Documents, such as [BENCHMARK].

2. Specification of Requirements

 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
 document are to be interpreted as described in RFC 2119 [RFC2119].
 [RFC2119] key words in this document are used to ensure
 methodological control, which is very important in the specification
 of benchmarks.  This document does not specify a network-related
 protocol.

Manral, et al. Informational [Page 1] RFC 4062 OSPF Benchmarking Terminology April 2005

3. Common Definitions

 Definitions in this section are well-known industry and benchmarking
 terms that may be defined elsewhere.

 o    White Box (Internal) Measurements

Definition

           White box measurements are those reported and collected on
           the Device Under Test (DUT) itself.

Discussion

           These measurements rely on output and event recording,
           along with the clocking and time stamping available on the
           DUT itself.  Taking measurements on the DUT may impact the
           actual outcome of the test, since it can increase processor
           loading, memory utilization, and timing factors.  Some
           devices may not have the required output readily available
           for taking internal measurements.

           Note: White box measurements can be influenced by the
           vendor's implementation of various timers and processing
           models.  Whenever possible, internal measurements should be
           compared to external measurements to verify and validate
           them.

           Because of the potential for variations in collection and
           presentation methods across different DUTs, white box
           measurements MUST NOT be used as a basis for comparison in
           benchmarks.  This has been a guiding principle of the
           Benchmarking Methodology Working Group.

 o    Black Box (External) Measurements

Definition

           Black box measurements infer the performance of the DUT
           through observation of its communications with other
           devices.

Discussion

           One example of a black box measurement is when a downstream
           device receives complete routing information from the DUT,
           it can be inferred that the DUT has transmitted all the
           routing information available.  External measurements of

Manral, et al. Informational [Page 2] RFC 4062 OSPF Benchmarking Terminology April 2005

           internal operations may suffer in that they include not
           just the protocol action times, but also propagation
           delays, queuing delays, and other such factors.

           For the purposes of [BENCHMARK], external techniques are
           more readily applicable.

 o    Multi-device Measurements

Measurements assessing communications (usually in

combination with internal operations) between two or more

           DUTs.  Multi-device measurements may be internal or
           external.

4. Terms Defined Elsewhere

 Terms in this section are defined elsewhere and are included only as
 they apply to [BENCHMARK].

 o    Point-to-Point Links

Definition

           See [OSPF], Section 1.2.

Discussion

           A point-to-point link can take less time to converge than a
           broadcast link of the same speed because it does not have
           the overhead of DR election.  Point-to-point links can be
           either numbered or unnumbered.  However, in the context of
           [BENCHMARK] and [OSPF], the two can be regarded as the
           same.

 o    Broadcast Link

Definition

           See [OSPF], Section 1.2.

Discussion

           The adjacency formation time on a broadcast link can be
           greater than that on a point-to-point link of the same
           speed because DR election has to take place.  All routers
           on a broadcast network form adjacency with the DR and BDR.

Manral, et al. Informational [Page 3] RFC 4062 OSPF Benchmarking Terminology April 2005

           Asynchronous flooding also takes place through the DR.  In
           the context of convergence, it may take more time for an
           LSA to be flooded from one DR-other router to another
           because the LSA first has to be processed at the DR.

 o    Shortest Path First Execution Time

Definition

           The time taken by a router to complete the SPF process, as
           described in [OSPF].

Discussion

           This does not include the time taken by the router to
           install routes in the forwarding engine.

           Some implementations may force two intervals, the SPF hold
           time and the SPF delay, between successive SPF
           calculations.  If an SPF hold time exists, it should be
           subtracted from the total SPF execution time.  If an SPF
           delay exists, it should be noted in the test results.

Measurement Units

           The SPF time is generally measured in milliseconds.

 o Hello Interval

Definition

           See [OSPF], Section 7.1.

Discussion

           The hello interval must be the same for all routers on a
           network.

           Decreasing the hello interval can allow the router dead
           interval (below) to be reduced, thus reducing convergence
           times in those situations where the router dead interval's
           timing out causes an OSPF process to notice an adjacency
           failure.  Further discussion of small hello intervals is
           given in [OSPF-SCALING].

Manral, et al. Informational [Page 4] RFC 4062 OSPF Benchmarking Terminology April 2005

 o    Router Dead Interval

Definition

           See [OSPF], Section 7.1.

Discussion

           This is advertised in the router's Hello Packets in the
           Router-DeadInterval field.  The router dead interval should
           be some multiple of the HelloInterval (perhaps 4 times the
           hello interval) and must be the same for all routers
           attached to a common network.

5. Concepts

5.1. The Meaning of Single Router Control Plane Convergence

 A network is termed as converged when all the devices within the
 network have a loop-free path to each possible destination.  However,
 because we are not testing network convergence but testing
 performance for a particular device within a network, this definition
 needs to be streamlined to fit within a single device view.

 In this case, convergence will mean the point in time when the DUT
 has performed all actions needed in order to react to the change in
 the topology represented by the test condition.  For instance, an
 OSPF device must flood any new information it has received, rebuild
 its shortest path first (SPF) tree, and install any new paths or
 destinations in the local routing information base (RIB, or routing
 table).

 Note that the word "convergence" has two distinct meanings: the
 process of a group of individuals meeting at the same place, and the
 process of an individual coming to the same place as an existing
 group.  This work focuses on the second meaning of the word, so we
 consider the time required for a single device to adapt to a network
 change to be Single Router Convergence.

 This concept does not include the time required for the control plane
 of the device to transfer the information required to forward packets
 to the data plane.  It also does not include the amount of time
 between when the data plane receives that information and when it is
 able to forward traffic.

Manral, et al. Informational [Page 5] RFC 4062 OSPF Benchmarking Terminology April 2005

5.2. Measuring Convergence

 Obviously, there are several elements to convergence, even under the
 definition given above for a single device, including (but not
 limited to) the following:

 o    The time it takes for the DUT to pass the information about a
      network event on to its neighbors.

 o    The time it takes for the DUT to process information about a
      network event and to calculate a new Shortest Path Tree (SPT).

 o    The time it takes for the DUT to make changes in its local RIB
      reflecting the new shortest path tree.

5.3. Types of Network Events

 A network event is an event that causes a change in the network
 topology.

 o    Link or Neighbor Device Up

      The time needed for an OSPF implementation to recognize a new
      link coming up on the device, to build any necessary
      adjacencies, to synchronize its database, and to perform all
      other actions necessary to converge.

 o    Initialization

      The time needed for an OSPF implementation to be initialized, to
      recognize any links across which OSPF must run, to build any
      needed adjacencies, to synchronize its database, and to perform
      other actions necessary to converge.

 o    Adjacency Down

      The time needed for an OSPF implementation to recognize a link
      down/adjacency loss based on hello timers alone, to propagate
      any information as necessary to its remaining adjacencies, and
      to perform other actions necessary to converge.

 o    Link Down

      The time needed for an OSPF implementation to recognize a link
      down based on layer 2-provided information, to propagate any
      information as needed to its remaining adjacencies, and to
      perform other actions necessary to converge.

Manral, et al. Informational [Page 6] RFC 4062 OSPF Benchmarking Terminology April 2005

6. Security Considerations

 This document does not modify the underlying security considerations
 in [OSPF].

7. Acknowledgements

 The authors would like to thank Howard Berkowitz (hcb@clark.net),
 Kevin Dubray (kdubray@juniper.net), Scott Poretsky
 (sporetsky@avici.com), and Randy Bush (randy@psg.com) for their
 discussion, ideas, and support.

8. Normative References

 [BENCHMARK]    Manral, V., White, R., and A. Shaikh, "Benchmarking
                Basic OSPF Single Router Control Plane Convergence",
                RFC 4061, April 2005.

 [OSPF]         Moy, J., "OSPF Version 2", STD 54, RFC 2328, April
                1998.

 [RFC2119]      Bradner, S., "Key words for use in RFCs to Indicate
                Requirement Levels", BCP 14, RFC 2119, March 1997.

9. Informative References

 [OSPF-SCALING] Choudhury, Gagan L., Editor, "Prioritized Treatment of
                Specific OSPF Packets and Congestion Avoidance", Work
                in Progress, August 2003.

Manral, et al. Informational [Page 7] RFC 4062 OSPF Benchmarking Terminology April 2005

Authors' Addresses

 Vishwas Manral,
 SiNett Corp,
 Ground Floor,
 Embassy Icon Annexe,
 2/1, Infantry Road,
 Bangalore, India

 EMail: vishwas@sinett.com

 Russ White
 Cisco Systems, Inc.
 7025 Kit Creek Rd.
 Research Triangle Park, NC 27709

 EMail: riw@cisco.com

 Aman Shaikh
 AT&T Labs (Research)
 180 Park Av, PO Box 971
 Florham Park, NJ 07932

 EMail: ashaikh@research.att.com

Manral, et al. Informational [Page 8] RFC 4062 OSPF Benchmarking Terminology April 2005

Full Copyright Statement

 Copyright (C) The Internet Society (2005).

 This document is subject to the rights, licenses and restrictions
 contained in BCP 78, and except as set forth therein, the authors
 retain all their rights.

 This document and the information contained herein are provided on an
 "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
 OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
 ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
 INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
 INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Intellectual Property

 The IETF takes no position regarding the validity or scope of any
 Intellectual Property Rights or other rights that might be claimed to
 pertain to the implementation or use of the technology described in
 this document or the extent to which any license under such rights
 might or might not be available; nor does it represent that it has
 made any independent effort to identify any such rights.  Information
 on the procedures with respect to rights in RFC documents can be
 found in BCP 78 and BCP 79.

 Copies of IPR disclosures made to the IETF Secretariat and any
 assurances of licenses to be made available, or the result of an
 attempt made to obtain a general license or permission for the use of
 such proprietary rights by implementers or users of this
 specification can be obtained from the IETF on-line IPR repository at
 http://www.ietf.org/ipr.

 The IETF invites any interested party to bring to its attention any
 copyrights, patents or patent applications, or other proprietary
 rights that may cover technology that may be required to implement
 this standard.  Please address the information to the IETF at ietf-
 ipr@ietf.org.

Acknowledgement

 Funding for the RFC Editor function is currently provided by the
 Internet Society.

Manral, et al. Informational [Page 9]