GENWiki

Premier IT Outsourcing and Support Services within the UK

User Tools

Site Tools


rfc:ien:ien12
                                             LLG 8-Jun-77 13:01  29364
   

IEN # 12 L. Garlick / SRI-ARC Supercedes: None R. Rom / SRI-ARC Replaces: None J. Postel /SRI-ARC

                                                         15 March 1977

Section: 2.4.4.1

             Issues in Reliable Host-to-Host Protocols
                        Lawrence L. Garlick
                            Raphael Rom
                         Jonathan B. Postel
                           March 15, 1977
                    Augmentation Research Center
                    Stanford Research Institute
                   Menlo Park, California  94025
                                  
                           (415) 326-6200
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
                              ABSTRACT
     Fully reliable network host-to-host protocols have recently
     gained significant attention, primarily due to more strin-
     gent security requirements of network users.  This paper
     will discuss issues related to one such protocol, which is
     supported by the Transmission Control Program (TCP).  The
     protocol, first introduced in 1974, features end-to-end pos-
     itive acknowledgement, retransmission, internetwork
     addressing capabilities, and ordered delivery.
     The issues of interest in this paper are protocol correct-
     ness and completeness, protocol efficiency, and complexity
     of implementation.  The discussion will suggest alterations
     and extensions to TCP.
     Flow control heuristics using TCP's windowing techniques are
     explored.  Flow control information is augmented to allow
     fair apportionment of bandwidth, better bandwidth utiliza-
     tion through optimistic credits, flow control credits
     matched to the type of traffic, and increased performance
     for high precedence connections.
     An alternative for selecting the startup sequence number of
     a connection is presented.  It is suggested that the
     resynchronization method for sequence number space manage-
     ment should be abandoned because it is overly complicated
     and can actually fail when the data stream is stopped by
     flow control.
     The need for the separation of data and control channels is
     motivated, introducing the notion of a reliable subchannel.
     The findings are presented both to further the understanding
     of reliable protocols and to encourage intelligent
     implementations of TCP.
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
             Issues in Reliable Host-to-Host Protocols
                                  
                                                                     2
   INTRODUCTION                                                      3
     Due to numerous advances in computer communications, there
     has been a tremendous growth in computer networking.  This
     has led to the need for parallel advances in distributed
     computing protocols.  Typical of these advances are the
     packet switching network protocols developed for the ARPA
     network.  The need for a protocol that supports distributed
     process-to-process communication was realized early by ARPA
     network designers and the ARPA host-to-host protocol (AHHP)
     became the reference point for such process-to-process
     protocols.                                                     3a
     The AHHP has been very successful in providing a basis for
     abundant research in distributed computing and in providing
     a prototype for process-to-process protocols.  As experience
     with networking has grown, new applications, new topologies,
     new network access methods, and new higher level protocols
     have emerged.  The AHHP has not been entirely suited for the
     new requirements that have resulted from this experience.      3b
     End-to-end reliability is an example of a new requirement
     needed by host-to-host protocols.  It has been a concern for
     builders of both secure applications and higher level
     protocols.  There are two important motivations for strin-
     gent reliability requirements.  First, security measures,
     such as encryption, are often applied at the host-to-host
     level or lower.  Second, higher level protocols, such as the
     ARPA TELNET protocol, should not be required to handle
     transmission error checking.  The AHHP does not provide
     host-to-host acknowledgement; it relies upon subnet and
     host-to-subnet protocols to deliver messages reliably.
     While the performance of the AHHP has been almost error
     free, it has been known to lose messages; thus it cannot be
     considered a fully reliable protocol.                          3c
     Other deficiencies in AHHP include addressing constraints,
     weak error recovery, simplex connections, and large overhead
     for passing flow control information.                          3d
     TCP, which, throughout this paper will be an abbreviation
     for both the Transmission Control Program and the protocol
     it supports, corrects the deficiencies of AHHP.  TCP was
   Garlick, Rom, & Postel                                       page 1
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
     initially designed to be a reliable internetwork host-to-
     host protocol [Reference 1], as well as a solution to many
     of the problems of the AHHP.  When the special internetwork
     addressing considerations are ignored (as they shall be in
     this paper), it represents a significant advancement in
     host-to-host protocols.  Among its reliability features are
     positive acknowledgement, retransmission, and sequencing of
     data and controls.  It guarantees the error free delivery of
     each message for which it claims successful delivery.  Other
     improvements include duplex connections and the ability to
     use a network address (socket) in several connections.         3e
     The paper is organized around three issues--a discussion of
     flow control techniques for TCP, alternate strategies for
     the management of connection sequence number space, and the
     need for a control subchannel for each TCP connection.  To
     provide further context for the discussion, a brief summary
     of interesting TCP features is presented.  It is assumed
     that the reader is somewhat familiar with the AHHP and has
     been exposed to the early literature on TCP-like protocols
     [References 1, 2, 6].  A glossary of abbreviations and
     terms, and appendices that magnify a few of the more in-
     volved issues can be found at the end of the paper.            3f
   TCP:  A RELIABLE TRANSMISSION PROTOCOL                            4
     Network Characteristics                                        4a
       TCP does not depend on the transmission medium for its re-
       liability, i.e., it is assumed that the subnetwork may be
       unreliable.  The subnet need not ensure the orderly or
       errorless delivery of subnet packets, or account for lost
       packets.  TCP functions correctly in the face of large
       packet lifetimes, and the opening and closing of
       connections in quick succession.
     Connections                                                    4b
       Logical connections are established for process-to-process
       (user-to-user) communication.  TCP connections are full-
       duplex channels established between source and destination
       sockets (network-wide process names).  A socket may be a
       party to more than one connection, but only one connection
       can exist between any pair of sockets.
   Garlick, Rom, & Postel                                       page 2
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
       TCP provides the means by which a connection between the
       processes is established, controlled during the transfer
       of data, and terminated at the completion of the session.
       Connection management requires the exchange of controls
       between TCP's.  There are controls for connection
       synchronization, out-of-band signalling (interrupt), data
       flushing, resynchronization, and connection closing.  As
       described below, controls accompany data whenever possible
       to avoid the overhead of separate control packets.
     Packaging and Headers                                          4c
       TCP packages user letters (messages) into packets suitable
       for transmission over a subnetwork.  Each letter or par-
       tial letter is prefixed by a TCP header, which includes
       fields for addressing, sequencing, acknowledgements, flow
       control, controls, and error checking.  The header is
       optionally followed by a block of data.  The smallest unit
       of data transfer and the unit of sequencing is the 8-bit
       byte (octet).
     Sequencing                                                     4d
       Sequence numbers are used as acknowledgement identifiers
       and as an ordering mechanism.  They are assigned to each
       octet of data and to those controls that need
       synchronization with the data stream.  Only one sequence
       number is sent with each TCP header; it represents the se-
       quence number assigned to the first control or data in the
       packet.  This means that data and control sequence numbers
       come from the same name space.  The packet length is used
       to determine the highest sequence number consumed by the
       packet.
       Reuse of sequence numbers is allowed only for duplicate
       retransmissions.  The sequence number space is managed by
       a cooperatively by the sender and the receiver, as will be
       discussed later.
     Acknowledgement and Retransmission                             4e
       A TCP acknowledgement represents the successful delivery
       of some number of octets to the receiving process's buffer
       or to the remote TCP (controls).  It is sent to the
       transmitting TCP in the acknowledgement field of a subse-
       quent TCP header.  The sequence number placed in this
       field is the highest sequence number acknowledged by the
   Garlick, Rom, & Postel                                       page 3
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
       receiver and implies acknowledgement of all previous
       octets.  If packets arrive out of order, an
       acknowledgement cannot be sent for octets with sequence
       numbers higher than the missing octets, since that would
       implicitly acknowledge the missing data.
       Packets can be retransmitted at will until they are
       acknowledged; however, bandwidth may be underutilized if
       improper retransmission policies are followed.  Duplicates
       naturally arise from retransmissions that occur prior to
       the receipt of an acknowledgment and are detected and han-
       dled as described below.
     Synchronization and Resynchronization                          4f
       TCP is expected to run in a network with relatively long
       packet lifetimes and relatively short times between the
       closing and opening of a connection.  Therefore, several
       problems must be solved concerning detection of old dupli-
       cate packets, that is, packets that have sequence numbers
       assigned by old instances of a connection between the same
       sockets.  These problems are how to select startup se-
       quence numbers, how to reliably exchange new sequence num-
       bers, and how to determine when resynchronization of se-
       quence numbers is necessary.
       The exchange of sequence numbers at synchronization or
       resynchronization time is accomplished using a "three-way
       handshake" method [References 2, 4, 5].  This method pro-
       vides positive acknowledgement of the exchanged sequence
       numbers and is sufficient to handle the problem of
       simultaneous connection establishment attempts.
       A solution to the other two problems has been an Initial
       Sequence Number curve [References 4, 5, 6], that is used
       by the sender as a mechanism for 1) selecting the first
       sequence number for a connection and 2) detecting when the
       consumption of sequence numbers is not progressing in a
       manner that will guarantee that old duplicates can be
       reliably identified by the receiving TCP.
       The management of the sequence number space will be dis-
       cussed in section 4.
   Garlick, Rom, & Postel                                       page 4
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
     Flow Control                                                   4g
       Flow control is exerted by the receiver by issuing
       credits, which represent the receiving process's
       willingness to buffer data.  Credits are passed in the TCP
       header in the window size field.  The window size is added
       to the last acknowledged sequence number (the window's
       left edge) to give the highest allowable sequence number
       that may be sent (the window's right edge).  Flow control
       is discussed in further detail in section 3.
     Packet Acceptance Checking                                     4h
       The receiving TCP is responsible for the detection of
       packets with improper sequence numbers.  These may have
       sequence numbers that are either old duplicates (from pre-
       vious connections) or illegal because they are not within
       an acceptable flow control range.
       To determine the action to be taken for a newly received
       packet, acceptability ranges are defined.  The following
       three ranges are mutually exclusive and collectively
       exhaustive of the sequence number space (see Figure 1):
         Acknowledge-deliver range (ADR)
           The packet has arrived in-order and does not exceed
           the receiving process's buffer space.  Data will be
           placed in the buffer and an acknowledgement will be
           generated to indicate successful delivery.
         Acknowledge-only range (AOR)
           A duplicate packet has arrived, as a result of
           retransmission.  It will be acknowledged, but not de-
           livered, since delivery has already occurred.
         Discard range (DR)
           An illegal packet has arrived.  It may be an old du-
           plicate or a packet that cannot be delivered due to
           flow control.
       Appendix A provides more details of the packet acceptance
       policy.
   Garlick, Rom, & Postel                                       page 5
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
   Garlick, Rom, & Postel                                       page 6
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
   FLOW CONTROL TECHNIQUES                                           5
     Flow control is basically a mechanism to prevent the re-
     ceiving process's buffers from overflowing.  A good flow
     control scheme must handle a whole spectrum of problems that
     result from performing this basic duty.  This section first
     discusses general flow control goals and methods, and then
     specific techniques for use with TCP that could significa-
     ntly improve protocol performance.  Where suggestions occur,
     they represent an enhancement to the flow control scheme
     used in the initial versions of TCP.                           5a
     The goals of an ambitious flow control scheme include the
     following:                                                     5b
       Receiver's Allocation
         Any flow control strategy should consider the buffer
         space offered by a receiving user, since this represents
         a depository for incoming messages and relieves the TCP
         of resource allocation problems.
       Congestion Prevention
         The flow control strategy should prevent queueing of
         messages in the protocol module (TCP), so that TCP re-
         sources can be used to handle those messages that have a
         high probability of being delivered immediately.
         Congestion in the subnet can be caused by a
         retransmission protocol like TCP, since each
         unacknowledged packet is retransmitted.  The flow con-
         trol scheme should make it easy to slow or stop
         retransmission from the sender.
       Deadlock Prevention
         When congestion does occur, resources must be available
         to handle traffic-clearing messages.  Controls and flow
         control information must be delivered and interpreted
         even when data is queued.
       Fair Apportionment Of Bandwidth
         In a virtual connection environment, it is important to
         be able to fairly allocate the available bandwidth to
   Garlick, Rom, & Postel                                       page 7
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
         users, based on a variety of criteria.  One criterion
         may be precedence of the user or the connection.  Anoth-
         er may be the mode of traffic, e.g., interactive traffic
         may get preference over bulk traffic.
       Bandwidth Utilization For Various Modes Of Transmission
         A network will usually serve several types of user
         communities and thus should be able to adapt the flow
         control strategy to the needs of the user.  For example,
         transmission patterns for interactive users and bulk
         transfer users are quite different.  Those differences
         should be reflected in the flow control strategies.
       Interplay With Subnet Flow Control
         Often the interfaces between modules representing levels
         of protocol can cause flow control problems [Reference
         8].  For instance, the subnet flow control of the
         ARPANET is adversely affected whenever a host does not
         readily accept incoming data from the packet switch
         (IMP).  TCP is especially flexible in this regard, be-
         cause it can absorb congested traffic from the subnet
         and discard it if necessary.
     Exchanging Flow Control Information                            5c
       A windowing scheme to convey flow control information has
       been used for many different types of protocols.  It is an
       efficient technique that is useful whenever positive
       acknowledgement and retransmission are used for reliable
       transmission.  Flow control information is passed in the
       header of a packet as a window size.  It is used in con-
       junction with the acknowledgement sequence number (the
       window's left edge) to determine the highest sequence num-
       ber that can be transmitted with some assurance that it
       will be acknowledged without retransmission.  The
       acknowledge sequence number plus the window size gives the
       right edge of the flow control window.
       A nonzero window size gives permission to send a message
       of a certain length.  It is an "oversend" to send messages
       with sequence numbers that exceed the window right edge.
       In TCP, oversends will occur occasionally, since the flow
       control information is always slightly out of date and it
       is possible to withdraw flow control credits.  Occassional
       oversends are not a problem, because the receiver can al-
   Garlick, Rom, & Postel                                       page 8
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
       ways discard incoming data without sending
       acknowledgements.
     Determining the Window Size                                    5d
       The TCP acknowledgement and retransmission scheme allows
       flexibility in determining the correct flow control window
       size.  The window size should indicate the willingness of
       the receiving process to provide buffer space.  The window
       size could represent exactly the available buffer space
       that the user has offered for letter receiving (the
       conservative strategy), or it could reflect some expected
       buffer space, based on previous allocations (the
       optimistic strategy).
       Conservative Guaranteed Allocation
         The conservative approach to window size setting gives
         the receiving process almost full control over the flow
         control mechanism.  By assuring the sender that there
         will be space for a particular number of octets, the
         policy reduces discards thus reducing the number of
         retransmissions.  (Some messages may still be discarded
         if they arrive out of order and sufficient reassembly
         space is not available.)
         There are some disadvantages to the conservative
         strategy of window size setting.  Flow control informa-
         tion is always slightly out of date when it is finally
         received.  The receiving process could have drastically
         increased or decreased its allocation, making the infor-
         mation useless.  Unless a process provides for double
         buffering, the window very likely will go from a fixed
         size (whatever the users buffer is) to zero, each time a
         message is passed on to the receiving process.  Depend-
         ing on the scheduling algorithm in the host, this could
         result in windows of size zero, totally inhibiting mes-
         sage flow.  Before messages can flow again, a packet
         with flow control information must arrive at the source.
         Thus, a round trip delay is experienced between messages
         and there is an increase of dataless packets in the net-
         work.
         Another related problem is that large single buffers may
         be used to receive small letters.  If a window of say
         size k is advertised and a packet of size << k arrives
         that includes the end of a letter, then the destination
   Garlick, Rom, & Postel                                       page 9
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
         buffer is returned to the receiving process.  The previ-
         ous flow control credit, which was large, is withdrawn
         and the window becomes zero.  In the interim, the sender
         may have sent several small letters, thinking the
         receiver has the buffers to accept them.  The receiving
         TCP, knowing that the receiving process has no available
         buffer space, will advertise a zero window.  By the time
         the window information arrives at the sending TCP, it
         likely will be an inaccurate report and cause further
         delays.
       Optimistic Credits
         The alternative to the conservative approach is to send
         flow control information that is a good estimate of the
         expected receiver's available space [References 3,7].
         Thus, the window size should be a function of previous
         window sizes as well as the current available space.
         The window size should be an average, weighted very
         heavily toward the current time, so that a process that
         is truly rejecting data will soon reflect a very small
         window.
         This method could even be mixed with heuristics to force
         the window to zero after a fixed period without re-
         ceiving.
         Optimistic allocation can do much to help solve the
         problem of drastic window size changes experienced with
         the conservative scheme.  In granting permission to
         transmit messages before the user has allocated buffer
         space, it fills the pipe and allows a smoother flow.  It
         is still reliable, because any message can be discarded
         in the receiver since it will be retransmitted later.
         The disadvantages of the method are its instability when
         faced with very irregular receiving patterns.  A poorly
         behaving receiver can still sabotage this policy, but
         not as easily as with conservative technique.  As will
         be shown below, an optimistic strategy may be quite
         dynamic with respect to recent receiving patterns,
         connection precedence, and the fair sharing of the
         available bandwidth.
         It may be possible to determine the semantics associated
         with the window size by exchanging transmission mode or
         topological information.  When a connection is opened,
   Garlick, Rom, & Postel                                      page 10
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
         the transmission mode (e.g., interactive, bulk) and the
         topology (e.g., satellite link) could be exchanged.
         This would be used to determine the weighting of previ-
         ous window sizes in calculating the current window.
         To demonstrate the idea of an optimistic flow control
         policy, a method for setting the receive window size is
         given in Appendix B.
     Zero Flow Control Windows                                      5e
       It may be necessary to stop the flow on a TCP connection,
       i.e., stop all new transmissions and unnecessary
       retransmissions.  This is required when there are no user
       receive buffers into which data can be placed.  A zero re-
       ceive window indicates an unwillingless to receive data.
       This reluctance is conveyed to the remote TCP by sending a
       packet with zero in the window size field.
       When interpreting packets, each TCP must read window sizes
       on all packets, even those that acknowledge old
       duplicates.  This is necessary for setting the window to
       zero when there is no data to carry the flow control in-
       formation.
       TCP must perform special functions with regard to sending
       packets into a zero window.  If no data is being sent on
       the connection, a zero window is of no concern to the
       sending TCP.  If there is data to be sent, it must be
       queued.  If necessary, new data from the sending process
       must be rejected.  The creation of new packets must be
       suspended entirely, and retransmission must be suspended,
       except for flushing controls, synchronizing controls, and
       the window opening control mentioned below.
       Opening a window of size zero also presents some special
       problems [Reference 6].  Since a window size can accompany
       each packet, it seems that the normal data packet and
       acknowledgement transmissions should be sufficient to vary
       the size of the windows.  However, when the remote TCP is
       showing a zero receive window, it is difficult to send a
       window change reliably.  A data packet cannot be sent be-
       cause the closed window indicates that only controls
       should be retransmitted; moreover, there may be no data to
       send.  If ACKs are used and they arrive out of order, it
       may be impossible to tell if the window is opening or
       closing.
   Garlick, Rom, & Postel                                      page 11
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
       The problem of opening a window of size zero is solved by
       using a pair of controls, one sent by the local TCP that
       is making its window size nonzero (WOPEN) and one that is
       sent by the foreign TCP to acknowledge the opening (WACK).
       These are special controls that must be handled immediate-
       ly, without regard for flow control restrictions.  If con-
       trols can be blocked by data, as in the present TCP, then
       the WOPEN must be tagged with, but must not consume a se-
       quence number.
   SEQUENCE NUMBER SPACE MANAGEMENT                                  6
     The second area of the current TCP protocol that needs at-
     tention is that of the reliable handling of the sequence
     number space.  In a packet-switching network with
     alternative routing schemes, a packet can have a relatively
     long lifetime, especially if the topology of the network in-
     cludes satellite links.  Due to misrouting, a packet can ar-
     rive at its destination minutes or even hours late, depend-
     ing on the topology.  A reliable protocol must be able to
     determine if such a packet is deliverable, acknowledgeable,
     or if it must be discarded without acknowledgement.  If dur-
     ing the packet's transit time the connection is closed or
     broken due to a crash with loss of memory, then the packet
     is no longer valid.  If the connection is reestablished,
     using the same source and destination addresses, then the
     arrival of the old packet can cause confusion in the re-
     ceiving TCP.  A reliable mechanism must exist to guarantee
     that the receiving TCP can distinguish packets of the cur-
     rent connection from packets of an old connection.             6a
     Resynchronization, suggested by Tomlinson [Reference 4,5],
     is one such mechanism.  Resynchronization is used in this
     paper to denote the mechanism itself, rather than the stage
     of the mechanism when the actual resetting of the sequence
     numbers is done.  The scheme is based on selecting initial
     sequence numbers (ISN's) from a curve in the sequence-
     number/time plane.  When a new connection is opened, its
     first sequence number is taken from the ISN curve.  If the
     consumption of sequence numbers is satisfactory, i.e., simi-
     lar in slope to the ISN curve, resynchronization of sequence
     numbers need not occur.  However, if the rate of consumption
     is too slow, resynchronization may be required to avoid
     colliding with the ISN curve.  The ISN curve has a parallel
     boundary (defining a "forbidden zone") that indicates that
   Garlick, Rom, & Postel                                      page 12
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
     no new sequence numbers may be assigned and that
     resynchronization must take place immediately.  If this is
     not done and if a crash occurs, sequence numbers assigned in
     the forbidden zone could conflict with the ISN chosen for
     the new connection.  See Appendix C, and References 4, 5, 6
     for further details of the resynchronization mechanism.        6b
     A few of the problems related to implementing
     resynchronization are discussed below.                         6c
       Understanding and Documenting the Problem
         Even though the resynchronization method is a workable
         one, it is not at all straightforward.  It takes
         numerous pages and illustrations just to document the
         concept [Reference 4,5,6].  As has been pointed out in
         the past by weathered ARPANET protocol implementers, a
         protocol must be reasonably easy to understand and easy
         to document.  After all, if the network is
         heterogeneous, it will be implemented on numerous kinds
         of hardware by system programmers with various degrees
         of skill.
       Testing for the Need to Resynchronize
         The protocol requires that if a connection is broken due
         to a system crash, the sequence number chosen at startup
         must be one that cannot be confused with any sequence
         number still in the network for the old instance of that
         connection.  To satisfy this requirement, periodic
         runtime checking must be done to determine if the se-
         quence number consumption rate is satisfactory, i.e., if
         it is approaching the forbidden zone.  This check must
         be done at fixed time intervals, not just when sequence
         numbers are being assigned.  The check may result in the
         need to resynchronize even (and especially) if the
         connection is idle.
       Resynchronization and Flow Control
         The need to resynchronize may occur at any time, and the
         resynchronization must proceed in a timely manner if
         normal activity is to continue.  However, since
         resynchronization means changing from the old sequence
         numbers to new sequence numbers and since the
         resynchronization control must be acknowledged (marked
         with an "old" sequence number), all data marked with the
   Garlick, Rom, & Postel                                      page 13
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
         "old" numbers must be acknowledged before the
         resynchronization control is acknowledged.  If data is
         not being accepted because the user is not receiving,
         then resynchronization cannot proceed.  If
         resynchronization cannot proceed, then neither new con-
         trols nor new data may be sent.
       The Loss of a Truly Out-Of-Band Signal
         Due to the flow control problem mentioned above, all
         controls can be blocked during a resynchronization pro-
         cess.  This includes the interrupt, which is supposed to
         be an out-of-band signal.  Losing the out-of-band capa-
         bility, even in rare instances, is an unfortunate defi-
         ciency.  Higher-level protocols that rely on an out-of-
         band signal could be severely crippled by the inability
         to interrupt a "runaway" process.  In fact, it is the
         runaway process, by not accepting data, that will soon
         force resynchronization and will not be interruptable.
       Extra Connection States and Controls
         When a state diagram is used to represent a TCP
         connection, 40% of the connection states are a result of
         the resynchronization mechanism [Reference 6].  These
         seven extra states allow for simultaneous
         resynchronization attempts and resynchronization
         attempts during connection closing (with no data loss).
         One extra control is required to support
         resynchronization.  It is believed that more would be
         required for satisfactory solutions to the problems of
         resynchronizing a connection that is blocked by data
         flow control and for support of a true out-of-band sig-
         nal.
       Decentralized Code
         Code to support resynchronization would be scattered
         throughout many modules of the protocol implementation.
         There must be a watchdog for detecting the forbidden
         zone.  There would be heuristics strewn throughout the
         control sending and parsing modules.  Also, to solve the
         flow control and interrupt problems mentioned above,
         special provisions must be made for either flushing data
         or saving old sequence numbers.
   Garlick, Rom, & Postel                                      page 14
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
     An Alternative to Resynchronization                            6d
       An alternative to resynchronization is a strategy that
       uniquely names each instance of a connection.  The name
       (or incarnation number) is passed in each packet and is
       used by the receiver to filter out packets from old
       connections.  The incarnation number is generated from
       clock time; thus, like the resynchronization method, no
       crash-proof memory is required.
       Each time a TCP comes up, it determines its incarnation
       number from a clock.  The appropriate clock resolution and
       wraparound period is a factor of the maximum packet
       lifetime for the network or interconnected network.  Let
       us assume that the clock has a resolution of one minute
       and a wraparound period of 256 minutes.  The resulting
       incarnation number is 8 bits long, and is used to assure
       the receiver that any message received with this
       incarnation number is from the active connection and not
       an old one.  The uniqueness of the incarnation number al-
       lows the resetting of the sequence number space to zero at
       initialization of each new path (first connection between
       two users).
       When a connection is closed, a TCP must save the last se-
       quence number used.  It must retain the number for time
       MPL (maximum packet lifetime).  Saving the sequence number
       and the time of a closed connection solves the problem of
       the repeated opening and closing of the same connection
       (source and destination).  It does not solve the problems
       created by TCP or host computer crashes.
       When connection establishment is requested, the list of
       old connections must be searched by (source, destination).
       If a match is found, the sequence number plus one is the
       first sequence number used when the connection is opened.
       If there is no match, then numbering can start at zero.
       Management of the old connection list entails removal of
       outdated items.  This can be handled, for the most part,
       during normal searching.  When list storage becomes
       scarce, a simple garbage collection routine can be
       invoked.
       There are two problems with the method using incarnation
       numbers.  First, there is some concern about the size of
       the old connection list.  It would not be surprising to
       see 1000 connections per hour for an average host.  The
   Garlick, Rom, & Postel                                      page 15
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
       fact that TCP allows a socket to be party to many
       connections will lead to fewer source and destination
       pairs; thus, many connections will be reused.  (This is in
       contrast to the ARPA network, where restrictions in socket
       usage result in contact connections being used to spawn
       direct, dynamically named service connections.)  Another
       factor that alleviates concern about the space required
       for the old connection list is the recent progress in
       inexpensive memories.
       The second problem is how to keep the incarnation number
       small enough to be sent in each header and still keep the
       clock cycle (name space) large enough to ensure
       uniqueness.  It is felt that an incarnation number field
       greater than 8 bits is excessive header overhead.  To ac-
       commodate this, the resolution of the clock is
       constrained, which leads to the following restriction ap-
       plied at host startup time.  When a host comes up after a
       crash, it must delay at least MPL / 2**8 before any
       connections are opened, so that a unique TCP incarnation
       number is always chosen.  A startup delay of one minute is
       probably sufficient for the internetting case since it
       implies a maximum packet lifetime (MPL) of 256 minutes.
   THE NEED FOR A CONTROL SUBCHANNEL                                 7
     In earlier versions of TCP, data, controls, and out-of-band
     signals (also a control) are all multiplexed onto one
     logical channel.  This means that one set of sequence num-
     bers is used for their orderly and reliable delivery.          7a
     One advantage of a single logical channel is the savings in
     the TCP header.  Protocol overhead is a serious matter,
     since it is suffered with each message.  Let us assume that
     it is desirable to allow piggybacking of activity from each
     channels.  Since each logical channel requires header fields
     for both a sequence number and an acknowledgement number,
     header sizes increase by twice the sequence number field
     size as each new channel is added.                             7b
     A second advantage to one logical channel is the ability to
     synchronize the control stream with the data stream.
     Synchronization of the control and data streams is useful
     for handling interrupts and connection closing (without data
     loss).  However, synchronization of streams can result in
   Garlick, Rom, & Postel                                      page 16
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
     unwanted interdependencies, since the acknowledgement of a
     control may require the acknowledgement of preceding data.     7c
     Two disadvantages of the single sequence number space scheme
     have been discovered recently:  reassembly of data mixed
     with controls is costly when packets arrive out of order,
     and a true out-of-band signal is not being provided.  The
     first problem is an efficiency matter that has plagued early
     implementers [Reference 9].  User buffer space cannot be
     used for the reassembly of out of order packets because
     there is no way to know if the unarrived packets contain
     only data or if controls are intermixed with the data.         7d
     The essence of the second problem is that the
     acknowledgement scheme requires that acknowledgement of a
     sequence number is implicit acknowledgement of all preceding
     sequence numbers.  Since interrupts must be acknowledged for
     reliability, the transmission of an interrupt can be blocked
     by data flow control in the receiver.  This was noticed by
     Cerf initially (Reference 2) and an attempt was made to
     rectify the matter by giving the interrupt extra semantics--
     that it always flushes unacknowledged data.  This solution
     is probably sufficient unless resynchronization methods are
     used for sequence number selection.                            7e
     As mentioned earlier, when the resynchronization method is
     used, there is no clean solution to the problem of achieving
     both synchronization with the data stream and independence
     of data flow control.  This is due to the fact that the
     resynchronizing control can be blocked by data flow control
     but cannot be flushed.                                         7f
     A compromise solution when using resynchronization is to
     separate controls and interrupts from the data channel, mak-
     ing a control subchannel.  The control sequence number is
     the composite of the data channel sequence number (DCSN) and
     the subchannel sequence number (SCSN).  This serves the dual
     purpose of synchronizing the two streams and using the
     resynchronization mechanism of the data channel for all
     subchannels.  A subchannel allows reliable transmission even
     when the data channel is inactive, without flushing data.      7g
     From the SCSN, the number of control fields, and the last
     SCSN received, the receiver can determine if subchannel
     traffic is coming in order and thus, whether it can be
     acknowledged.                                                  7h
   Garlick, Rom, & Postel                                      page 17
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
     The field size holding the SCSN determines the wraparound
     point in the SCSN space.  The SCSN space is initialized to
     zero when the DCSN is synchronized.  It IS NOT reset with
     each DCSN change.                                              7i
     There is no flow control information passed for the
     subchannel.  Discarding controls (without acknowledgement)
     is the flow control mechanism.  Since the sequence number
     space is small compared to that needed to prevent wraparound
     in the worst case, the TCP must keep track of the DCSN to
     which the first SCSN was assigned.  If wraparound of the
     SCSN space occurs, in the rare event that many controls are
     sent while the data channel is blocked, then the control
     channel becomes blocked.  This is very unlikely because a
     long series of controls will probably contain a string of
     interrupts, and successfully delivered interrupts will usu-
     ally cause the receiving process to unblock the data chan-
     nel.                                                           7j
     Acceptability Test for Subchannel Traffic                      7k
       The acceptability test of items on the subchannel is a
       composite test of both sequence numbers.  First the DCSN
       is checked to see if it would be acknowledged if it were
       an octet received on the data channel.  Only if it would
       have been discarded will the item on the subchannel be
       discarded.  Having passed the DCSN test, the SCSN is
       checked to see if the item is deliverable and
       acknowledgeable with respect to the SCSN sequence number
       space.  The SCSN test is less involved than the DCSN test
       because there is no flow control range.  To be believable,
       the SCSN must fall in the range of SCSN's sent and SCSN's
       for which acknowledgements have been received.  This is a
       check for everything except the existence of old
       duplicates from old instances of the connection, which is
       made by checking the DCSN.
   Garlick, Rom, & Postel                                      page 18
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
     A Scenario Using a Control Subchannel                          7l
       Let us examine a short scenario between TCP A and TCP B.
       The scenario assumes connections have been established and
       transmission has proceeded normally.  Only those header
       fields that relate to data and control channels will be
       indicated.  Note that the control length can be determined
       by the receiver from other fields in the header.  The fol-
       lowing shorthand will be used in the scenario:
         DSN - data sequence number
         DL - length of data in octets
         DACK - acknowledgement for all preceding data octets
         CSN - control sequence number
         CACK - acknowledgement for all preceding controls
   Garlick, Rom, & Postel                                      page 19
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
       #1 from TCP A
     -----------------------------------
     !  DSN ! DL ! DACK ! CSN ! CACK  ! ====>
     !  100 !  2 !  200 !   5 !   25  ! ====>
     -----------------------------------
       sends 2 data octets (100 & 101),
        acks data through 200;
       sends 1 control (5), acks controls
        through 25.
       
       #2 from TCP A
     -----------------------------------
     !  DSN ! DL ! DACK ! CSN ! CACK  ! ====>
     !  102 !  3 !  200 !   5 !   25  ! ====>
     -----------------------------------
       sends 3 data octets (102-104),
        acks data through 200;
       sends no controls,
        acks controls through 25.
       
       #3 from TCP A
     -----------------------------------
     !  DSN ! DL ! DACK ! CSN ! CACK  ! ====>
     !  105 !  3 !  201 !   6 !   25  ! ====>
     -----------------------------------
       sends 3 data octets (105-107),
        acks data through 201;
       sends 1 control (6),
        acks controls through 25.
       
                                #4 from TCP B
                             ----------------------------------
                       <==== !  DSN ! DL ! DACK ! CSN ! CACK  !
                       <==== !  202 !  1 !  101 !  26 !    6  !
                             ----------------------------------
                            Having received #1, #3, but not #2,
                              sends 1 data octets (202),
                                acks data through 101;
                              sends 1 control (26),
                               acks controls through 6.
       
     The main things to notice from this scenario are that data
     and controls are still piggybacked, as in the current
     version of TCP, and that there is a degree of independence
     between the two channels.  As the scenario shows, TCP B can
     acknowledge controls that have arrived in order even though
   Garlick, Rom, & Postel                                      page 20
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
     it has not received data in order.  Moreover, TCP B is able
     to use the latest data sequence number to test the accep-
     tability of the latest control sequence numbers.
   SUMMARY                                                           8
     Several suggestions have been presented here for the im-
     provement of TCP.  The suggestions relate to improved effi-
     ciency, simplification of implementation, and protocol
     functionality.  The motivation for the suggestions is more
     than to improve a specific protocol.  It is also to focus
     attention on a set of issues that are common to all reliable
     host-to-host protocols.                                        8a
     Flow control ideas have been discussed, with attention to
     implementation ideas that satisfy fairly ambitious goals.
     Window management techniques have been suggested that could
     improve efficiency.  A window setting method was presented
     that features optimistic credits that are a function of past
     credits, congestion, and available buffer space.               8b
     An alternative to the resynchronization method of sequence
     number space management has been given.  The suggested meth-
     od is based on passing TCP incarnation numbers and keeping
     an old connection list.  The method is simple to implement,
     requires no nonvolatile memory, and still guarantees reli-
     able detection of illegal packets.                             8c
     Finally, the need for the separation of data and control
     channels was motivated.  The solution, a reliable
     subchannel, is achievable with no separate sequence number
     space maintenance.                                             8d
     It is hoped that each of these suggestions will be imple-
     mented in future versions of TCP.  There are
     interdependencies involved; that is, some of the stated
     problems become less severe when others are solved.  For ex-
     ample, if resynchronization is abandoned, then the argument
     for separate channels is motivated only by the need for the
     efficient reassembly of out of order packets.                  8e
     Of all the suggestions, the most important is that concern-
     ing a new approach to sequence number space management.
     However, if resynchronization methods are retained, then a
   Garlick, Rom, & Postel                                      page 21
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
     subchannel for controls is a must.  Otherwise, a truly out-
     of-band signal is lost.                                        8f
     The discussion of flow control indicated areas that should
     gain attention as more experience with TCP is gained.  This
     should be an area for significant measurement, under many
     different transmission modes.
   Garlick, Rom, & Postel                                      page 22
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
   REFERENCES
     [1]     Cerf, V. and R. Kahn, "A Protocol for Packet Network
             Intercommunication," IEEE Transactions on Communica-
             tion, Vol COM-20, No. 5, May 1974.
     [2]     Cerf, V., Y. Dalal, C. Sunshine, "Specification of
             Internet Transmission Control Program," INWG General
             Note #72, December 1974 (Revised).
     [3]     Sunshine, C., "Interprocess Communication Protocols
             for Computer Networks," Digital Systems Laboratory
             Technical Note #105, December 1975.
     [4]     Tomlinson, R., "Selecting Sequence Numbers," INWG
             Protocol Note #2, September 1974.
     [5]     Dalal, Y., "More on Selecting Sequence Numbers,"
             INWG Protocol Note #4, October 1974.
     [6]     Postel, J., L. Garlick, R. Rom, "Transmission Con-
             trol Protocol Specification (AUTODIN II)," SRI-ARC
             Catalog #35938 & #35939, July 1976.
     [7]     Sunshine, C., "Factors In Interprocess Communication
             Protocol Efficiency For Computer Networks," Proc.
             National Computer Conf., 1976, AFIPS Press, pp
             571-576.
     [8]     Herrmann, Jeff, "Flow Control in the ARPA Network,"
             Networks, Vol 1, Number 1, June 1976.
     [9]     Burchfiel, J., W. Plummer, R. Tomlinson, "Proposed
             Revisions to the TCP," INWG Protocol Note #44, Sep-
             tember 1976.
   Garlick, Rom, & Postel                                      page 23
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
   GLOSSARY
     AHHP:  ARPANET host-to-host protocol.
     control:  commands passed between TCP's that are used to co-
     ordinate connection management.
     DCSN:  data channel sequence number.
     host:  a computer that is connected to the network and that
     executes programs on behalf of its users.  A host may pro-
     vide services to other computers on the network.
     ISN:  Initial sequence number; the first sequence number
     used when a connection is synchronized or resynchronized.
     MPL:  maximum packet lifetime.
     octet:  eight bits.
     SCSN:  subchannel sequence number; control channel sequence
     number.
     socket:  an entity defining one end of a TCP connection; the
     inter-network-wide name of a process port.
     subnetwork:  the network of computers that provides a com-
     munication medium for network hosts.  The nodes of a
     subnetwork may function as host interface points as well as
     store and forward computers.
     TCP:  Transmisssion Control Program and the protocol it
     implements.
     window:  a dynamic range in the sequence number space used
     in flow control management.
   Garlick, Rom, & Postel                                      page 24
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
   APPENDIX A:  PACKET ACCEPTANCE
     This appendix provides details of the TCP packet acceptance
     testing scheme.  It should clarify the possible actions the
     receiving TCP may take when it receives an arbitrary packet.
     Remember, the receiver is responsible for the detection of
     packets with improper sequence numbers from either old
     connections or ill-behaving TCP's.  For notation, let
       ADR = acknowledge and deliver range
       AOR = acknowledge only range
       DR = discard range
       S = size of sequence number space (number per octet)
       x = sequence number to be tested
       FCLE = flow control left window edge
       ADRE = (FCLE+ADR) mod S = Ack-deliver right edge (Discard
                left edge - 1)
       AOLE = (FCLE-AOR) mod S =  Ack-only left edge (Discard
                right edge + 1)
       TSE = time since connection establishment (in sec)
       MPL = maximum packet lifetime (in sec)
       TB = TCP bandwidth (in octets/sec)
     For any sequence number, x, and packet text length, l, if
       (AOLE <= x <= ADRE) mod S  and
       (AOLE <= x+l-1 <= ADRE) mod S
     then the packet should be acknowledged.
     If x and l satisfy
       (FCLE <= x <= ADRE) mod S  and
       (FCLE <= x+l-1 <= ADRE) mod S
   Garlick, Rom, & Postel                                      page 25
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
     then x can also be delivered to the user; however, ordered
     delivery requires that x = FCLE.
     A packet is not in a range only if all of it lies outside a
     range.  When a packet falls in more than one range, prece-
     dence is ADR, then AOR, then DR.  When a packet falls in the
     AOR then an ACK should be sent, even if a packet has to be
     created.  The ACK will specify the current left window edge.
     This assures acknowledgment of all duplicates.
     ADRE is exactly the maximum sequence number ever
     "advertised" through the flow control window, plus one.
     This allows for controls to be accepted even though
     permission for them may never have been explicitly given.
     Of course, each time a control with a sequence number equal
     to the ADRE is sent, the ADRE must be incremented by one.
     AOR is set so that old duplicates (from previous
     incarnations of the connection) can be detected and dis-
     carded.  Thus
       AOR = Min(TSE, MPL) * TB.
   Garlick, Rom, & Postel                                      page 26
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
   APPENDIX B:  WINDOW SIZE SETTING
     To demonstrate the idea of an optimistic policy for window
     size setting, a method for setting the receive window size
     is given [Reference 6].  The scheme satisfies the flow con-
     trol goals discussed earlier.  Several parameters have been
     vaguely unspecified since they can be determined only after
     considerable testing and measurement of a specific TCP im-
     plementation.
     First, some notation:
       B - Total bandwidth of the TCP, given unlimited user re-
       sources
       N - The number of connections in the TCP
       CONGEST - A congestion factor which reflects available TCP
       resources (CONGEST =< 1)
       WLT - The long term window
       W - The current window
       AVWT - Weighting coefficient for available buffer space
       OLDWT - Weighting coefficient for old window (OLDWT = 1 -
       AVWT)
       Tot - Total user buffer space
       Avail - The unfilled part of Tot
     The long term window might look like:
       WLT = B/N * CONGEST.
     The algorithm used to update the current window is the fol-
     lowing. Upon the processing of a user's receive request
     (buffer offering), the local receive window is set so that:
       W = MINIMUM(WLT, Tot).
   Garlick, Rom, & Postel                                      page 27
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
     Each time a packet is sent for this connection, the local
     TCP sets the receive window and the packet header window
     size field so that
       W = (AVWT * Avail/Tot) * WLT + (OLDWT * W)   (for nonzero
       Tot)
       and
       W = OLDWT * W                   (for Tot = 0).
     It is important to note that a user's receive buffer is re-
     turned when an End-of-Letter is received.  Thus, a small
     letter sent to a large buffer can cause the Avail and Tot to
     vary abruptly, even though there may be a smooth flow of
     letters.
     This window size setting scheme meets the goals mentioned in
     section 3 in the following ways:
       WLT is dependent upon the number of the connections,
       thereby administering fairness among connections.  It also
       considers the level of congestion in the receiving TCP,
       assuming some measure of resource availability can be pro-
       vided.
       The window size will never exceed the bandwidth allocated
       to the connection.  The algorithm may sometimes give cre-
       dit to a "well behaving" process by setting his window to
       greater than the actual buffer available. This window will
       be reduced if the process does not supply new receive
       buffers promptly.
       The current window size is dependent upon previous window
       sizes and upon the rate at which the process makes letter
       space available.  If a process fails to make such space
       available, its receive window will be reduced by OLDWT
       every time a packet is sent.  (The TCP may also apply a
       threshold mechanism by which a window is set to zero when
       it is reduced below the threshold.)
       The algorithm can be modified slightly to support high
       throughput for high precedence connections.  Parameter WLT
       cAn be made dependent on some criterion for the high pri-
       ority traffic.  Categories of priority can be used with
       some guaranteed service (part of the bandwidth) given the
       highest priority categories.
   Garlick, Rom, & Postel                                      page 28
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
   APPENDIX C:  RESYNCHRONIZATION DETAILS
     In Figure 2, we show the history of sequence numbers used by
     a particular connection.  The lines labeled "ISN" represent
     the maximum permitted rate at which sequence numbers can be
     used, however, this may be different than the maximum
     throughput rate for the TCP.
     Suppose that the TCP supporting the connection fails at "C"
     and must be restarted.  Assume, also, that the sequence num-
     ber selected to restart is drawn from the value of ISN at
     the time event "C" occurred.  The shaded area between "C"
     and "B" represents the maximum expected time that packets
     emitted at "C" can stay in the net.  Clearly, the ISN line
     intersects this shaded area, indicating that, after the
     restart, it is possible that packets emitted at "C" may be-
     come undistinguishable from those potentially emitted along
     the ISN curve.  To correct this flaw, the sequence number
     currently to be used on the connection must be
     resynchronized before running into the forbidden zone to the
     left of the ISN line.
     Testing for the need to resynchronize
       As packets are produced and sequence numbers assigned to
       them, the TCP must check for two possible conditions which
       indicate that resynchronization is needed.  The first is
       that sequence numbers are being used up so fast that they
       advance faster than ISN.  The other is that they advance
       so slowly that ISN "catches up with them."
       The basic method of selecting an initial sequence number
       is to delay for an arbitrary period labelled a "clock
       tick" or STEP and then select the new ISN.
       In Figure 2, three sequence number histories are traced,
       ending in points "A", "B", and "C".  In the trace labelled
       "A," sequence numbers are used at such a rate that point
       "A" lies beyond ISN plus one STEP.  If the connection were
       to fail and be restarted at "A," the new ISN would be just
       below point "A" and would introduce potential unwanted
       duplicates.
       This situation can be detected before transmission of the
       packet.  Let L be the length of the data in octets.  Let
       SEQ represent the proposed sequence number of the packet,
       and SEQ+L-1 be the sequence number implicitly associated
   Garlick, Rom, & Postel                                      page 29
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
       with the last octet of packet data.  Also, let SMPL be the
       sequence numbers consumed at maximum TCP throughput during
       a maximum packet lifetime.  If ISN+STEP (at the moment
       that SEQ is to be assigned) lies in the range [SEQ,
       SEQ+L-1], then the type "A" ISN failure is about to occur.
       The solution is to send only as much text as is allowed
       (which does not result in the failure) and WAIT for the
       clock to tick again.
       The situation in curve "B" is quite different.  In this
       case, the connection is using numbers so slowly that the
       forbidden zone preceding the ISN curve has advanced and
       run into the connection sequence number curve.  There are
       two solutions.  One is to wait for the packet lifetime
       plus one clock step to expire (in which case the sequence
       history will pop out of the forbidden zone again).  The
       other is to actively resynchronize the connection.  The
       test for the type "B" situation is whether sequence number
       SEQ lies in the range [ISN, ISN+SMPL+STEP].
       Note that all tests for inclusion must be modulo S, the
       size of the sequence number space, to account for the wrap
       around of sequence numbers.
       Curve "C" in Figure 2 shows a sequence number trace which
       tends, on the average, to lie within legal values at all
       times.
   Garlick, Rom, & Postel                                      page 30
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
   Garlick, Rom, & Postel                                      page 31
                                             LLG 8-Jun-77 13:01  29364
   Issues in Reliable Host-to-Host Protocols
   As presented at the Second Berkeley Workshop on Distributed
   Data Management and Computer Networks, May 1977, at Berkeley,
   California.####;
   Garlick, Rom, & Postel                                       page 0
/data/webs/external/dokuwiki/data/pages/rfc/ien/ien12.txt · Last modified: 2001/06/25 18:27 by 127.0.0.1

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki