GENWiki

Premier IT Outsourcing and Support Services within the UK

User Tools

Site Tools


rfc:ien:ien141
   IEN 141
   INDRA Note 897
   11th April 1980
                    Message System Issues
                        C. J. Bennett
        ABSTRACT:  This  INDRA  Note  discusses   the
        design  choices for the message server system
        to  be  built  at  UCL.   Particular   issues
        considered include: the nature of the UK user
        community; the nature of the message  service
        to  be  offered  on  the  server; the message
        formats and transfer protocols  to  be  used;
        addressing;  interworking  with  the  ARPANET
        community; and  the  design  of  the  message
        management system on the message server.
                      Table of Contents
1. Introduction...........................................1
2. The User Community.....................................1
3. Message Movement.......................................2
   3.1 Message Format.....................................2
      3.1.1 Message Format Staging........................3
   3.2 Message Protocol...................................3
   3.3 Message Transport..................................4
      3.3.1 FTP Staging...................................5
   3.4 Addressing.........................................5
   3.5 Status Reporting...................................8
4. Message Server Design..................................8
   4.1 User Interface.....................................8
   4.2 Message Management.................................10
5. Conclusions............................................12
   1. Introduction
     Electronic message services  have  historically  been
   one  of  the most successful services to have developed
   from the use  of  packet  switched  computer  networks.
   However,  these  facilities  have not been available to
   users of United Kingdom research data networks  in  the
   past,  and  UK  users who wished to send mail to remote
   sites were  required  to  obtain  mailboxes  on  remote
   machines  in the United States, accessible via ARPANET.
   With the development of public networks, in  particular
   IPSS  and  PSS,  and  in  view  of the UKPO's policy of
   requiring users to move to these  networks,  it  is  no
   longer  economically  feasible to continue this mode of
   usage.
     For these reasons  it  is  proposed  that  University
   College  London  will  develop  a message server system
   based  on  a  PDP-11/35  running  UNIX  and  accessible
   initially to users through the DARPA Catenet, and later
   through PSS. This server would allow users to  exchange
   messages  with  other  users on the same site, users of
   ARPANET mail systems, and eventually users of other  UK
   and  US message servers.  The aim of this INDRA note is
   to identify the design constraints on this  system  and
   to suggest approaches that may be taken to meet them.
   2. The User Community
     Five major groups of users can be identified who  can
   be  expected  to  interact  with  such a service in the
   short term.  These are:
    (i)    Current  users  of  the  ARPANET  mail  system,
           especially  UK  users who have (until recently)
           had dialin access through the TIP. The  message
           server  would  become the prime mail server for
           this group. US users of ARPANET systems must be
           able to send messages to this site.  This group
           will require messages  formatted  according  to
           the  rules specified in RFC 733 (as modified by
           actual practice).
    (ii)   Users of the DARPA Catenet, who will  be  using
           at  least  three  formats  for  intersite mail:
           those of RFC 733; those of  the  Internet  Mail
           Protocol  as defined in IEN 85; and the private
           formats being developed by RSRE.
    (iii)  Users who wish to exchange messages between the
           UCL  server  and other servers which may become
           available  through   PSS.   This   group   will
           initially require only PSS access to the server

Bennett [Page 1] INDRA Note 897, IEN 141 Message System Issues

           and will exchanges messages locally, but in the
           longer  term  it  can be anticipated that other
           mail servers will emerge on PSS.
    (iv)   Users who wish to  exchange  messages  with  US
           message  servers  available through Telenet and
           IPSS. In particular,  such  traffic  may  arise
           through the US EDUNET project.
    (v)    UCL users who will  exchange  messages  through
           the  UCL  ring,  and  who will wish to exchange
           messages with users in one or more of the other
           three categories.
   3. Message Movement
     This section is concerned with  the  questions  which
   affect  the  movement  of  messages between the message
   server and other message sites.  Four  major  questions
   must be considered: choice of message format; choice of
   transport mechanism; mail protocol; and addressing.
   3.1 Message Format
     The message  format  may  be  based  on  one  of  the
   following choices:
    (i)    ARPANET Format (RFC 733)
    (ii)   Internet Mail Format
    (iii)  RSRE Mail Format
    (iv)   Other format not currently in use  amongst  the
           user  community,  such  as those that may arise
           through the work  of  IFIP  TC6.5,  or  through
           Telenet and EDUNET.
     Of these choices,  only  the  first  is  feasible  at
   present.  It  is  that which is most widely used at the
   moment,  as  it  provides  the  current  ARPANET   mail
   service, and the internal UCL Unix mail service, and it
   is intended that it shall be  used  for  initial  DARPA
   Catenet  mail.  The  DARPA Internet Mail format is very
   experimental, and although it  is  expected  to  remain
   stable for the time being no experience has been gained
   with it. Much the same  comment  applies  to  the  RSRE

Bennett [Page 2] INDRA Note 897, IEN 141 Message System Issues

   system. The fourth choice involves either obtaining  an
   existing commercial system such as COMET, or devising a
   new format from scratch. Both these possibilities would
   result  in  considerable  delay,  and a UCL home-brewed
   format would be unlikely to be any  more  satisfactory,
   and  would  be  much less acceptable to the users, than
   other alternatives.
     As it may be anticipated that the server will have to
   interwork  eventually  with other formats, notably that
   of RSRE and whatever emerges amongst the EDUNET  group,
   the  development  of  other  formats  should be closely
   tracked. It is expected that conversion will eventually
   take  place through the use of a common Internet Format
   such as that being  developed  in  the  DARPA  Internet
   scheme.
   3.1.1 Message Format Staging
     One result of this is that users who will  eventually
   require  a  different format for messages for their own
   server - initially, RSRE in particular - will require a
   conversion  between  the  two. It is expected that this
   will take place at the UCL message  server.   As  noted
   above,  it  is  to  be hoped that conversions will take
   place through a common intermediary format.
     An important longterm question in this regard is  how
   widely   the   UCL   message   server  system  will  be
   distributed in the UK. If  other  message  servers  are
   built along the same lines, then the format chosen will
   become a __ _____ UK standard, at least  among  the  UK
   research community.
   3.2 Message Protocol
     The current ARPANET message protocol is essentially a
   trivial   extension   to  the  ARPANET  file  transfer,
   obtained through the  MAIL  option.  This  causes  each
   message to be sent as a separate file to be appended to
   the message file of an individual user  at  that  site.
   Given  future use of IPSS and PSS this is an uneconomic
   option. There are two reasons for this.
    (i)    Demultiplexing for a message  which  is  to  be
           copied to several users at the same site occurs
           at  the  sender,  not  the  receiver.   Thus  a
           message  for N users at site X is transferred N
           times, even though it is identical. If  mailers

Bennett [Page 3] INDRA Note 897, IEN 141 Message System Issues

           were capable of  parsing  the  message  headers
           properly, the message need only be sent once.
    (ii)   For each message transferred  a  separate  data
           connection  is  set  up.  Thus  a  queue  of  N
           messages for M sites (M < N) will require N + M
           calls   to   be  made.  If  the  messages  were
           mailbagged by site, only 2M calls need be made.
           (Note  that  if FTP control and data were mixed
           on the same call, as in the NIFTP (see  below),
           these figures reduce to N and M respectively).
     Both  these  changes  have  some  impact  on  message
   format.  The  first  requires,  as  a minimum, that all
   recipients of a message at a given site be  visible  in
   the To: and Cc: fields - that is, it is not possible if
   the mailing list facility is used in its current  form.
   In  such  cases,  the sender must provide the list, and
   the receiver must recognise that this  list  should  be
   suppressed  or  separated from the users' copies. It is
   to be hoped that the Internet group  will  accept  this
   proposal  as a minimum change to be made for use in the
   Catenet, and that similar procedures will be set up  by
   other groups.
     Mailbagging requires that  different  messages  in  a
   file   transferred  must  be  clearly  delimited.  This
   requires a mailbag structure to be  defined  -  at  the
   very  least,  by defining a standard message separator.
   However,  it  does   not   require   restructuring   of
   individual  messages.  This  is  a  much more important
   change than the first, and as the saving is  likely  to
   be  less,  it is proposed here that it should await the
   results of experiments with the Internet Mail Protocol.
   3.3 Message Transport
     There are two  major  choices  to  be  made  for  the
   message  transport service, namely the TCP FTP, derived
   from the ARPANET FTP, and the NI FTP.  It  is  expected
   that  the  first  will  be  used  for  mail  within the
   Catenet, using the same MAIL option as used within  the
   ARPANET. As has been seen above, however, this protocol
   is unsuited to our needs because it is  uneconomic.  It
   may   be   retained   initially,  as  it  gives  direct
   compatibility with other Catenet sites.

Bennett [Page 4] INDRA Note 897, IEN 141 Message System Issues

     In the slightly longer term, the NI FTP is  the  more
   attractive   option.  The  reasons  for  this  are  its
   independence of specific  transport  services  and  the
   fact  that  it  will  be  widely adopted in the UK. UCL
   already has implementations on its research Unix and at
   ISIE  (though  these will have to be changed to reflect
   the final specification); an implementation at RSRE  is
   planned;  and future mail servers in the UK will prefer
   to use it. The fact that many of these will  run  above
   X25  networks  while  Catenet  sites  will  use  TCP is
   immaterial; the  necessary  transport-level  conversion
   will  be  handled  by  the  UCL Protocol Convertor. The
   existing ARPANET FTP is demonstrably NCP-specific,  and
   the  TCP  version  of  this  will  at  the  minimum  be
   Catenet-specific in its use of Telnet.
   3.3.1 FTP Staging
     An important consequence of this is that FTP  staging
   will be required, for three reasons.
    (i)    It will be necessary to stage messages into and
           out  of the ARPANET. This applies regardless of
           the FTP used, as ARPANET mail is restricted  to
           use of the ARPANET FTP.
    (ii)   It will be necessary to stage messages  between
           mailers  in  the  Catenet using the TCP FTP and
           those using the NI FTP. If UCL does  decide  to
           use  the  TCP  FTP,  this  decision  is  merely
           postponed until a UK community emerges based on
           the NI FTP.
    (iii)  It  may  eventually  be  necessary   to   stage
           messages   between   UCL   and   Telenet/Tymnet
           servers, even if they adopt a common format, if
           a different transport mechanism is used.
   It is proposed here that experiments with the first two
   stagings  be performed at ISIE, or some other TOPS20 on
   the ARPANET which has all three systems. In  its  final
   form,  the  staging  system  would  consist of a daemon
   which would process the mail file at a special  account
   and  forward  messages  to  the appropriate sites.  The
   structure of such a system is shown in Figure 1.
   3.4 Addressing
     Only four message  sites  in  the  UK  are  initially

Bennett [Page 5] INDRA Note 897, IEN 141 Message System Issues

               Figure 1: Staging Daemon System
   expected  to  be  heavily  involved  in   the   system.
   Initially,  development  will  be  in  the  UCL message
   server itself (UCL-MUnix), while at a later  stage  the
   UCL  teaching and research machines (UCL-TUnix and UCL-
   RUnix), and at least one machine at  RSRE  will  become
   involved.  While  other message servers may emerge at a
   later date, it is not expected that  this  will  happen
   rapidly.  Staging  to Catenet and ARPANET sites will be
   through ISIE; the problem of staging to  Telenet/Tymnet

Bennett [Page 6] INDRA Note 897, IEN 141 Message System Issues

   sites must be considered if and when it arises.
     The UK sites should be able to exchange mail directly
   through  the  use  of addresses of the form 'user@site'
   (e.g.  Ruth@UCL-TUnix).   This  format  could  be  used
   throughout  the  mailing  address  space,  although  it
   involves the message sites not  under  UCL  control  to
   make  special  modifications  to their mailers. Thus an
   ARPANET  mailer  presented  with   a   return   address
   'Ruth@UCL-TUnix'  would  have  to  recognise  that this
   should be sent to ISIE; the ISIE mailer would  have  to
   recognise  that  the message should be added to the UCL
   daemon's mailbox and the UCL daemon would then  forward
   the message to UCL-TUnix.
     Two  other  alternatives  are  source   routing   and
   hierarchical  addressing.  A  source routed form of the
   address might be identical in appearance to the ARPANET
   (by  making  'UCL' a synonym for ISIE, in much the same
   way  the  'UDel-EE'  is  a  synonym  for  'Rand-Unix'),
   although for parsing purposes it would be preferable to
   rearrange  it:  (Ruth-(TUnix@(UCL))).  Local   messages
   would  then  appear  as: Ruth-TUnix. An ARPANET address
   would appear to a message server user in  a  form  such
   as: Kirstein-ISI@ISIE. Staging message servers would be
   required to parse the address into intermediate  forms.
   Further,  the  terminal  staging server for the catenet
   and  for  ARPANET  would  be   required   to   suppress
   intermediate  fields. Thus the UCL daemon at ISIE would
   have to transform all addresses of the form:  Kirstein-
   ISI@ISIE to Kirstein@ISI and back again for traffic in
   the reverse direction. Source routing is  the  favoured
   solution of the University of Delaware's MMDF group.
     Hierarchical  addressing  is  actually  the  official
   ARPANET  standard  as described in RFC 733, although it
   is not implemented. It is also the solution favoured in
   Postel's  Internet  system. Under this scheme UCL would
   refer  to  a  widely-known   addressing   domain,   and
   addresses  would  take  the form: Kirstein-ISI@ARPA and
   Ruth-TUnix@UCL. In practice, since only  two  hops  and
   only  one  staging point are involved the two forms are
   virtually synonymous - which is  a  good  argument  for
   postponing  a  real decision until we see an addressing
   hierarchy actually emerging! The  differences  will  be
   seen  when an RSRE server becomes active. In this case,
   an ARPANET site has the choice of the following forms:
        Bryan@NSide               (global)
        Bryan-NSide@PPSN          (hierarchical)
        Bryan-NSide-MUnix@ISIE    (source routing)

Bennett [Page 7] INDRA Note 897, IEN 141 Message System Issues

     Note that in any form changes of the type  above  are
   required   to   ARPANET   mailers.   With   global  and
   hierarchical  addressing,  ARPANET   tables   must   be
   modified  to recognise mail servers (global address) or
   mail address spaces (hierarchical  address).   This  is
   not  required  with  source routing.  The mailer at the
   staging site must additionally recognise  that  account
   names  taking  a certain format should automatically be
   accepted and routed to the  UCL  mail  daemon  at  that
   site. Both solutions therefore require some structuring
   of the address. In the examples above, a  hyphen  ('-')
   has  been  used as a component separator. In fact, this
   is probably a bad choice. Two possibilities are:
    (i)    Use of some other separator, such as %.
    (ii)   Use of the comment fields allowed by  the  mail
           protocol.
   The second choice has the convenient side  effect  that
   the  account  checking procedure need not be changed at
   the staging site, as  addresses  may  then  look  like:
   UCLfor   a   source-routed
   format). However not all message preparation facilities
   will include comment fields (e.g. 'answer' under MSG).
     Since this note was first drafted  my  attention  has
   been  drawn  to  RFC754  (Out-of-Net Host Addresses for
   Mail  by  J.  Postel).   This   note   considers   four
   solutions:  three  are variants on the global solution,
   and the fourth involves name structuring. Postel's note
   favours  a structured name solution. This is compatible
   with  either  a   source   routed   or   hierarchically
   structured solution.
   3.5 Status Reporting
     Finally in this section there is the issue of  status
   reporting.   Currently,  most ARPA-type message systems
   give an  immediate  report,  with  possibly  a  mailer-
   generated  message if there is some subsequent failure.
   For staged or mailbagged messages an  immediate  report
   of  success  can only imply success at the first stage.
   Thus it is important that staging daemons which  cannot
   successfully  deliver  a  message  must  be prepared to
   generate messages indicating why failure occurred. This
   can  be  done  simply  through  the  use of the current
   message generation mechanism.

Bennett [Page 8] INDRA Note 897, IEN 141 Message System Issues

   4. Message Server Design
   4.1 User Interface
     The primary service  which  must  be  provided  is  a
   reliable,  efficient  and  cheap  method of sending and
   processing text messages  exchanged  amongst  the  user
   community.  It  is not intended to provide a multimedia
   service, although this is an important research goal of
   the  program.  Within  this  constraint,  a user of the
   message server must be able to:
    (i)    Prepare messages.
    (ii)   Send messages to remote users.
    (iii)  Receive messages from remote users.
    (iv)   Read messages.
    (v)    Be assured that messages are safely stored  and
           are recoverable in the event of system failure.
    (vi)   Be able to obtain adequate online help  on  the
           use of the server.
   In addition it is desirable that the user be able to:
    (i)    Prepare message files which  may  not  be  sent
           immediately.
    (ii)   Archive and dearchive messages.
    (iii)  Manipulate messages in file structures  of  his
           own creation.
    (iv)   Answer and forward messages.
    (v)    Obtain hardcopy listings.
    (vi)   Maintain mailing lists.
    (vii)  Annotate messages.
     This list is clearly not exhaustive, and the aims  of
   the user interface should be continually reevaluated in
   the light of user experience,  development  experience,
   and  the  recommendations of other message groups, such
   as IFIP TC6.5.  Nor does it imply any evaluation of the
   difficulty  of implementation: answering and forwarding

Bennett [Page 9] INDRA Note 897, IEN 141 Message System Issues

   messages  should  be  comparatively  trivial;  while  a
   satisfactory remote hardcopy listing service is a major
   problem.
     Following the general approach taken in this note, it
   is  proposed that MSG be used at least initially as the
   basis of the user interface in the message server.  The
   user  would enter MSG automatically as his login shell.
   It is expected that the repertoire of commands will  be
   changed and extended in order to provide the full range
   of services listed above (e.g. for the  maintenance  of
   mailing  lists).  This  may  require  the single-letter
   command interface to be modified. It is  also  expected
   that  the  character-at-a-time interface and the use of
   TV editors would have to be altered to fit the needs of
   users  accessing  the  system  via XXX terminals, which
   favour line-oriented commands and editors. These issues
   will be reexamined in the light of experience gained.
   4.2 Message Management
     An important issue is  the  internal  design  of  the
   message  server. The current system of personal mailbox
   files each containing a copy of all messages is complex
   and wasteful in a Unix system solely devoted to message
   handling. It is proposed here that database  structures
   be examined in which only one copy of a message is kept
   in a central directory, and  that  the  user's  current
   mail  file,  and any other mail files he keeps, consist
   solely of descriptors pointing to the  message  and  to
   other   cross-referencing   descriptors  which  may  be
   needed. The structure of the system is shown in  Figure
   2.
     The details  of  the  descriptor  structure  are  not
   considered in this note. However, a number of important
   issues arise. The fundamental question is:  should  all
   messages be kept in a single file, or each message in a
   separate  file?  The  answer   chosen   has   important
   implications  for the limits on the size of the system,
   the method of updating the system, methods of accessing
   messages, and many other issues.
     In the second method, messages may be  found  rapidly
   by  filename,  and  garbage  collection is considerably
   simplified through the  use  of  Unix  file  management
   facilities,  but  on  average  256  bytes  (half a disc
   block) will be wasted per message. Further, at most  an
   entire  file  system  of 64K blocks can be allocated to
   message  service,  although  this  is  not  a   serious

Bennett [Page 10] INDRA Note 897, IEN 141 Message System Issues

           Figure 2: Message Management Structure
   restriction. Assuming that most messages will be small,
   of  the  order  of 2K characters, the file system would
   allow something less than 16K messages, wasting some 4K
   bytes  of  space. Thus a more serious limitation is the
   number of inodes (file descriptors)  allocated  to  the
   system,  which  is  currently  about 2^13 - allowing 8K
   files. Increasing this to 2^14  is  not  difficult  and
   will allow 16K files, of which a significant proportion
   would be for user descriptor information.

Bennett [Page 11] INDRA Note 897, IEN 141 Message System Issues

     The first method allows more efficient use  of  space
   and  places  a much looser restriction on the number of
   messages that may be retained,  but  requires  building
   searching and garbage collection facilities parallel to
   Unix's. In order  to  use  these,  moreover,  either  a
   complex  file  structure  must  be defined, or a master
   descriptor file retained.
     Pending further investigation, the second  choice  is
   favoured  at this stage. The fact that only one copy of
   a message need be kept  should  help  to  minimise  the
   effects  of  the  restrictions.  Ensuring this may be a
   problem, especially if multiple copies of a message are
   received.  Hence  an important aspect of the system may
   be to examine incoming messages and attempt  to  detect
   duplicates of existing messages.
   5. Conclusions
     The message system discussed here is  centred  around
   text  messages  based largely on ARPANET-style formats,
   at least  initially.  Nevertheless  there  are  several
   important  issues  which  must  be resolved in order to
   bring up a workable system. These issues include:
    (i)    Economic use of transfer and storage resources.
    (ii)   The structure  of  UCL-style  mail  daemons  at
           staging site(s).
    (iii)  The  modification  of  other  mail  servers  to
           handle UCL mail.
    (iv)   Basic addressing style.
    (v)    Detailed user interface.
    (vi)   Message management issues.
   This note has indicated some lines of approach to these
   problems.  They  will  be  examined  in  more detail in
   future notes, prior to the commencement of actual  work
   on  the  system  later  this  year.  It  is  clear that
   satisfactory   progress   requires   cooperation    and
   discussion   with  other  parties,  notably  the  DARPA
   Catenet group and groups using various  public  carrier
   services.  While  the  projects  of the former are more
   advanced at this point, it is expected that the  latter
   groups  will  become increasingly important in the long
   term.

Bennett [Page 12]

/data/webs/external/dokuwiki/data/pages/rfc/ien/ien141.txt · Last modified: 2001/06/25 19:10 by 127.0.0.1

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki