The Totem Multiple-Ring Ordering and Topology Maintenance Protocol

Abstract

The Totem multiple-ring protocol provides reliable totally ordered delivery of messages across multiple local-area networks interconnected by gateways. This consistent message order is maintained in the presence of network partitioning and remerging, and of processor failure and recovery. The protocol provides accurate topology change information as part of the global total order of messages. It addresses the issue of scalability and achieves a latency that increases logarithmically with system size by exploiting process group locality and selective forwarding of messages through the gateways. Pseudocode for the protocol and an evaluation of its performance are given. Categories and Subject Descriptors: C.2.1 [Computer-Communication Networks]: Network Architecture and Design - network communications; C.2.2 [Computer-Communication Networks]: Network Protocols - protocol architecture; C.2.4 [Computer-Communication Networks]: Distributed Systems - network operating systems; C.2.5 [Computer-Communication Networks]: Local Networks - rings; D.4.4 [Operating Systems]: Communications Management - network communication; D.4.5 [Operating Systems]: Reliability - fault-tolerance; D.4.7 [Operating Systems]: Organization and Design - distributed systems.