Distributed hashtable on pre-structured overlay networks

Abstract

Internet overlay services must adapt to the substrate network topology and link properties to achieve high performance. A common overlay structure management layer is desirable for enhancing the architectural modularity of service design and deployment. A shared substrate-aware overlay structure can also save redundant per-service link-selection probing when overlay nodes participate in multiple services. Despite the benefits, the concept of building services on a common structure management layer does not work well with recently proposed scalable distributed hashtable (DHT) protocols that employ protocol-specific overlay structures. In this paper, we present the design of a self-organizing DHT protocol based on the Landmark Hierarchy. Coupled with a simple low-latency overlay structure management protocol, this approach can support low-latency DHT lookup without any service-specific requirement on the overlay structure. Using simulations and experimentation on 51 PlanetLab sites, we measure the performance of the proposed scheme in terms of lookup latency, load balance, and stability during node churns. © Springer-Verlag 2004.