Latent semantic indexing in peer-to-peer networks

Abstract

Searching in decentralized peer-to-peer networks is a challenging problem. In common applications such as Gnutella, searching is performed by randomly forwarding queries to all peers, which is very inefficient. Recent researches utilize metadata or correlations of data and peers to steer search process, in order to make searching more purposeful and efficient. These efforts can be regarded as primitively taking advantage of Latent Semantics inhering in association of peers and data. In this paper, we introduce latent semantics analysis to peer-to-peer networks and demonstrate how it can improve searching efficiency. We characterize peers and data with latent semantic indexing (LSI) defined as K-dimensional vectors, which indicates the similarities and latent correlations in peers and data. We propose an efficient decentralized algorithm derived from maximizing-likelihood to automatically learn LSI from existing associations of peers and data (i.e. from (peer, data) pairs). In our simulations, searching efficiency can be greatly improved based on LSI, even with the simplest greedy search preference. Our approach is a framework to exploit inherent associations and semantics in peer-to-peer networks, which can be combined fundamentally with existing searching strategies and be utilized in most peer-to-peer applications. © Springer-Verlag 2004.