Distributed Computation of Top-k Degrees in Hidden Bipartite Graphs

Abstract

Hidden graphs are flexible abstractions that are composed of a set of known vertices (nodes), whereas the set of edges are not known in advance. To uncover the set of edges, multiple edge probing queries must be executed by evaluating a function f(u, v) that returns either true or false, if nodes u and v are connected or not respectively. Evidently, the graph can be revealed completely if all possible n(n - 1)/2 probes are executed for a graph containing n nodes. However, the function f() is usually computationally intensive and therefore executing all possible probing queries result in high execution costs. The target is to provide answers to useful queries by executing as few probing queries as possible. In this work, we study the problem of discovering the top-k nodes of a hidden bipartite graph with the highest degrees, by using distributed algorithms. In particular, we use Apache Spark and provide experimental results showing that significant performance improvements are achieved in comparison to existing centralized approaches.