Comparing Modularity Scores across Different Social Networks: Pitfalls, Illustrations, and Suggestions

Abstract

In the thriving field of network studies, there has been an emerging practice of comparing optimal modularity across social networks to evaluate the variation of network module–related substantive concepts, such as the level of consensus, polarization, or community boundary rigidity. This practice offers valuable insights and is often thoughtfully motivated, but it faces several conceptual and empirical challenges that merit careful consideration. Conceptually, the selected modularity metric may misalign with the substantive concepts researchers aim to measure. Empirically, estimated optimal modularity is highly sensitive to algorithm choice and to network characteristics unrelated to those substantive concepts, which can bias comparison results. The authors illustrate these issues with toy examples and systematic simulations and offer suggestions for more rigorous comparison practices. To show the practical significance of these lessons, the authors replicate an empirical study that examines the temporal trend of modularity scores for job mobility networks to evaluate the evolution of mobility boundary rigidity in the U.S. labor market.