Gradients of O-information: Low-order descriptors of high-order dependencies

Abstract

O-information is an information-theoretic metric that captures the overall balance between redundant and synergistic information shared by groups of three or more variables. To complement the global assessment provided by this metric, here we propose the gradients of the O-information as low-order descriptors that can characterize how high-order effects are localized across a system of interest. We illustrate the capabilities of the proposed framework by revealing the role of specific spins in Ising models with frustration, in Ising models with three-spin interactions, and in a linear vectorial autoregressive process. We also provide an example of practical data analysis on U.S. macroeconomic data. Our theoretical and empirical analyses demonstrate the potential of these gradients to highlight the contribution of variables in forming high-order informational circuits.