Detecting mesoscale structures by surprise

The importance of identifying the presence of mesoscale structures in complex networks can be hardly overestimated. So far, much attention has been devoted to the detection of communities, bipartite and core-periphery structures on binary networks: such an effort has led to the definition of a unified framework based upon the score function called surprise, i.e. a p-value that can be assigned to any given partition of nodes, on both undirected and directed networks. Here, we aim at making a step further, by extending the entire framework to the weighted case: after reviewing the application of the surprise-based formalism to the detection of binary mesoscale structures, we present a suitable generalization of it for detecting weighted mesoscale structures, a topic that has received much less attention. To this aim, we analyze four variants of the surprise; from a technical point of view, this amounts at employing four variants of the hypergeometric distribution: the binomial one for the detection of binary communities, the multinomial one for the detection of binary "bimodular" structures and their negative counterparts for the detection of communities and "bimodular" structures on weighted networks. On top of that, we define two "enhanced" variants of surprise, able to encode both binary and weighted constraints and whose definition rests upon two suitable generalizations of the hypergeometric distribution itself. As a result, we present a general, statistically-grounded approach to detect mesoscale structures on networks via a unified, suprise-based framework. To illustrate the performance of our methods, we, first, test them on a variety of well-established, synthetic benchmarks and, then, apply them to several real-world networks, i.e. social, economic, financial and ecological ones. Moreover, we attach to the paper a Python code implementing all the considered variants of surprise.Comment: 32 pages, 4 tables, 12 figures. Python code available at: https://github.com/EmilianoMarchese/SurpriseMeMor