Benchmarking search and decomposition on synthetic gene sets.

(A) Schematic of the procedure for the generation of synthetic gene sets. Starting from a pool of genes (dots) and a set of curated gene sets (color boundaries), the procedure entails three distinct strategies. One strategy picks genes from between one and four curated gene sets (top). Another strategy modulates the proportion of genes that are transferred from the curated sets into the synthetic sets (middle). The last strategy supplements each synthetic set with genes picked from the pool at random, modulating the signal-to-noise signal (bottom). The schematic is an exact repeat of a figure panel in the main manuscript. (B) Summary of performance of Fisher, search, and decomposition algorithms on an ensemble of 64,000 synthetic gene sets generated according to the schematic in (A). The evaluation metric is the sum of Jaccard Index (JI): for a benchmark set made up of n components, this metric takes the n top-ranked hits from each method, evaluates the JI between the hits and the true curated sets, and reports the sum of the JI values. Effectively, the total JI counts the number of expected sets found. Box plot bounds, center line, and whiskers represent 25%-75%, 50%, and 5%-95% quantiles. Boxes contain results from all benchmark sets generated according to a mixture of strategies. Rows stratify the data according to number of components, coverage, and signal. Columns show results computed using crossmap instances using uniform feature weighting and feature weighting based on information content (IC). In the latter, genes that appear in many gene sets are down-weighted as non-informative. The Fisher method does not utilize weighting, so Fisher results are exactly the same in the two columns. (C) Effect of diffusion on search and decomposition performance. In these calculations, diffusion appears to lower performance slightly. (TIF)