Mechanisms driving the formation and success of collaborations between scientists in biomedicine

See Has Part(s) field for link to corresponding datasetDespite decades of research into human social behavior, mechanisms that allow collaborations between scientists to form and thrive are not well understood. Surveys, interviews, and studies of laboratories inform most of what is known about collaboration; yet each method has limited scalability. Studies of extensive co-authorship networks offer an alternative approach; yet they tend to focus on only a few features or features of a certain type. Our understanding of collaboration has thus been muddled by two oversights. First, limited, biased samples (those restricted by size, scope, or dimension) lead to conclusions that might not generalize. Second, surveys and interviews do not necessarily capture reality, because people do not always do what they say they do. The present study overcomes these problems using an approach complementary to field study. First-time co-authors (with mutually exclusive publishing histories) reflect collaborative formation at scale---represented herein by ~1.4 million papers in PubMed in the period 1988 to 2009. With these data, collaborative formation and its "success" are modeled to assess the influence of several explanatory factors, including topics, personal characteristics, affiliations, citations (direct and indirect), and co-authorship network. Similarity, nearness, and complementarity are encoded in 30 base features that capture obvious as well as indirect connections between people. Results show that all the factors influence collaborative formation to varying degrees. Topical similarities dominate. Scientists rarely (2%) form collaborations with people who have never published on the same or similar topics or journals. Personal characteristics are second in influence; but their effect is more complex, because they combine elements that are similar (such as ethnicity and gender) with those that are complementary and change over time (such as professional age). Affiliation is third, for which the effect of shared institution type and geolocation (city, US state, or country) is positive. Citation is fourth. Direct citation (citation of one author by another) and indirect citation (in the form of co-citation) both have positive influence on collaborative formation. The co-authorship network is last; and its overall effect is minuscule. Scientists are more likely to form collaborations with others outside their overlapping co-authorship networks. Even when collaborations form within a co-authorship network (with the friend of a friend), other factors (such as topical and personal characteristics) are more influential in co-author selection. "Success" about a newly formed collaboration is coded in two different ways in this study: by "relative impact" of a paper (whether it has an above-average number of citations for its journal and year of publication) and by the "longevity" of the collaboration (whether the collaborators have at least one future paper together). The models of collaborative success show that all factors influence success and that the co-authorship network is the least explanatory. However, the factors behave differently than collaborative formation; the patterns are more complex. For example, personal characteristics are more complementary than similar, suggesting that what people tend to do is not necessarily what they should do. The foregoing casts new light on collaboration. The lessons are manyfold; but at least two deserve mention. First, topical, cultural, and geographical silos exist in science; yet "success" favors those willing and able to reach beyond those silos to embrace some diversity and complementarity. Second, the co-authorship network alone fails to capture almost everything that matters in predicting future links between people