Publishers/sources (Disinformation)

Recent research has mainly used two approaches to identify publishers or sources of disinformation: First, alternative media are identified as potential publishers of disinformation. Second, potential publishers of disinformation are identified via fact-checking websites. Samples created using those approaches can partly overlap. However, the two approaches differ in terms of validity and comprehensiveness of the identified population. Sampling of alternative media outlets is theory-driven and allows for cross-national comparison. However, researchers face the challenge to identify misinforming content published by alternative media outlets. In contrast, fact-checked content facilitates the identification of a given disinformation population; however, fact-checker often have a publication bias focusing on a small range of (elite) actors or sources (e.g. individual blogs, hyper partisan news outlets, or politicians). In both approaches it is important to describe, compare and, if possible, assign the outlets to already existing categories in order to enable a temporal and spatial comparison. Approaches to identify sources/publishers: Besides the operationalization of specific variables analyzed in the field of disinformation, the sampling procedure presents a crucial element to operationalize disinformation itself. Following the approach of detecting disinformation through its potential sources or publishers (Li, 2020), research analyzes alternative media (Bachl, 2018; Boberg, Quandt, Schatto-Eckrodt, & Frischlich, 2020; Heft et al., 2020) or identifies a various range of actors or domains via fact-checking sites (Allcott & Gentzkow, 2017; Grinberg et al., (2019); Guess, Nyhan & Reifler, 2018). Those two approaches are explained in the following. Alternative media as sources/publishers The following procedure summarizes the approaches used in current research for the identification of relevant alternative media outlets (following Bachl, 2018; Boberg et al., 2020; Heft et al., 2020). Snowball sampling to define the universe of alternative media outlets may consists of the following steps: Sample of outlets identified in previous research Consultation of search engines and news articles Departing from a potential prototype, websites provide information about digital metrics (Alexa.com or Similarweb.com). For example, Similarweb.com shows three relevant lists per outlet: “Top Referring Sites” (which websites are sending traffic to this site), “Also visited websites” (overlap with users of other websites), and “Competitors & Similar Sites” (similarity defined by the company) Definition of alternative media outlets Journalistic outlets (for example, excluding blogs and forums) with current, non-fictional and regular content Self-description of the outlets in a so-called “about us” section or in a mission statement, which underlines the relational perspective of being an alternative to the mainstream media. This description may for example include keywords such as alternative, independent, unbiased, critical or is in line with statements like “presenting the real/true views/facts” or “covering what the mainstream media hides/leaves out”. Use of predefined dimensions and categories of alternative media (Frischlich, Klapproth, & Brinkschulte, 2020; Holt, Ustad Figenschou, & Frischlich, 2019) Sources/publishers via fact-checking sites Following previous research in the U.S., Guess et al. (2018) identified “Fake news domains” (focusing on pro-Trump and pro-Clinton content) which published two or more articles that were coded as “fake news” by fact-checkers (derived from Allcott & Gentzkow, 2017). Grinberg et al. (2019) identified three classes of “fake news sources” differentiated by severity and frequency of false content (see Table 1). These three sources are part of a total of six website labels. The researchers additionally coded the sites into reasonable journalism, low quality journalism, satire and sites that were not applicable. The coders reached a percentual agreement of 60% for the labeling of the six categories, and 80% for the distinction of fake and non-fake categories. Table 1. Three classes of “fake news sources” by Grinberg et al. (2019) Label Specification Identification Definition Black domains Based on previous studies: These domains published at least two articles which were declared as “fake news” by fact-checking sites. Based on preexisting lists constructed by fact-checkers, journalists and academics (Allcott & Gentzkow, 2017; Guess et al., 2018) Almost exclusively fabricated stories Red domains Major or frequent falsehoods that are in line with the site's political agenda. Prejudiced: Site presents falsehoods that focus upon one group with regards to race / religion / ethnicity / sexual orientation. Major or frequent falsehoods with little regard for the truth, but not necessarily to advance a certain political agenda. By the fact-checker snopes.com as sources of questionable claims; then manually differentiated between red and orange domains Falsehoods that clearly reflected a flawed editorial process Orange domains Moderate or occasional falsehoods to advance political agenda. Sensationalism: exaggerations to the extent that the article becomes misleading and inaccurate. Occasionally prejudiced articles: Site at times presents individual articles that contain falsehoods regarding race / religion / ethnicity / sexual orientation Openly states that the site may not be inaccurate, fake news, or cannot be trusted to provide factual news. Moderate or frequent falsehoods with little regard for the truth, but not necessarily to advance a certain political agenda. Conspiratorial: explanations of events that involves unwarranted suspicion of government cover ups or supernatural agents. By the fact-checker snopes.com as sources of questionable claims; then manually differentiated between red and orange domains Negligent and deceptive information but are less systemically flawed Supplementary materials: https://science.sciencemag.org/content/sci/suppl/2019/01/23/363.6425.374.DC1/aau2706_Grinberg_SM.pdf (S5 and S6) Coding scheme and source labels: https://zenodo.org/record/2651401#.XxGtJJgzaUl (LazerLab-twitter-fake-news-replication-2c941b8\domains\domain_coding\data) References Allcott, H., & Gentzkow, M. (2017). Social media and fake news in the 2016 election. Journal of Economic Perspectives, 31(2), 211–236. Bachl, M. (2018). (Alternative) media sources in AfD-centered Facebook discussions. Studies in Communication | Media, 7(2), 256–270. Bakir, V., & McStay, A. (2018). Fake news and the economy of emotions. Digital Journalism, 6(2), 154–175. Boberg, S., Quandt, T., Schatto-Eckrodt, T., & Frischlich, L. (2020, April 6). Pandemic populism: Facebook pages of alternative news media and the corona crisis -- A computational content analysis. Retrieved from http://arxiv.org/pdf/2004.02566v3 Farkas, J., Schou, J., & Neumayer, C. (2018). Cloaked Facebook pages: Exploring fake Islamist propaganda in social media. New Media & Society, 20(5), 1850–1867. Frischlich, L., Klapproth, J., & Brinkschulte, F. (2020). Between mainstream and alternative – Co-orientation in right-wing populist alternative news media. In C. Grimme, M. Preuss, F. W. Takes, & A. Waldherr (Eds.), Lecture Notes in Computer Science. Disinformation in open online media (Vol. 12021, pp. 150–167). Cham: Springer International Publishing. Grinberg, N., Joseph, K., Friedland, L., Swire-Thompson, B., & Lazer, D. (2019). Fake news on Twitter during the 2016 U.S. Presidential election. Science (New York, N.Y.), 363(6425), 374–378. Guess, A., Nagler, J., & Tucker, J. (2019). Less than you think: Prevalence and predictors of fake news dissemination on Facebook. Science Advances, 5(1). https://doi.org/10.1126/sciadv.aau4586 Guess, A., Nyhan, B., & Reifler, J. (2018). Selective exposure to misinformation: Evidence from the consumption of fake news during the 2016 US presidential campaign. European Research Council, 9(3), 1–14. Heft, A., Mayerhöffer, E., Reinhardt, S., & Knüpfer, C. (2020). Beyond Breitbart: Comparing right?wing digital news infrastructures in six Western democracies. Policy & Internet, 12(1), 20–45. Holt, K., Ustad Figenschou, T., & Frischlich, L. (2019). Key dimensions of alternative news media. Digital Journalism, 7(7), 860–869. Nelson, J. L., & Taneja, H. (2018). The small, disloyal fake news audience: The role of audience availability in fake news consumption. New Media & Society, 20(10), 3720–3737