Evaluation of untrustworthy journals: Transition from formal criteria to a complex view

Not all the journals included in credible indices meet the ethical rules of COPE, DOAJ, OASPA, and WAME, and there may also be trustworthy journals excluded from these indices, which means they cannot be used as whitelists for trustworthy journals. Equally, the many methods suggested to determine trustworthiness are not reliable because they include questionable criteria. The question arises whether valid criteria for identifying an untrustworthy journal can be determined and whether other assessment procedures are necessary. Since 2017, the Masaryk University Campus Library has been developing a suitable evaluation method for journals. A list of 19 criteria based on those originally suggested by Beall, COPE, DOAJ, OASPA, and WAME were reduced to 10 objectively verifiable criteria following two workshops with librarians. An evaluation of 259 biomedical journals using both the list of 19 and then 10 criteria revealed that 74 journals may have been incorrectly assessed as untrustworthy using the longer list. The most common reason for failure to comply was in the provision of sufficient editorial information and declaration of article processing charges. However, our investigation revealed that no criteria can reliably identify predatory journals. Therefore, a complex evaluation is needed combining objectively verifiable criteria with analysis of a journal's content and knowledge of the journal's background