The List of Questionable Journals: Is It Trustworthy?

Abstract The list of questionable journals serves to stratify academic journals, distinguishing between good and bad journals, thereby providing a basis for academic information filtering and research evaluation. The formulation of such lists has been a topic of controversy in the literature, with limited attention paid to whether the criteria used to stratify journals in these lists can be trusted. In this study, we take the example of the warning journal list published by the National Science Library of the Chinese Academy of Sciences and categorize the journals based on their warning levels and warning years. Then, we compare them to determine if there are differences in key academic indicators, thereby analyzing whether the journal lists effectively identify academic journals with quality risks. Our study found that the patterns of differences in key academic indicators among different journal stratums were not consistent, and the same inconsistency was observed during the mobility between stratums. Furthermore, citation metrics such as Journal Impact Factor and Journal Citation Indicator may not have been considered as criteria for stratifying journals in the warning journal list, but this lack of detailed explanation from the creators. This indicates that the list of questionable journals needs to be more scientifically evaluated, and there is a greater need for sharing detailed criteria and data. Our study also provides recommendations for institutions in the future when formulating lists of questionable journals.