Variety, Velocity, Veracity, and Viability: Evaluating the Contributions of Crowdsourced and Professional Fact-checking

The tension between the increasing need for fact-checking and the limited capacity of fact-check providers inspired several crowdsourced approaches to address this challenge.However, little is known about how effectively crowdsourced fact-checking might perform, and there is no comprehensive framework to evaluate such fact-check providers.We fill this gap by proposing such a framework, using four dimensions (Variety, Velocity, Veracity, and Viability) to assess and compare the contributions of a crowdsourced fact-checking community and professional fact-checking sites. Our analysis shows the different focus these two types of sites have in terms of topic coverage (variety) and demonstrates that while crowdsourced fact-checkers are much faster than professionals (velocity) to answer new requests, these fact-checkers often build on the existing professional knowledge for repeated requests.In addition, our findings indicate that the accuracy of the crowdsourced community (veracity) parallels that of the professional sources; and that the crowdsourced fact-checks are perceived quite close to professionals in terms of objectivity, clarity, and persuasiveness (viability).