News Source Credibility in the Eyes of Different Assessors

Mensio, Martino and Alani, Harith (2019). News Source Credibility in the Eyes of Different Assessors. In: Proceedings of the Conference for Truth and Trust Online 2019.

URL: https://truthandtrustonline.com/wp-content/uploads...

Abstract

With misinformation being one of the biggest issues of current times, many organisations are emerging to offer verifications of information and assessments of news sources. However, it remains unclear how they relate in terms of coverage, overlap and agreement. In this paper we introduce a comparison of the assessments produced by different organisations, in order to measure their overlap and agreement on news sources. Relying on the general term of credibility, we map each of the different assessments to a unified scale. Then we compare two different levels of credibility assessments (source level and document level) by using the data published by various organisations, including fact-checkers, to see which sources they assess more than others, how much overlap there is between them, and how much agreement there is between their verdicts. Our results show that the overlap between the different origins is generally quite low, meaning that different experts and tools provide evaluations for a rather disjoint set of sources, also when considering fact-checking. For agreement, instead we find that there are origins that agree more than others on the verdicts.

Viewing alternatives

Item Actions

Export

You can export this page using these formats