The Open UniversitySkip to content

News Source Credibility in the Eyes of Different Assessors

Mensio, Martino and Alani, Harith (2019). News Source Credibility in the Eyes of Different Assessors. In: Conference for Truth and Trust Online, 4-5 Oct 2019, London, UK, (In Press).

Full text available as:
PDF (Version of Record) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Download (282kB) | Preview
Google Scholar: Look up in Google Scholar


With misinformation being one of the biggest issues of current times, many organisations are emerging to offer verifications of information and assessments of news sources. However, it remains unclear how they relate in terms of coverage, overlap and agreement. In this paper we introduce a comparison of the assessments produced by different organisations, in order to measure their overlap and agreement on news sources. Relying on the general term of credibility, we map each of the different assessments to a unified scale. Then we compare two different levels of credibility assessments (source level and document level) by using the data published by various organisations, including fact-checkers, to see which sources they assess more than others, how much overlap there is between them, and how much agreement there is between their verdicts. Our results show that the overlap between the different origins is generally quite low, meaning that different experts and tools provide evaluations for a rather disjoint set of sources, also when considering fact-checking. For agreement, instead we find that there are origins that agree more than others on the verdicts.

Item Type: Conference or Workshop Item
Project Funding Details:
Funded Project NameProject IDFunding Body
Co-InformNot SetNot Set
Keywords: misinformation; credibility; assessments;comparison
Academic Unit/School: Faculty of Science, Technology, Engineering and Mathematics (STEM) > Knowledge Media Institute (KMi)
Faculty of Science, Technology, Engineering and Mathematics (STEM)
Item ID: 62771
Depositing User: Martino Mensio
Date Deposited: 06 Aug 2019 11:37
Last Modified: 24 Oct 2019 11:25
Share this page:

Download history for this item

These details should be considered as only a guide to the number of downloads performed manually. Algorithmic methods have been applied in an attempt to remove automated downloads from the displayed statistics but no guarantee can be made as to the accuracy of the figures.

Actions (login may be required)

Policies | Disclaimer

© The Open University   contact the OU