Detecting Factual Reporting News Sources and Political Bias: A New Approach to Media Information Analysis

The developed technology aims to estimate the political bias and factual reliability of media outlets through how these outlets link to one another over an extended period, or else their "longitudinal web interactions". This approach differs from existing models that rely on social media metadata or content analysis, where manual evaluation is critical and time-consuming and as of today off-shelf large language models (LLMs) still fail to provide reliable and verified results.

To prove their hypothesis, the team exploits large-scale graphs of news sources connected by hyperlinks, forming a network of media interactions. They showed that factual reporting and political bias could be predicted by analyzing how media sources interact, independently of language or content, combining past and future interactions and an initial set of ground-truth labels using reinforcement-learning algorithms.

These results mark an advance in automating media bias profiling, which has traditionally depended on manual verification by fact-checkers. Beyond identifying factual reporting and political bias, this approach provides a broader tool for understanding the dynamic media landscape, paving the way for future research into shifts in political bias and other media characteristics.

The research team has also released the largest annotated dataset of media sources categorized by factual reporting and political bias. This dataset includes over 17K media outlets, providing a resource for future studies on media profiling and bias detection.

This research was funded by the EU Horizon 2020 program as part of the CRiTERIA project, aimed at bridging the gap between (combined) evidence for events, trends, biases, risks, threats etc., and threat analysis results in the context of migration.

The related paper is available at https://publications.idiap.ch/publications/show/5368