Ranking for Learning: Studying Users’ Perceptions of Relevance, Understandability, and Engagement

Ghafourian, Yasin; Hanbury, Allan and Knoth, Petr (2023). Ranking for Learning: Studying Users’ Perceptions of Relevance, Understandability, and Engagement. In: Linking Theory and Practice of Digital Libraries. TPDL 2023. (Alonso, Omar; Cousijn, Helena; Silvello, Gianmaria; Marrero, Mónica; Teixeira Lopes, Carla and Marchesin, Stefano eds.), Lecture Notes in Computer Science (LNCS) volume 14241, Springer, Cham, pp. 284–291.

DOI: https://doi.org/10.1007/978-3-031-43849-3_25

Abstract

General-purpose search engines are frequently used to retrieve content for learning. However, their ranking strategies are typically optimised for relevance, which means that they do not take into account other criteria important in the learning context, such as the understandability and the degree of engagement of the retrieved resources. We have conducted a user study to assess the extent to which ranking algorithms used by a popular search engine satisfy the expectations of users who are learning by searching. We study the relationships between users’ perceptions of topical relevance, engagement, and understandability for retrieved documents with respect to their ranks. While we observe that the perceived user-assigned rank is strongly associated with all dimensions of relevance under study, specifically engagement (p=0.89), understandability (p=0.58) and topical relevance (p=0.88), the relationship between SERP ranks and user-assigned ranks appears unstable, indicating that learners are not necessarily always served well by general-purpose search engines.

Viewing alternatives

Metrics

Item Actions

Export

You can export this page using these formats Digital Object Identifier - DOI