The Open UniversitySkip to content
 

Investigating Order Effects in Multidimensional Relevance Judgment using Query Logs

Uprety, Sagar and Song, Dawei (2018). Investigating Order Effects in Multidimensional Relevance Judgment using Query Logs. In: ICTIR '18: Proceedings of the 2018 ACM SIGIR International Conference on Theory of Information Retrieval, ACM Press, New York pp. 191–194.

DOI (Digital Object Identifier) Link: https://doi.org/10.1145/3234944.3234972
Google Scholar: Look up in Google Scholar

Abstract

There is a growing body of research which has investigated relevance judgment in IR being influenced by multiple factors or dimensions. At the same time, the Order Effects in sequential decision making have been quantitatively detected and studied in Mathematical Psychology. Combining the two phenomena, there have been some user studies carried out which investigate the Order Effects and thus incompatibility in different dimensions of relevance. In this work, we propose a methodology for carrying out such an investigation in large scale and real world data using query logs of a web search engine, and device a test to detect the presence of an irrational user behavior in relevance judgment of documents. We further validate this behavior through a Quantum Cognitive explanation of the Order and Context effects.

Item Type: Conference or Workshop Item
Copyright Holders: 2018 Association for Computing Machinery
ISBN: 1-4503-5656-7, 978-1-4503-5656-5
Extra Information: originally presented at the 2018 ACM SIGIR International Conference on Theory of Information Retrieval, Tianjin, China, 14-17 Sep 2018.
Keywords: Multidimensional Relevance; Quantum Cognition
Academic Unit/School: Faculty of Science, Technology, Engineering and Mathematics (STEM) > Computing and Communications
Faculty of Science, Technology, Engineering and Mathematics (STEM)
Item ID: 56725
SWORD Depositor: Jisc Publications-Router
Depositing User: Jisc Publications-Router
Date Deposited: 25 Sep 2018 09:40
Last Modified: 07 Dec 2018 11:12
URI: http://oro.open.ac.uk/id/eprint/56725
Share this page:

Metrics

Altmetrics from Altmetric

Citations from Dimensions

Actions (login may be required)

Policies | Disclaimer

© The Open University   contact the OU