Copy the page URI to the clipboard
Zhang, Peng; Cervino Beresi, Ulises; Song, Dawei and Hou, Yuexian
(2010).
URL: http://www.mansci.uwaterloo.ca/~msmucker/publicati...
Abstract
Conventional information retrieval (IR) evaluation relies on static relevance judgements in test collections. These, however, are insufficient for the evaluation of interactive IR (IIR) systems. When users browse search results, their decisions on whether to keep a document may be infuenced by several factors including previously seen documents. This makes user-centred relevance judgements not only dynamic but also dependent on previous judgements. In this paper, we propose to use a probabilistic automaton (PA) to model the dynamics of users' relevance judgements. Based on the initial judgement data that can be collected in a proposed user study, the estimated PA can further simulate more dynamic relevance judgements, which are of potential usefulness for the evaluation of IIR systems.
Viewing alternatives
Download history
Item Actions
Export
About
- Item ORO ID
- 35133
- Item Type
- Conference or Workshop Item
- Keywords
- interactive IR; dynamic relevance judgement; probabilistic automaton; simulation
- Academic Unit or School
-
Faculty of Science, Technology, Engineering and Mathematics (STEM) > Computing and Communications
Faculty of Science, Technology, Engineering and Mathematics (STEM) - Copyright Holders
- © 2010 The Authors
- Related URLs
- Depositing User
- Dawei Song