Zhang, Peng; Cervino Beresi, Ulises; Song, Dawei and Hou, Yuexian
(2010).
|
PDF (Version of Record)
- Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Download (214Kb) | Preview |
| URL: | http://www.mansci.uwaterloo.ca/~msmucker/publicati... |
|---|---|
| Google Scholar: | Look up in Google Scholar |
Abstract
Conventional information retrieval (IR) evaluation relies on static relevance judgements in test collections. These, however, are insufficient for the evaluation of interactive IR (IIR) systems. When users browse search results, their decisions on whether to keep a document may be infuenced by several factors including previously seen documents. This makes user-centred relevance judgements not only dynamic but also dependent on previous judgements. In this paper, we propose to use a probabilistic automaton (PA) to model the dynamics of users' relevance judgements. Based on the initial judgement data that can be collected in a proposed user study, the estimated PA can further simulate more dynamic relevance judgements, which are of potential usefulness for the evaluation of IIR systems.
| Item Type: | Conference Item |
|---|---|
| Copyright Holders: | 2010 The Authors |
| Keywords: | interactive IR; dynamic relevance judgement; probabilistic automaton; simulation |
| Academic Unit/Department: | Faculty of Science, Technology, Engineering and Mathematics (STEM) > Computing and Communications Faculty of Science, Technology, Engineering and Mathematics (STEM) |
| Related URLs: | |
| Item ID: | 35133 |
| Depositing User: | Dawei Song |
| Date Deposited: | 31 Oct 2012 08:56 |
| Last Modified: | 04 Aug 2016 06:56 |
| URI: | http://oro.open.ac.uk/id/eprint/35133 |
| Share this page: |     |
Download history for this item
These details should be considered as only a guide to the number of downloads performed manually. Algorithmic methods have been applied in an attempt to remove automated downloads from the displayed statistics but no guarantee can be made as to the accuracy of the figures.