The Open UniversitySkip to content
 

A Probabilistic Automaton for the Dynamic Relevance Judgement of Users

Zhang, Peng; Cervino Beresi, Ulises; Song, Dawei and Hou, Yuexian (2010). A Probabilistic Automaton for the Dynamic Relevance Judgement of Users. In: ACM SIGIR2010 Workshop on Simulation of Interaction (SemInt), 19-23 July 2010, Geneva, Switzerland.

Full text available as:
[img]
Preview
PDF (Version of Record) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Download (214Kb) | Preview
URL: http://www.mansci.uwaterloo.ca/~msmucker/publicati...
Google Scholar: Look up in Google Scholar

Abstract

Conventional information retrieval (IR) evaluation relies on static relevance judgements in test collections. These, however, are insufficient for the evaluation of interactive IR (IIR) systems. When users browse search results, their decisions on whether to keep a document may be infuenced by several factors including previously seen documents. This makes user-centred relevance judgements not only dynamic but also dependent on previous judgements. In this paper, we propose to use a probabilistic automaton (PA) to model the dynamics of users' relevance judgements. Based on the initial judgement data that can be collected in a proposed user study, the estimated PA can further simulate more dynamic relevance judgements, which are of potential usefulness for the evaluation of IIR systems.

Item Type: Conference Item
Copyright Holders: 2010 The Authors
Keywords: interactive IR; dynamic relevance judgement; probabilistic automaton; simulation
Academic Unit/Department: Mathematics, Computing and Technology > Computing & Communications
Related URLs:
Item ID: 35133
Depositing User: Dawei Song
Date Deposited: 31 Oct 2012 08:56
Last Modified: 16 Nov 2012 04:17
URI: http://oro.open.ac.uk/id/eprint/35133
Share this page:

Actions (login may be required)

View Item
Report issue / request change

Policies | Disclaimer

© The Open University   + 44 (0)870 333 4340   general-enquiries@open.ac.uk