The Open UniversitySkip to content

A Probabilistic Automaton for the Dynamic Relevance Judgement of Users

Zhang, Peng; Cervino Beresi, Ulises; Song, Dawei and Hou, Yuexian (2010). A Probabilistic Automaton for the Dynamic Relevance Judgement of Users. In: ACM SIGIR2010 Workshop on Simulation of Interaction (SemInt), 19-23 Jul 2010, Geneva, Switzerland.

Full text available as:
PDF (Version of Record) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Download (219kB) | Preview
Google Scholar: Look up in Google Scholar


Conventional information retrieval (IR) evaluation relies on static relevance judgements in test collections. These, however, are insufficient for the evaluation of interactive IR (IIR) systems. When users browse search results, their decisions on whether to keep a document may be infuenced by several factors including previously seen documents. This makes user-centred relevance judgements not only dynamic but also dependent on previous judgements. In this paper, we propose to use a probabilistic automaton (PA) to model the dynamics of users' relevance judgements. Based on the initial judgement data that can be collected in a proposed user study, the estimated PA can further simulate more dynamic relevance judgements, which are of potential usefulness for the evaluation of IIR systems.

Item Type: Conference or Workshop Item
Copyright Holders: 2010 The Authors
Keywords: interactive IR; dynamic relevance judgement; probabilistic automaton; simulation
Academic Unit/School: Faculty of Science, Technology, Engineering and Mathematics (STEM) > Computing and Communications
Faculty of Science, Technology, Engineering and Mathematics (STEM)
Related URLs:
Item ID: 35133
Depositing User: Dawei Song
Date Deposited: 31 Oct 2012 08:56
Last Modified: 07 Dec 2018 19:34
Share this page:

Download history for this item

These details should be considered as only a guide to the number of downloads performed manually. Algorithmic methods have been applied in an attempt to remove automated downloads from the displayed statistics but no guarantee can be made as to the accuracy of the figures.

Actions (login may be required)

Policies | Disclaimer

© The Open University   contact the OU