The Open UniversitySkip to content

On modeling rank-independent risk in estimating probability of Relevance

Zhang, Peng; Song, Dawei; Wang, Jun; Zhao, Xiaozhao and Hou, Yuexian (2011). On modeling rank-independent risk in estimating probability of Relevance. In: The 7th Asia Information Retrieval Societies Conference (AIRS2011), 18-20 Dec 2011, Dubai, United Arab Emirates.

Full text available as:
Full text not publicly available (Version of Record)
Due to publisher licensing restrictions, this file is not available for public download
Click here to request a copy from the OU Author.
DOI (Digital Object Identifier) Link:
Google Scholar: Look up in Google Scholar


Estimating the probability of relevance for a document is fundamental in information retrieval. From a theoretical point of view, risk exists in the estimation process, in the sense that the estimated probabilities may not be the actual ones precisely. The estimation risk is often considered to be dependent on the rank. For example, the probability ranking principle assumes that ranking documents in the order of decreasing probability of relevance can optimize the rank effectiveness. This implies that a precise estimation can yield an optimal rank. However, an optimal (or even ideal) rank does not always guarantee that the estimated probabilities are precise. This means that part of the estimation risk is rank-independent. It imposes practical risks in the applications, such as pseudo relevance feedback, where different estimated probabilities of relevance in the first-round retrieval will make a difference even when two ranks are identical. In this paper, we will explore the effect and the modeling of such rank-independent risk. A risk management method is proposed to adaptively adjust the rank-independent risk. Experimental results on several TREC collections demonstrate the effectiveness of the proposed models for both pseudo-relevance feedback and relevance feedback.

Item Type: Conference or Workshop Item
Copyright Holders: 2011 Springer-Verlag
Project Funding Details:
Funded Project NameProject IDFunding Body
Not SetNot SetUK’s EPSRC (EP/F014708/2)
Not SetNot SetChina’s NSFC (61070044)
Not SetNot SetEU’s Marie Curie Actions-IRSES (247590)
Extra Information: Published in: M.V.M. Salem et al. (Eds.): AIRS 2011, LNCS 7097, pp. 13–24, 2011
Keywords: probability of relevance; estimation; risk management; ranking-independent risk; language modeling
Academic Unit/School: Faculty of Science, Technology, Engineering and Mathematics (STEM) > Computing and Communications
Faculty of Science, Technology, Engineering and Mathematics (STEM)
Related URLs:
Item ID: 34143
Depositing User: Catherine McNulty
Date Deposited: 07 Aug 2012 14:58
Last Modified: 08 Dec 2018 18:35
Share this page:


Altmetrics from Altmetric

Citations from Dimensions

Actions (login may be required)

Policies | Disclaimer

© The Open University   contact the OU