The Open UniversitySkip to content
 

On modeling rank-independent risk in estimating probability of Relevance

Zhang, Peng; Song, Dawei; Wang, Jun; Zhao, Xiaozhao and Hou, Yuexian (2011). On modeling rank-independent risk in estimating probability of Relevance. In: The 7th Asia Information Retrieval Societies Conference (AIRS2011), 18-20 December 2011, Dubai, United Arab Emirates.

Full text available as:
Full text not publicly available
Due to copyright restrictions, this file is not available for public download
Click here to request a copy from the OU Author.
URL: http://www.uowdubai.ac.ae/airs2011/
DOI (Digital Object Identifier) Link: http://dx.doi.org/10.1007/978-3-642-25631-8_2
Google Scholar: Look up in Google Scholar

Abstract

Estimating the probability of relevance for a document is fundamental in information retrieval. From a theoretical point of view, risk exists in the estimation process, in the sense that the estimated probabilities may not be the actual ones precisely. The estimation risk is often considered to be dependent on the rank. For example, the probability ranking principle assumes that ranking documents in the order of decreasing probability of relevance can optimize the rank effectiveness. This implies that a precise estimation can yield an optimal rank. However, an optimal (or even ideal) rank does not always guarantee that the estimated probabilities are precise. This means that part of the estimation risk is rank-independent. It imposes practical risks in the applications, such as pseudo relevance feedback, where different estimated probabilities of relevance in the first-round retrieval will make a difference even when two ranks are identical. In this paper, we will explore the effect and the modeling of such rank-independent risk. A risk management method is proposed to adaptively adjust the rank-independent risk. Experimental results on several TREC collections demonstrate the effectiveness of the proposed models for both pseudo-relevance feedback and relevance feedback.

Item Type: Conference Item
Copyright Holders: 2011 Springer-Verlag
Project Funding Details:
Funded Project NameProject IDFunding Body
Not SetNot SetUK’s EPSRC (EP/F014708/2)
Not SetNot SetChina’s NSFC (61070044)
Not SetNot SetEU’s Marie Curie Actions-IRSES (247590)
Extra Information: Published in: M.V.M. Salem et al. (Eds.): AIRS 2011, LNCS 7097, pp. 13–24, 2011
Keywords: probability of relevance; estimation; risk management; ranking-independent risk; language modeling
Academic Unit/Department: Mathematics, Computing and Technology > Computing & Communications
Related URLs:
Item ID: 34143
Depositing User: Catherine McNulty
Date Deposited: 07 Aug 2012 14:58
Last Modified: 26 Oct 2012 01:38
URI: http://oro.open.ac.uk/id/eprint/34143
Share this page:

Altmetrics

Scopus Citations

Actions (login may be required)

View Item
Report issue / request change

Policies | Disclaimer

© The Open University   + 44 (0)870 333 4340   general-enquiries@open.ac.uk