Zhang, Peng; Song, Dawei; Wang, Jun; Zhao, Xiaozhao and Hou, Yuexian
On modeling rank-independent risk in estimating probability of Relevance.
In: The 7th Asia Information Retrieval Societies Conference (AIRS2011), 18-20 December 2011, Dubai, United Arab Emirates.
Full text available as:
Estimating the probability of relevance for a document is fundamental in information retrieval. From a theoretical point of view, risk exists in the estimation process, in the sense that the estimated probabilities may not be the actual ones precisely. The estimation risk is often considered to be dependent on the rank. For example, the probability ranking principle assumes that ranking documents in the order of decreasing probability of relevance can optimize the rank effectiveness. This implies that a precise estimation can yield an optimal rank. However, an optimal (or even ideal) rank does not always guarantee that the estimated probabilities are precise. This means that part of the estimation risk is rank-independent. It imposes practical risks in the applications, such as pseudo relevance feedback, where different estimated probabilities of relevance in the first-round retrieval will make a difference even when two ranks are identical. In this paper, we will explore the effect and the modeling of such rank-independent risk. A risk management method is proposed to adaptively adjust the rank-independent risk. Experimental results on several TREC collections demonstrate the effectiveness of the proposed models for both pseudo-relevance feedback and relevance feedback.
|External Project Funding Details:
|Funded Project Name||Project ID||Funding Body|
|Not Set||Not Set||UK’s EPSRC (EP/F014708/2)|
|Not Set||Not Set||China’s NSFC (61070044)|
|Not Set||Not Set||EU’s Marie Curie Actions-IRSES (247590)|
||Published in: M.V.M. Salem et al. (Eds.): AIRS 2011, LNCS 7097, pp. 13–24, 2011
||probability of relevance; estimation; risk management; ranking-independent risk; language modeling
||Mathematics, Computing and Technology > Computing & Communications
||07 Aug 2012 14:58
||26 Oct 2012 01:38
Actions (login may be required)