The Open UniversitySkip to content
 

Application of aboutness to functional benchmarking in information retrieval

Wong, Kam-Fai; Song, Dawei; Bruza, Peter and Cheng, Chun-Hung (2001). Application of aboutness to functional benchmarking in information retrieval. ACM Transactions on Information Systems (TOIS), 19(4) pp. 337–370.

Full text available as:
[img]
Preview
PDF (Not Set) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Download (287Kb)
DOI (Digital Object Identifier) Link: http://doi.org/10.1145/502795.502796
Google Scholar: Look up in Google Scholar

Abstract

Experimental approaches are widely employed to benchmark the performance of an information retrieval (IR) system. Measurements in terms of recall and precision are computed as performance indicators. Although they are good at assessing the retrieval effectiveness of an IR system, they fail to explore deeper aspects such as its underlying functionality and explain why the system shows such performance. Recently, inductive (i.e., theoretical) evaluation of IR systems has been proposed to circumvent the controversies of the experimental methods. Several studies have adopted the inductive approach, but they mostly focus on theoretical modeling of IR properties by using some metalogic. In this article, we propose to use inductive evaluation for functional benchmarking of IR models as a complement of the traditional experiment-based performance benchmarking. We define a functional benchmark suite in two stages: the evaluation criteria based on the notion of "aboutness," and the formal evaluation methodology using the criteria. The proposed benchmark has been successfully applied to evaluate various well-known classical and logic-based IR models. The functional benchmarking results allow us to compare and analyze the functionality of the different IR models.

Item Type: Journal Article
ISSN: 1558-2868
Extra Information: Link leads to publishing source. Copyright held by ACM Press.
Keywords: aboutness; functional benchmarking; inductive evaluation; logic-based information retrieval
Academic Unit/Department: Faculty of Science, Technology, Engineering and Mathematics (STEM) > Computing and Communications
Faculty of Science, Technology, Engineering and Mathematics (STEM)
Item ID: 2998
Depositing User: Users 12 not found.
Date Deposited: 23 Aug 2006
Last Modified: 03 Aug 2016 20:16
URI: http://oro.open.ac.uk/id/eprint/2998
Share this page:

Altmetrics

Scopus Citations

Download history for this item

These details should be considered as only a guide to the number of downloads performed manually. Algorithmic methods have been applied in an attempt to remove automated downloads from the displayed statistics but no guarantee can be made as to the accuracy of the figures.

▼ Automated document suggestions from open access sources

Actions (login may be required)

Policies | Disclaimer

© The Open University   + 44 (0)870 333 4340   general-enquiries@open.ac.uk