The Open UniversitySkip to content

AutoEval: an evaluation methodology for evaluating query suggestions using query logs

Albakour, M-Dyaa; Kruschwitz, Udo; Nanas, Nikolaos; Kim, Yunhyong; Song, Dawei; Fasli, Maria and De Roeck, Anne (2011). AutoEval: an evaluation methodology for evaluating query suggestions using query logs. In: 33rd European Conference on Information Retrieval (ECIR2011), 19-21 Apr 2011, Dublin, Ireland, Springer-Verlag, pp. 605–610.

Full text available as:
PDF (Accepted Manuscript) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Download (212kB) | Preview
DOI (Digital Object Identifier) Link:
Google Scholar: Look up in Google Scholar


User evaluations of search engines are expensive and not easy to replicate. The problem is even more pronounced when assessing adaptive search systems, for example system-generated query modification suggestions that can be derived from past user interactions with a search engine. Automatically predicting the performance of different modification suggestion models before getting the users involved is therefore highly desirable. AutoEval is an evaluation methodology that assesses the quality of query modifications generated by a model using the query logs of past user interactions with the system. We present experimental results of applying this methodology to different adaptive algorithms which suggest that the predicted quality of different algorithms is in line with user assessments. This makes AutoEval a suitable evaluation framework for adaptive interactive search engines.

Item Type: Conference or Workshop Item
Copyright Holders: 2011 Springer-Verlag
Extra Information: Advances in Information Retrieval
33rd European Conference on IR Research, ECIR 2011
Dublin, Ireland, April 18-21, 2011
edited by Paul Clough, Colum Foley, Cathal Gurrin, Gareth J.F. Jones, Wessel Kraaij, Hyowon Lee, Vanessa Murdoch
Berlin : Springer-Verlag, 2011
Lecture Notes in Computer Science, 6611
ISBN 978-3-642-20160-8
pp. 605-610
Academic Unit/School: Faculty of Science, Technology, Engineering and Mathematics (STEM) > Computing and Communications
Faculty of Science, Technology, Engineering and Mathematics (STEM)
Research Group: Centre for Research in Computing (CRC)
Related URLs:
Item ID: 34686
Depositing User: Dawei Song
Date Deposited: 18 Oct 2012 11:04
Last Modified: 07 Dec 2018 23:55
Share this page:


Altmetrics from Altmetric

Citations from Dimensions

Download history for this item

These details should be considered as only a guide to the number of downloads performed manually. Algorithmic methods have been applied in an attempt to remove automated downloads from the displayed statistics but no guarantee can be made as to the accuracy of the figures.

Actions (login may be required)

Policies | Disclaimer

© The Open University   contact the OU