Little, Suzanne; Llorente, Ainhoa and Rüger, Stefan
(2010). An overview of evaluation campaigns in multimedia retrieval.
In: Müller, H.; Clough, P.; Deselaers, T. and Caputo, B. eds.
ImageCLEF: Experimental Evaluation in Visual Information Retrieval.
Berlin/Heidelberg: Springer-Verlag, pp. 507–522.
This chapter presents an academic and research perspective on the impact and importance of ImageCLEF and similar evaluation workshops in multimedia information retrieval (MIR). Three main themes are examined: the position of ImageCLEF compared with other evaluation conferences; general views on the usefulness of evaluation conferences and possible alternatives, and the impact and real–world meaning of evaluation metrics used within ImageCLEF. We examine the value of ImageCLEF, and related evaluation conferences, for the multimedia IR researcher as providing not only a forum for assessing and comparing outcomes but also serving to promote research aims, provide practical guidance (e.g. standard data sets) and inspire research directions.
Actions (login may be required)