The Open UniversitySkip to content
 

Reflections on five years of evaluating semantic search systems

Uren, Victoria; Sabou, Marta; Motta, Enrico; Fernandez, Miriam; Lopez, Vanessa and Lei, Yuangui (2010). Reflections on five years of evaluating semantic search systems. International Journal of Metadata, Semantics and Ontologies (IJMSO), 5(2) pp. 87–98.

DOI (Digital Object Identifier) Link: http://dx.doi.org/10.1504/IJMSO.2010.033280
Google Scholar: Look up in Google Scholar

Abstract

Evaluations of semantic search systems are generally small scale and ad hoc due to the lack of appropriate resources such as test collections, agreed performance criteria and independent judgements of performance. By analysing our work in building and evaluating semantic tools over the last five years, we conclude that the growth of the semantic web led to an improvement in the available resources and the consequent robustness of performance assessments. We propose two directions for continuing evaluation work: the development of extensible evaluation benchmarks and the use of logging parameters for evaluating individual components of search systems.

Item Type: Journal Article
ISSN: 1744-263X
Keywords: semantic search; evaluation benchmarks; semantic web; metadata; ontologies; logging parameters; semantics
Academic Unit/Department: Knowledge Media Institute
Interdisciplinary Research Centre: Centre for Research in Computing (CRC)
Item ID: 23446
Depositing User: Kay Dave
Date Deposited: 12 Oct 2010 14:58
Last Modified: 25 Oct 2012 14:42
URI: http://oro.open.ac.uk/id/eprint/23446
Share this page:

Actions (login may be required)

View Item
Report issue / request change

Policies | Disclaimer

© The Open University   + 44 (0)870 333 4340   general-enquiries@open.ac.uk