The Open UniversitySkip to content

Reflections on five years of evaluating semantic search systems

Uren, Victoria; Sabou, Marta; Motta, Enrico; Fernandez, Miriam; Lopez, Vanessa and Lei, Yuangui (2010). Reflections on five years of evaluating semantic search systems. International Journal of Metadata, Semantics and Ontologies (IJMSO), 5(2) pp. 87–98.

DOI (Digital Object Identifier) Link:
Google Scholar: Look up in Google Scholar


Evaluations of semantic search systems are generally small scale and ad hoc due to the lack of appropriate resources such as test collections, agreed performance criteria and independent judgements of performance. By analysing our work in building and evaluating semantic tools over the last five years, we conclude that the growth of the semantic web led to an improvement in the available resources and the consequent robustness of performance assessments. We propose two directions for continuing evaluation work: the development of extensible evaluation benchmarks and the use of logging parameters for evaluating individual components of search systems.

Item Type: Journal Article
ISSN: 1744-263X
Keywords: semantic search; evaluation benchmarks; semantic web; metadata; ontologies; logging parameters; semantics
Academic Unit/Department: Faculty of Science, Technology, Engineering and Mathematics (STEM) > Knowledge Media Institute (KMi)
Faculty of Science, Technology, Engineering and Mathematics (STEM)
Interdisciplinary Research Centre: Centre for Research in Computing (CRC)
Centre for Policing Research and Learning (CPRL)
Item ID: 23446
Depositing User: Kay Dave
Date Deposited: 12 Oct 2010 14:58
Last Modified: 17 Nov 2016 14:50
Share this page:


Scopus Citations

▼ Automated document suggestions from open access sources

Actions (login may be required)

Policies | Disclaimer

© The Open University   + 44 (0)870 333 4340