The Open UniversitySkip to content

An information-theoretic framework for semantic-multimedia retrieval

Magalhães, João and Rüger, Stefan (2010). An information-theoretic framework for semantic-multimedia retrieval. ACM Transactions on Information Systems (TOIS), 28(4), article no. 19.

Full text available as:
PDF (Accepted Manuscript) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Download (549kB)
DOI (Digital Object Identifier) Link:
Google Scholar: Look up in Google Scholar


This article is set in the context of searching text and image repositories by keyword. We develop a unified probabilistic framework for text, image, and combined text and image retrieval that is based on the detection of keywords (concepts) using automated image annotation technology. Our framework is deeply rooted in information theory and lends itself to use with other media types.

We estimate a statistical model in a multimodal feature space for each possible query keyword. The key element of our framework is to identify feature space transformations that make them comparable in complexity and density. We select the optimal multimodal feature space with a minimum description length criterion from a set of candidate feature spaces that are computed with the average-mutual-information criterion for the text part and hierarchical expectation maximization for the visual part of the data. We evaluate our approach in three retrieval experiments (only text retrieval, only image retrieval, and text combined with image retrieval), verify the framework’s low computational complexity, and compare with existing state-of-the-art ad-hoc models.

Item Type: Journal Item
Copyright Holders: 2010 ACM
ISSN: 1558-2868
Extra Information: This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution.
Keywords: algorithms; measurement; experimentation; indexing; search, multimedia; automated keyword annotation
Academic Unit/School: Faculty of Science, Technology, Engineering and Mathematics (STEM) > Knowledge Media Institute (KMi)
Faculty of Science, Technology, Engineering and Mathematics (STEM)
Item ID: 28193
Depositing User: Stefan Rüger
Date Deposited: 16 Feb 2011 12:18
Last Modified: 12 Jun 2020 02:57
Share this page:


Altmetrics from Altmetric

Citations from Dimensions

Download history for this item

These details should be considered as only a guide to the number of downloads performed manually. Algorithmic methods have been applied in an attempt to remove automated downloads from the displayed statistics but no guarantee can be made as to the accuracy of the figures.

Actions (login may be required)

Policies | Disclaimer

© The Open University   contact the OU