The Open UniversitySkip to content
 

Salience in the generation of multimodal referring acts

Piwek, Paul (2009). Salience in the generation of multimodal referring acts. In: Proceedings of the 2009 international conference on Multimodal interfaces - ICMI-MLMI '09, p. 207.

Full text available as:
[img]
Preview
PDF (Accepted Manuscript) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Download (1MB) | Preview
DOI (Digital Object Identifier) Link: https://doi.org/10.1145/1647314.1647351
Google Scholar: Look up in Google Scholar

Abstract

Pointing combined with verbal referring is one of the most paradigmatic human multimodal behaviours. The aim of this paper is foundational: to uncover the central notions that are required for a computational model of multimodal referring acts that include a pointing gesture. The paper draws on existing work on the generation of referring expressions and shows that in order to extend that work with pointing, the notion of salience needs to play a pivotal role. The paper starts by investigating the role of salience in the generation of referring expressions and introduces a distinction between two opposing approaches: salience-first and salience-last accounts. The paper then argues that these differ not only in computational efficiency, as has been pointed out previously, but also lead to incompatible empirical predictions. The second half of the paper shows how a salience-first account nicely meshes with a range of existing empirical findings on multimodal reference. A novel account of the circumstances under which speakers choose to point is proposed that directly links salience with pointing. Finally, this account is placed within a multi-dimensional model of salience for multimodal reference.

Item Type: Conference or Workshop Item
Extra Information: SESSION: Gaze, gesture, and reference (Oral) table of contents
Pages 207-210
ISBN:978-1-60558-772-1
Academic Unit/School: Faculty of Science, Technology, Engineering and Mathematics (STEM) > Computing and Communications
Faculty of Science, Technology, Engineering and Mathematics (STEM)
Research Group: Centre for Research in Computing (CRC)
Item ID: 19167
Depositing User: Paul Piwek
Date Deposited: 09 Dec 2009 09:59
Last Modified: 08 Dec 2018 03:38
URI: http://oro.open.ac.uk/id/eprint/19167
Share this page:

Metrics

Altmetrics from Altmetric

Citations from Dimensions

Download history for this item

These details should be considered as only a guide to the number of downloads performed manually. Algorithmic methods have been applied in an attempt to remove automated downloads from the displayed statistics but no guarantee can be made as to the accuracy of the figures.

Actions (login may be required)

Policies | Disclaimer

© The Open University   contact the OU