The Open UniversitySkip to content
 

Salience in the generation of multimodal referring acts

Piwek, Paul (2009). Salience in the generation of multimodal referring acts. In: Proceedings of the 2009 international conference on Multimodal interfaces - ICMI-MLMI '09, p. 207.

DOI (Digital Object Identifier) Link: http://doi.org/10.1145/1647314.1647351
Google Scholar: Look up in Google Scholar

Abstract

Pointing combined with verbal referring is one of the most paradigmatic human multimodal behaviours. The aim of this paper is foundational: to uncover the central notions that are required for a computational model of multimodal referring acts that include a pointing gesture. The paper draws on existing work on the generation of referring expressions and shows that in order to extend that work with pointing, the notion of salience needs to play a pivotal role. The paper starts by investigating the role of salience in the generation of referring expressions and introduces a distinction between two opposing approaches: salience-first and salience-last accounts. The paper then argues that these differ not only in computational efficiency, as has been pointed out previously, but also lead to incompatible empirical predictions. The second half of the paper shows how a salience-first account nicely meshes with a range of existing empirical findings on multimodal reference. A novel account of the circumstances under which speakers choose to point is proposed that directly links salience with pointing. Finally, this account is placed within a multi-dimensional model of salience for multimodal reference.

Item Type: Conference Item
Extra Information: SESSION: Gaze, gesture, and reference (Oral) table of contents
Pages 207-210
ISBN:978-1-60558-772-1
Academic Unit/Department: Faculty of Science, Technology, Engineering and Mathematics (STEM) > Computing and Communications
Faculty of Science, Technology, Engineering and Mathematics (STEM)
Interdisciplinary Research Centre: Centre for Research in Computing (CRC)
Item ID: 19167
Depositing User: Paul Piwek
Date Deposited: 09 Dec 2009 09:59
Last Modified: 04 Oct 2016 10:30
URI: http://oro.open.ac.uk/id/eprint/19167
Share this page:

Altmetrics

Scopus Citations

▼ Automated document suggestions from open access sources

Actions (login may be required)

Policies | Disclaimer

© The Open University   + 44 (0)870 333 4340   general-enquiries@open.ac.uk