Pictures in Your Mind: Using Interactive Gesture-Controlled Reliefs to Explore Art

Reichinger, Andreas; Garcia Carrizosa, Helena; Wood, Joanna; Schröder, Svenja; Löw, Christian; Luidolt, Laura Rosalia; Schimkowitsch, Maria; Fuhrmann, Anton; Maierhofer, Stefan and Purgathofer, Werner (2018). Pictures in Your Mind: Using Interactive Gesture-Controlled Reliefs to Explore Art. ACM Transactions on Accessible Computing, 11(1), article no. 2.

DOI: https://doi.org/10.1145/3155286

Abstract

Tactile reliefs offer many benefits over the more classic raised line drawings or tactile diagrams, as depth, 3D shape, and surface textures are directly perceivable. Although often created for blind and visually impaired (BVI) people, a wider range of people may benefit from such multimodal material. However, some reliefs are still difficult to understand without proper guidance or accompanying verbal descriptions, hindering autonomous exploration.

In this work, we present a gesture-controlled interactive audio guide (IAG) based on recent low-cost depth cameras that can be operated directly with the hands on relief surfaces during tactile exploration. The interactively explorable, location-dependent verbal and captioned descriptions promise rapid tactile accessibility to 2.5D spatial information in a home or education setting, to online resources, or as a kiosk installation at public places.

We present a working prototype, discuss design decisions, and present the results of two evaluation studies: the first with 13 BVI test users and the second follow-up study with 14 test users across a wide range of people with differences and difficulties associated with perception, memory, cognition, and communication. The participant-led research method of this latter study prompted new, significant and innovative developments.

Viewing alternatives

Metrics

Item Actions

Export

You can export this page using these formats Digital Object Identifier - DOI

About

• Item ORO ID
• 54257
• Item Type
• Journal Item
• ISSN
• 1936-7228
• Project Funding Details

• Funded Project Name	Project ID	Funding Body
  Deep Pictures	P24352-N23	Austrian Science Fund (FWF)
  Erasmus+ project AMBAVis	2014-1-AT01-KA204-001014	European Commission
  ARCHES	693229	EC (European Commission): FP(inc.Horizon2020, H2020, ERC)

• Keywords
• Blind; low vision; learning disability; cognitive disability; auditory interface design for all; gestures; multimodal interaction
• Academic Unit or School
• Faculty of Wellbeing, Education and Language Studies (WELS) > Education, Childhood, Youth and Sport > Education
  Faculty of Wellbeing, Education and Language Studies (WELS) > Education, Childhood, Youth and Sport
  Faculty of Wellbeing, Education and Language Studies (WELS)
• Copyright Holders
• © 2018 The Authors
• SWORD Depositor
• Jisc Publications-Router
• Depositing User
• Jisc Publications-Router