Towards Adaptive Multi-modal Augmentative and Alternative Communication for Children with CP

Zisman, Andrea; Katz, Dmitri; Bennasar, Mohamed; Alrimawi, Faeq; Price, Blaine and Johnston, Anthony (2024). Towards Adaptive Multi-modal Augmentative and Alternative Communication for Children with CP. In: Computers Helping People with Special Needs. 19th International Conference, ICCHP 2024, 8-12 Jul 2024, Linz, Austria, pp. 159–167.

DOI: https://doi.org/10.1007/978-3-031-62849-8_20

Abstract

Effective communication can pose significant challenges for non-verbal children with Cerebral Palsy (CP). Augmentative and Alternative Communication (AAC) systems help many but can fail to meet the needs of some users. This research proposes a hybrid adaptive approach, utilizing sensors and machine learning (ML) algorithms to create a personalized mobile communication system for those whose abilities are ill-suited to existing approaches. The system aims to tailor to individual abilities, reducing the need for users to adapt to system requirements. Online surveys gathered data on gestures, actions, and sounds used by non-verbal CP children, informing a classification system and functional requirements. The participants reported 28 communication messages with diverse means of expression. Representative examples and their classification highlight the intricacies of non-verbal communication. The proposed architecture emphasizes real-time classification, multiple sensors, and a feedback loop for continuous improvement, enhancing communication for non-verbal children with CP.