Copy the page URI to the clipboard
Yu, Yijun; Wang, Xiaozhu; Dil, Anton and Rauf, Irum
(2019).
URL: https://wcol2019.ie/
Abstract
While teaching the art of Computer Programming, students with visual impairments (VI) are disadvantaged, because speech is their preferred modality. Existing accessibility assistants can only read out predefined texts sequentially, word-for-word, sentence-for-sentence, whilst the presentations of programming concepts could be conveyed in a more structured way. Earlier we have shown that deep neural networks such as Tree-Based Convolutional Neural Networks (TBCNN) and Gated Graph Neural Networks (GGNN) can be used to classify algorithms across different programming languages with over 90% accuracy. Furthermore, TBCNN or GGNN have been shown useful for generating natural and conversational dialogues from natural language texts. In this paper, we propose a novel pedagogy called “Programming Assistant”, by creating a personal tutor that can respond to voice commands, which trigger an explanation of programming concepts, hands-free. We generate dialogues using DNNs, which substitute code with the names of algorithms characterising the programs, and we read aloud descriptions of the code. Furthermore, the application of the dialogue generation can be embodied into an Alexa Skill, which turns them into fully natural voices, forming the basis of a smart assistant to handle a large number of formative questions in teaching the Art of Computer Programming at a distance.
Viewing alternatives
Download history
Item Actions
Export
About
- Item ORO ID
- 62778
- Item Type
- Conference or Workshop Item
- Project Funding Details
-
Funded Project Name Project ID Funding Body SAUSE: Secure, Adaptive, Usable Software Engineering EP/R013144/1 (previous: EP/R005095/1) EPSRC (Engineering and Physical Sciences Research Council) Why Johnny doesn’t write secure software? Secure Software Development by the Masses Not Set EPSRC (Engineering and Physical Sciences Research Council) ESTEEM Not Set Not Set - Keywords
- Transformative Online Pedagogies; Deep Neural Networks; Algorithm Classification; Chat Bots; Alexa Skill; Programming Assistant
- Academic Unit or School
-
Faculty of Science, Technology, Engineering and Mathematics (STEM) > Computing and Communications
Faculty of Science, Technology, Engineering and Mathematics (STEM) - Research Group
- Centre for Research in Computing (CRC)
- Depositing User
- Yijun Yu