The Open UniversitySkip to content

Teaching the Art of Computer Programming at a Distance by Generating Dialogues using Deep Neural Networks

Yu, Yijun; Wang, Xiaozhu; Dil, Anton and Rauf, Irum (2019). Teaching the Art of Computer Programming at a Distance by Generating Dialogues using Deep Neural Networks. In: 28th ICDE World Conference on Online Learning, 3-7 Nov 2019, Dublin, Ireland, (In Press).

Full text available as:
PDF (Version of Record) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Download (434kB) | Preview
Google Scholar: Look up in Google Scholar


While teaching the art of Computer Programming, students with visual impairments (VI) are disadvantaged, because speech is their preferred modality. Existing accessibility assistants can only read out predefined texts sequentially, word-for-word, sentence-for-sentence, whilst the presentations of programming concepts could be conveyed in a more structured way. Earlier we have shown that deep neural networks such as Tree-Based Convolutional Neural Networks (TBCNN) and Gated Graph Neural Networks (GGNN) can be used to classify algorithms across different programming languages with over 90% accuracy. Furthermore, TBCNN or GGNN have been shown useful for generating natural and conversational dialogues from natural language texts. In this paper, we propose a novel pedagogy called “Programming Assistant”, by creating a personal tutor that can respond to voice commands, which trigger an explanation of programming concepts, hands-free. We generate dialogues using DNNs, which substitute code with the names of algorithms characterising the programs, and we read aloud descriptions of the code. Furthermore, the application of the dialogue generation can be embodied into an Alexa Skill, which turns them into fully natural voices, forming the basis of a smart assistant to handle a large number of formative questions in teaching the Art of Computer Programming at a distance.

Item Type: Conference or Workshop Item
Project Funding Details:
Funded Project NameProject IDFunding Body
SAUSE: Secure, Adaptive, Usable Software EngineeringEP/R013144/1 (previous: EP/R005095/1)EPSRC (Engineering and Physical Sciences Research Council)
Why Johnny doesn’t write secure software? Secure Software Development by the MassesNot SetEPSRC (Engineering and Physical Sciences Research Council)
ESTEEMNot SetNot Set
Keywords: Transformative Online Pedagogies; Deep Neural Networks; Algorithm Classification; Chat Bots; Alexa Skill; Programming Assistant
Academic Unit/School: Faculty of Science, Technology, Engineering and Mathematics (STEM) > Computing and Communications
Faculty of Science, Technology, Engineering and Mathematics (STEM)
Research Group: Centre for Research in Computing (CRC)
Item ID: 62778
Depositing User: Yijun Yu
Date Deposited: 07 Aug 2019 08:04
Last Modified: 06 Sep 2019 03:37
Share this page:

Download history for this item

These details should be considered as only a guide to the number of downloads performed manually. Algorithmic methods have been applied in an attempt to remove automated downloads from the displayed statistics but no guarantee can be made as to the accuracy of the figures.

Actions (login may be required)

Policies | Disclaimer

© The Open University   contact the OU