Recording multimodal pair-programming dialogue for reference resolution by conversational agents

Domingo, Cecilia (2023). Recording multimodal pair-programming dialogue for reference resolution by conversational agents. In: Proceedings of the 2023 International Conference on Multimodal Interaction (ICM '23), Association for Computing Machinery, New York, NY, USA pp. 731–735.

DOI: https://doi.org/10.1145/3577190.3614231

Abstract

Pair programming is a collaborative technique which has proven highly beneficial in terms of the code produced and the learning gains for programmers. With recent advances in Programming Language Processing (PLP), numerous tools have been created that assist programmers in non-collaborative settings (i.e., where the technology provides users with a solution, instead of discussing the problem to develop a solution together). How can we develop AI that can assist in pair programming, a collaborative setting? To tackle this task, we begin by gathering multimodal dialogue data which can be used to train systems in a basic subtask of dialogue understanding: multimodal reference resolution, i.e., understanding which parts of a program are being mentioned by users through speech or by using the mouse and keyboard.

Viewing alternatives

Metrics

Item Actions

Export

You can export this page using these formats Digital Object Identifier - DOI