Data collection techniques used to study programming - and AESOP

Paine, Carina (2000). Data collection techniques used to study programming - and AESOP. Technical Report 2000/04; Department of Computing, The Open University.

DOI: https://doi.org/10.21954/ou.ro.0001602d

Abstract

All empirical studies of programmers require suitable methodologies and techniques. Different methods of data collection are appropriate in different circumstances. Surprisingly, the initial research in the study of programming focused on experiments, when observational methods would seem more appropriate for a new discipline [1]. Recently there has been a shift to observational methods due to their ability to capture more complex aspects of programming. "I don't think that programming is too complex a behaviour to be studied, but it may be too complex to be studied in a laboratory" [2].AESOP, An Electronic Student Observatory Project, is a collection of computer-based data collection tools. Our educational environment is one in which students study independently at a distance, off-line, using software developed for the Open University course M206, Computing: An Object-oriented approach. The aim of the project is to discover how students learn to program, by observing students unobtrusively, electronically, and automatically, and to record these observations in a manner that is useful for both instruction and research. This documents reviews the differences between experimental and non-experimental methods, and discusses the advantages and disadvantages of the various data gathering techniques in the empirical study of programming, including new computer aided techniques. The data collection techniques used for AESOP are also described, and the reasons for our choice of techniques is explained.