EMMA - a Computer Assisted Assessment System based on Latent Semantic Analysis

Haley, Debra Trusso; Thomas, Pete; Petre, Marian and De Roeck, Anne (2008). EMMA - a Computer Assisted Assessment System based on Latent Semantic Analysis. Technical Report 2008/14; Department of Computing, The Open University.

DOI: https://doi.org/10.21954/ou.ro.0001607e

Abstract

We present EMMA (ExaM Marking Assistant), a Latent Semantic Analysis (LSA) based Computer Assisted Assessment System (CAA) we have developed as part of ELeGI (www.elegi.org)-"A semantic Grid for human learning is the vision behind the European ELeGI Integrated Project for the implementation of future learning scenarios based on ubiquitous, collaborative, experiential-based and contextualized learning through the design, implementation and validation of the Learning Grid". Assessment is an important component of learning and can have a strong impact on student progress. EMMA can provide both formative and summative assessment that is unbiased and repeatable as well as the almost instant feedback that is most useful for student learning. Our work has demonstrated that, even though the theory of LSA is over 15 years old, many of the details that make LSA a practical assessment technique are not known by the research community beyond the original LSA developers. In this paper, we summarise what we have learned about LSA, give an overview of how EMMA works, describe the types of questions that EMMA can assess, and evaluate its results as compared to human markers, and outline a plan for further research.