Comparing machine graded diagrams with human markers: some observations

Thomas, Pete (2004). Comparing machine graded diagrams with human markers: some observations. Technical Report 2004/27; Department of Computing, The Open University.

DOI: https://doi.org/10.21954/ou.ro.00016015

Abstract

In this paper we examine the performance of an automatic (machine) grading algorithm for entity-relationship (E-R) diagrams by comparing it with human generated marks for a set of student answers to an assignment question. Using a variety of statistical tests it is shown that the performance of the automatic marker is very close to that of the human markers: the Pearson correlation coefficient is 0.964 (significant at the 0,01 level, 2-tailed, N=26) and the Kendall tau-b correlation coefficient is 0.919 (significant at the 0.01 level, 2-tailed, N=26). The investigation revealed deficiencies in both the machine and human markers. There is prima-facie evidence that the orientation (shape) of a diagram may influence humans to award lower marks than they should.

Viewing alternatives

Download history

Metrics

Public Attention

Altmetrics from Altmetric

Number of Citations

Citations from Dimensions

Item Actions

Export

About