Copy the page URI to the clipboard
Ball, Martin Huw (2007). Evaluation and Improvement of an Automatic Marking Tool for Diagrams. Student dissertation for The Open University module M801 MSc in Software Development Research Dissertation.
DOI: https://doi.org/10.21954/ou.ro.0001605e
Abstract
Within the teaching environment, the ability to automatically grade student assignments and examination answers offers many potential advantages, including improving the speed and standardisation of the marking process, and the release of time spent marking by tutors which can then be utilised for teaching. A marking tool has been developed by the Open University as an investigation into the feasibility of automating the grading of student Entity Relationship (E-R) diagram answers. Two small scale trials of the marking tool were performed by the developers. These showed that automated marking was potentially feasible, and that detailed failure analysis of the incorrectly marked diagrams could be used to develop and implement improvements to the marking tool. The research described in this dissertation extends the work of this team, using 2 new corpora of graded E-R diagrams. The first of these contains 197 diagrams and the second contains 32 diagrams. The second corpus introduced additional complexity as the model E-R diagram answer used entity subtypes within the solution diagram. This dissertation describes the work undertaken to establish an experimental procedure to grade the diagrams using the marking tool; the establishment of statistical and graphical techniques to measure the performance of the marking tool; the root cause analysis of marking tool deficiencies where diagrams are incorrectly graded; and the design, implementation and testing of improvements to the marking tool.