Validating the Reflective Writing Framework (RWF) for Assessing Reflective Writing in Computer Science Education Through Manual Annotation

Alrashidi, Huda; Joy, Mike; Ullmann, Thomas and Almujally, Nouf (2020). Validating the Reflective Writing Framework (RWF) for Assessing Reflective Writing in Computer Science Education Through Manual Annotation. In: Intelligent Tutoring Systems (Kumar, Vivekanandan and Troussas, Christos eds.), Lecture Notes in Computer Science, Springer International Publishing, Cham, pp. 323–326.

DOI: https://doi.org/10.1007/978-3-030-49663-0_38

Abstract

The accuracy of a framework for annotating reflective writing can be increased through the evaluation and revision of the annotation scheme to ensure the reliability and validity of the framework. To our knowledge, there is a lack of literature related to the accuracy of any reflective writing framework in Computer Science (CS) education. This paper describes a manual annotation scheme, applied during four pilot studies, to validate the authors’ novel Reflective Writing Framework (RWF) for CS education. The results show, through the pilot studies, that the accuracy of Inter-Rater Reliability (IRR) increases from 0.5 to 0.8, which was substantial and close to an almost perfect agreement. This paper contributes to CS education through the reliability and validity of the RWF that can be potentially used for generating an Intelligent Tutoring Systems (ITS) using machine learning algorithms.

Viewing alternatives

Download history

Metrics

Public Attention

Altmetrics from Altmetric

Number of Citations

Citations from Dimensions

Item Actions

Export

About