Are Assessment Practices Well Aligned Over Time? A Big Data Exploration

Rogaten, Jekaterina; Clow, Doug; Edwards, Chris; Gaved, Mark and Rienties, Bart (2020). Are Assessment Practices Well Aligned Over Time? A Big Data Exploration. In: Bearman, M; Dawson, P; Ajjawi, R.; Tai, J. and Boud, D. eds. Re-imagining University Assessment in a Digital World. The Enabling Power of Assessment, 7. Cham: Springer, pp. 147–164.

DOI: https://doi.org/10.1007/978-3-030-41956-1_11

Abstract

In the last twenty years a range of approaches have been adopted to facilitate Assessment of Learning as well as Assessment for Learning. With the increased interest in measuring learning gains using assessment data, it is important to recognise the potential limitations of using grades as proxies for learning. If there is a lack of alignment in terms of grade descriptors between modules within a qualification, students might perform really well on one module, and may underperform in a module that has relatively “harsh” grading policies. Using principles of Big Data, we explored whether students’ grade trajectories followed a consistent pattern over time based upon their abilities, efforts, and engagement in two distinct studies. In Study 1, we explored a relatively large dataset of 13,966 students using multi-level modelling, while in a more fine-grained Study 2 we focussed on the pathways of students choosing their first two modules in six large qualifications. The findings indicated substantial misalignments in how students progressed over time in 12 large qualifications in Study 1. In Study 2, our analyses provided further evidence that students’ grades did not seem to be well aligned. In all qualifications we found a highly significant effect of change over time depending on the achievement group. Based upon these findings, we provide clear recommendations how institutions might use similar insights into big data, and how they may improve the longitudinal alignment of grading trajectories by using consistent grading policies.

Viewing alternatives

Metrics

Item Actions

Export

You can export this page using these formats Digital Object Identifier - DOI

About

• Item ORO ID
• 71423
• Item Type
• Book Section
• ISBN
• 3-030-41956-8, 978-3-030-41956-1
• ISSN
• 2198-2643
• Project Funding Details

• Funded Project Name	Project ID	Funding Body
  A longitudinal mixed-method study of learning gain: applying Affective-Behaviour-Cognition framework at three institutions	Not Set	Office for Students

• Academic Unit or School
• Institute of Educational Technology (IET)
• Copyright Holders
• © 2020 Springer Nature Switzerland AG
• Depositing User
• Bart Rienties