Testing for over- and under-dispersion in physics degree outcomes

Sword, Astra (2023). Testing for over- and under-dispersion in physics degree outcomes. In: Physics Education Research Conference 2023, 19-20 Jul 2023, Sacramento, California, pp. 356–361.

DOI: https://doi.org/10.1119/perc.2023.pr.Sword

Abstract

As the scale of quantitative data available to physics education researchers grows, it is imperative that we critically assess how well the assumptions behind "standard" statistical methods apply in our field. In the present work, I give a background on a common statistical assumption used to analyse proportion data, the binomial assumption; I discuss scenarios in which this assumption may break down in the context of education research; and test this assumption using a large population-level data set. This data set comprises academic outcomes (rate of 'good degrees') for all undergraduate physics degree programs that ran in the UK across the 2012/13--2018/19 period (26,960 students across 79 programs). I estimate dispersion parameters and their significance for each program in the data set and discuss the implications of the results for analysing proportion data in physics education research.