Developing Learning Analytics for Epistemic Commitments in a Collaborative Information Seeking Environment

Knight, Simon James Goodwin (2016). Developing Learning Analytics for Epistemic Commitments in a Collaborative Information Seeking Environment. PhD thesis The Open University.

DOI: https://doi.org/10.21954/ou.ro.0000b43f

Abstract

Learning analytics sits at the confluence of learning, information, and computer sciences. Using a distinctive account of learning analytics as a form of assessment, I first argue for its potential in pedagogically motivated learning design, suggesting a particular construct – epistemic cognition in literacy contexts – to probe using learning analytics. I argue for a recasting of epistemic cognition as ‘epistemic commitments’ in collaborative information tasks drawing a novel alignment between information seeking and multiple document processing (MDP) models, with empirical and theoretical grounding given for a focus on collaboration and dialogue in such activities. Thus, epistemic commitments are seen in the ways students seek, select, and integrate claims from multiple sources, and the ways in which their collaborative dialogue is brought to bear in this activity. Accordingly, the empirical element of the thesis develops two pedagogically grounded literacy based tasks: a MDP task, in which pre-selected documents were provided to students; and a collaborative information seeking task (CIS), in which students could search the web. These tasks were deployed at scale (n > 500) and involved writing an evaluative review, followed by a pedagogically supported peer assessment task. Assessment outcomes were analysed in the context of a new epistemic commitments-oriented set of trace data, and psychometric data regarding the participants’ epistemic cognition. Demonstrating the value of the methodological and conceptual approach taken, qualitative analyses indicate clear epistemic activity, and stark differences in behaviour between groups, the complexity of which is challenging to model computationally. Despite this complexity, quantitative analyses indicate that up to 30% of variance in output scores can be modelled using behavioural indicators. The explanatory potential of behaviourally-oriented models of epistemic commitments grounded in tool-interaction and collaborative dialogue is demonstrated. The thesis provides an exemplification of theoretically positioned analytic development, drawing on interdisciplinary literatures in addressing complex learning contexts.