Copy the page URI to the clipboard
Bektik, Duygu; Ullmann, Thomas; Edwards, Chris and Herodotou, Christothea (2024). PVC-Challenges Response: AI for course content generation. Quality Enhancement and Innovation, The Institute of Educational Technology, The Open University, Milton Keynes, UK.
Abstract
Executive summary
This report describes the work and findings of the second phase of the PVC-RI IET challenge for Jan 2024 to July 2024. In the first phase we considered, through a literature review and a review of the OU curriculum development processes, the opportunities for generative AI (gen AI) and specifically GPT-4 to be used within the broad range of tasks associated with curriculum development. Some of these were practically explored using GPT-4 within the OU’s secure Microsoft Azure environment. In this second phase of the challenge, the focus is on using GPT-4 to produce new from existing curriculum considering the practicalities involved and efficiencies that could be achieved. Therefore, a 40-hour CPD offering was developed from a 150-hour microcredential on the same subject (Evaluating a course).
Having contributed to the Business Development Unit’s (BDU’s) project to rework material from existing modules into a short course, we were aware of the difficulties experienced without the close involvement of a subject specialist. Therefore, we selected an existing IET module to rework into another described within the proposed IET curriculum plan with colleagues with expertise both in the technologies involved and the subject matter. The new CPD course, produced from existing and licensed OU resources, has been assessed in terms of the following: a) quality of the AI-generated content produced, b) the time needed for producing this content and c) the range of iterations needed to improve prompts (prompting strategy). This report contributes an initial set of evidence-based recommendations as to how to effectively produce AI-generated creseurriculum using existing OU resources. The key role of subject experts in producing and revising prompts as well as reviewing final outcomes should be highlighted.
We found that GPT-4 with RAG is an effective productivity tool for drafting a smaller from a larger module when a suitable prompt strategy is selected and refined. We did not find any factual errors in the responses and were able to obtain references within the text that were correct. In terms of assessment, this CPD output makes full use of reflective activities that can contribute to a portfolio. Also, we included quizzes which GPT-4 supported well, for ideas and drafting. In addition, we describe a number of lessons we have learned and have formed recommendations that we will take into the ongoing use of the tool to develop a suite of CPD courses.
These recommendations will be used by the Curriculum Team in IET, responsible for the Masters in Online Teaching, and further refined between Aug 2024 to July 2025, as a ‘base recipe’ to enhance the rapid implementation of the IET Curriculum Growth Plan, involving the production of paid CPD courses. To support this next phase of development, the experienced QEI team will offer training to IET colleagues in the Curriculum team on using gen AI in the production of new curriculum. They will also monitor and document changes to the prompting approach resulting in a refined version of the evidence-based recommendations detailed in this report, such as whether we need to revise and adapt these for every new CPD (and what those adaptations should be).
CORE (COnnecting REpositories)
CORE (COnnecting REpositories)
