Copy the page URI to the clipboard
Erling, Elizabeth (2009). An investigation into the relationship between the use of academic language and attainment – with a focus on students from ethnic minorities. Open University.
URL: http://intranet.open.ac.uk/equality-diversity/obje...
Abstract
The research employed textual analysis, tutorial observation and interview procedures. Language specialists worked together with subject tutors in three OU courses to investigate the features of student texts that are associated with higher scoring assignments. The researchers then assessed whether students from BME backgrounds were less likely to produce these language features.
Texts were obtained from 78 students with a range of assignment marks, from the following ethnic groups: African, Caribbean, Pakistani, Other (Bangladeshi, Chinese and Asian Other) and White (White British and White Other). A total of 220 texts drawn from three subject areas were analysed.
The courses chosen for investigation were a level 1 course in Social Sciences, a level 1 course in Technology and a level 2 course in Health and Social Care (DD100, T175 and K204). The language specialists studied course materials, visited tutorials, met with students and analysed the first, mid and final tutor marked assignments (TMAs) of students in their group. The text analysis was carried out using the MASUS (Measuring Academic Skills of University Students) assessment procedure developed at the University of Sydney. The MASUS procedure assesses students’ academic writing skills in five areas: A. Use of source material; B. Structure and development of text; C. Control of academic writing style; D. Grammatical correctness; and E. Presentation. They are given a score in each of these areas: a low score indicates that the student needs support in this area of language use. The data from the MASUS procedure were analysed statistically and qualitatively.
Results
Analysis of the MASUS data indicated a strong correlation between students’ marks and their use of language (0.52). It also showed that:
• The relationship between language and attainment was stronger in earlier TMAs on all courses. There was a stronger correlation between language and attainment in the Social Sciences and Health and Social Care courses than in the Technology course.
• The areas of language use that related to the role of language in how content is presented (Use of source material and Structure and development of text) were particularly significant in determining a student’s marks on early TMAs. These are also the categories that tutor feedback focused on.
• There was no statistical correlation identified between students’ mark and the areas Academic writing style and Grammatical correctness. However, qualitative analysis indicated that these features distinguished high performers from middle performers. There was little tutor feedback on these categories, and their role in how content is presented and linked was not recognised by the tutors or the course materials.
A large number of the total sample – as many as 47%– had language scores that indicated a need for additional support in the areas identified, both in their initial TMAs and in TMAs later in the course (see Table 1 below). This percentage was higher for students in this sample from ethnic minorities. This may at least partially explain the comparatively poor attainment of students from ethnic minorities that has been identified.
In this sample, students from ethnic minorities were more likely than students who are White to receive low scores in the area Use of source material (35%, compared with 6%). Plagiarism and problems with referencing were also more likely to occur among students from ethnic minorities.
Students from ethnic minorities were more likely than students who are White to receive low scores in the area Structure and development of text (53%, compared with 17%). This gap in language performance narrowed in later TMAs, but is still notable.
Table 1: The number of students in sample with low language scores
Initial TMA Later TMA
Total % of BME % of White Total % of BME % of White
Use of source material 29% 35% 6% 24% 25% 6%
Structure and development 47% 53% 17% 39% 40% 11%
Academic writing style 45% 40% 56% 34% 28% 33%
Grammatical correctness 24% 23% 22% 39% 37% 22%
Presentation 17% 13% 28% 12% 12% 6%
(n=78) (n=60) (n=18) (n=69) (n=55) (n=14)
Students who are White were more likely than students from ethnic minorities to receive low scores in the area Academic writing style, particularly in the initial TMAs (56%, compared with 40%).
Students from ethnic minorities were more likely than students who are White to get low scores in the area Grammatical correctness, particularly in later TMAs (37%, compared to 22%). The number of students with low scores in the area Grammatical correctness increased over the period of the course. While this area was not found to correlate statistically with student attainment, the researchers who analysed the texts identified a number of grammatical features that appeared to be particularly important for high achievers:
• correct sentence structure
• use of noun phrases for topic development and conciseness
• correct and appropriate use of modals in argumentation
• use of passive voice for organising information
Apart from the first feature, the significance of these grammatical features lies not in their ‘correctness’ (which is a matter of form) but in their ‘appropriacy’ (which is a matter of how they are used). These features play an important role in students’ use of source materials and structuring and development of texts.
Other findings
The investigation suggests that the role of language in how content is presented and linked is not recognised by the course materials or the tutors.
Course materials
Tutors are encouraged to focus their marking on students’ understanding of key concepts in the course. The role that language plays in how these concepts are understood, presented and linked is not acknowledged in the courses’ assessment guidelines. As a result, the students perceive the significant amount of generic language support material available as peripheral.
Students’ success in assignments was found to be linked to following the conventions of the text type (or ‘genre’) and using the writing style (or ‘register’) expected by the tutor and referred to in the Assignment Guidelines. In some cases, the assignment guidelines were found to be complicated (e.g. 5½ pages long) and to demand a range of text types and writing styles. This presents further challenges for students unfamiliar with this range of genres and registers.
Mixed messages are conveyed to students about the style which is appropriate in academic writing. Students may be offered articles to read which are more ‘academic’ and formal in style, but the ‘voice’ of most OU material is friendly and relatively informal.
Tutor feedback
In all courses, aspects such as writing style, grammatical correctness and punctuation were not often commented on or corrected by tutors. At times, tutors did not comment on sections of texts where the researchers perceived a student’s ability to communicate ideas to be severely affected.
On some assignments, marks were allocated for grammatical correctness and structure, but some students obtained full marks when these aspects of their writing were judged as being far from perfect.
When tutors gave language feedback, they tended to ‘tell’ and not ‘show’. This did not seem to make a significant impact on student performance.
Tutors did not always provide good language models in their feedback.
Students in one group who had direct and sustained contact with a language specialist showed more progress in their writing development than those in the other groups.
Implications
The research suggests that language plays a significant role in student attainment. A large number of the students from ethnic minorities in the sample had language scores that indicated a need for additional support, particularly in the areas Structure and development of text. This area is particularly significant in determining students’ marks and could explain in part the gap in attainment.
But, this investigation suggests that the role of language in how content is presented and linked is not recognised by the course materials or the tutors. Nor does the generic language support available seem to be serving its purpose, as the number of students with low scores in the area Grammatical correctness increased as the courses progressed.
The findings suggest that it may not be in the students’ best interest to be waved through a course without sufficient attention paid to their language needs, as this may prove a significant barrier to achieving above a certain level (cf. O’Shea-Poon and Kimura 2008, who found that non-fluency in English appears to be a significant barrier to achieving above a certain level in OU courses).
Recommendations
It was not the primary purpose of this research to generate recommendations for practice but a number of good practices are suggested by the findings.
Tutors should be supported in raising their awareness of the importance of language in student attainment and in giving helpful feedback on students’ language use across the span of a course.
Only by linking grammar and vocabulary development to the categories Use of source material and Structure and development of text is it possible to go beyond notions of language as ‘a problem’ that interferes in students’ performance towards notions of language as ‘a resource’ to be developed as part of students’ academic attainment.
Feedback on students’ language use should concretely demonstrate how to do things, instead of abstractly telling them. The use of language in tutor feedback should also provide a model.
Students and tutors could be offered models of good writing practice in addition to the assignment descriptions they already receive in Assignment Guidelines. These could be exemplars of text types and writing styles appropriate to key assignments on a course (see, for example, the concept maps for TMAs designed as part of this project).
A diagnostic might be a useful tool to identify those students who could benefit from some support with academic English. Tutors could then target that support most effectively.
Further research
Further data should be collected on students’ language backgrounds.
Other useful studies to gain more insight into the relationship between language use and attainment would include further examination of:
• the grammatical constructions required in the structuring and development of assignments, particularly in the areas of grammar identified in this project; instances of plagiarism and problems with referencing; use of informal language
• tutor feedback on language use
• the research instruments used in this project.
Further information
Please contact Elizabeth Erling or Jim Donohue in OpenELT for any further information on this project, including a copy of the full report, which includes case studies of the three courses involved (DD100, T175 and K204): e.j.erling@open.ac.uk ; j.p.donohue@open.ac.uk. Contact Rachel Hawkins in Student Services regarding any actions: r.a.hawkins@open.ac.uk.
Project team
David Hann, John Kearsey, Kerry Bannister, Christine Buller, Christina Healey, Zoe Doyé, Chris Lee, Harish Mehra
Acknowledgements
With gratitude to
• Student Services, who generously funded this project;
• the project advisors: Rachel Hawkins, Roberta Nathan, Anne Jelfs, Erica Morris, Maggie Coates, Maki Kimura, Tony O’Shea-Poon, John Richardson;
• the tutors, course teams and Regions that allowed the project to take place; and
• the students who volunteered to participate.