Patrick Yarker looks at the shortcomings of a recent EEF research study, including its failure to understand the depth of the issue it purports to address.
The Education Endowment Foundation (EEF) is government-funded. Its research projects align closely with the DfE’s ‘Areas of Research Interest’. What the EEF looks into and how it reports what it finds carry weight with policymakers. This may be why a recent EEF publication, the Student Grouping Report Evaluation, has been seized on by right-wing commentators. The EEF study seems to show that, in maths lessons for Years 7 and 8, higher-attaining students make more progress when taught in sets grouped by prior attainment than when taught in mixed-attainment groups. Furthermore, it is claimed that setting ‘does not appear to seriously harm’ the attainment of lower-attaining students.
The study builds on prior work by members of the research team across a decade and more. All that time and effort appear to have generated underwhelming findings:
In conclusion, for students overall, those in mixed attainment classes made one month less progress in mathematics, compared with those taught in setted classes. This result was not statistically significant… However, we found that… high prior attaining students in schools using mixed attainment grouping made, on average, two fewer months’ progress in mathematics compared to students in schools using setting. This finding was statistically significant, meaning that it was unlikely to be due to chance. [p71]
Nick Gibb, in a typically misinformed article, was quick to talk up the finding about high prior-attaining students. He claimed the EEF research ‘proved’ setting (and streaming) was better for students than teaching in ‘mixed’ groups.
The research also found that:
For the subgroup of students in receipt of Free School Meals (FSM) there was effectively no difference in attainment, indicating that, contrary to some qualitative evidence, students from disadvantaged backgrounds are not further disadvantaged by the practice of setting. In addition, we found statistically significant, negative effects on general self-confidence for students in mixed attainment groups, which were small for the student group as a whole and for students in receipt of FSM, and moderate for the low-prior-attaining students. [p69]
Two of the researchers, interviewed in the TES, noted that this finding was a surprise. Previous research indicates that students in bottom sets may have a poorer experience than those in mixed groups.
Fair comparison?
I think the finding to do with general self-confidence may stem from a drawback with the design and implementation of the whole study. The research looked to compare a group of 27 schools teaching Year 7 and Year 8 maths in mixed attainment classes with 69 ‘matched’ schools which taught these year-groups in sets. The researchers tried to isolate and reveal the effects on student attainment and confidence of these differing grouping arrangements. But no classroom can be isolated from the school of which it is part, nor from the wider education system. Nor can differences in school ethos and feel, and in the ‘vibe’ of individual classrooms, be adequately controlled for or factored out. Even matching the less dynamic and complex contextual differences can prove tricky, as the report acknowledges:
It was envisaged that the matching would be conducted on the full set of variables identified for matching. This proved difficult to achieve... As a result, we conducted the matching exercise using key, mainly attainment, variables, because these were judged to play the most significant role. [p17]
That judgement risks leaving out of account much which affects how students present in class.
While the attainment variables may well have been matched as closely as possible, other characteristics of the two different groups of schools indicate how hard it is to establish grounds for fair comparison. The proportion of schools judged by Ofsted to require improvement was five times larger in one group than in the other (15% against 3%). The proportion of students in receipt of Free School Meals was more than ten per cent higher in one group than in the other (22% against 19%). And the proportion of students for whom English is not a first language was almost fifty per cent higher in one group than the other (20% against 14%). These characteristics speak to the being of each school as an institution. Discretely and in combination they will have a shaping and enduring influence on what goes on there.
Furthermore, the report notes that schools in the mixed attainment group were larger and that fewer were rated ‘Outstanding’ by Ofsted. Their teachers tended to be slightly less experienced than teachers in setting schools. These areas of mis-match are acknowledged in the report to pose a moderate to high threat to the validity of its findings (p81). I think that threat is especially salient when considering the way students describe how confident or unconfident they feel in their work in class.
No escaping the ‘ability’ discourse
But there is something more important at stake than research design. The research seeks to compare the effects of two different kinds of student grouping. But aren’t the practices, attitudes and enacted beliefs realised in the classroom more fundamentally at issue? Whether in a hierarchical and setted system or in one supposedly more mixed, the ways in which pupils are constructed and worked with in terms of fixed ‘ability’ endure. Recourse to the language of ‘attainment’ offers no escape. A student’s attainment (which is to say, their scores on particular tests) is understood to be a proxy for ‘ability’, itself seen as innate and largely immutable or fixed.
The language of attainment serves to conceal the discourse of ‘ability’, until the symbiotic relationship between the two surfaces when issues such as grouping or particular kinds of pedagogic practice are under discussion. So the Guardian report on the EEF research deploys as equivalents the terms ‘ability’ and ‘attainment’. Its headline speaks of ‘classes grouped by ability’, and the report continually equates ‘mixed attainment’ with ‘mixed ability’. The same conflation is apparent in what some teachers are quoted as saying in the EEF study.
Nor are the researchers themselves immune. They interpret certain findings, and make recommendations, based on their own fixed ‘ability’ thinking. For example, they consistently present students as falling into discrete units or groups, notably the high-attaining and the low-attaining, and never suggest or allow that a student in a low-attaining group could become high-attaining, or vice versa. Grouped in this way, students are reified, as are the groups of which they form a part:
We did however, see some examples of practice that might limit opportunities for high-prior attaining students, for example, there being a lack of resources to stretch high-prior attainers in the classroom… [p70]
… what strategies do setted schools use to challenge high attainers and which of these are most effective, and what strategies could mixed attainment schools use to effectively challenge high attainers within mixed attainment classes… [p72]
The problem of low attainment in mathematics remains a serious problem and it is therefore, important that research considers other approaches to raising the attainment of this group of students. [p72]
The researchers seem oblivious to this aspect of their perspective. They assert more than once that ‘more research is needed into methods to increase challenge and depth of understanding for all’ [p72] while continually confirming the apparent fixity of student attainment/’ability’ and the imperviousness of the reified groupings. At one point they seem to recommend that like should work only with like, to avoid limiting opportunities for high-attaining students:
[We saw] a focus on using high-prior attainers to support low-prior attainers rather than providing carefully structured pair or group work that might allow high-, middle-, and low-prior attainers to mutually support one another. [p70]
A similar framework seemed to underlie a conclusion which the researchers foregrounded in their TES interview:
It benefits high attainers to explain and articulate their mathematical ideas, and it benefits middle and lower attainers to hear and respond to those articulations.
High attainers explain and articulate; middle and low attainers hear and respond. There is no recognition that assigning these labels might be problematic in itself, or that opportunities to explain and articulate might benefit all students, as might the chance to respond.
This kind of stance confirms a fixed ‘ability’ lens and re-confirms the pervasiveness of the ‘ability’ discourse by which students are not deemed able to exceed or belie their given label. The low-attaining cannot become high-attaining, or vice versa, and the mid-attaining likewise stay where they are. All students are trapped in part by the way the given label dictates the kind of work they receive, the ways they are responded to be teachers, and how they see themselves as learners.
The researchers noted a lack of variation in the way lessons were taught across all schools:
We saw very little, small group or pair work in any schools, few open-ended tasks or problem-solving opportunities, and limited use of manipulatives. [p62]
But they appear not to have considered that this might be a factor in limiting the mathematical development of all students, not just of the high attaining.
Structures matter too
The researchers argued in their TES interview that: ‘schools can do mixed attainment well or less well - and similarly, that they can do setting well or less well.’ This suggests that the way students are grouped matters less than the way students are taught. But structures affect practice. Certain structures, such as setting or ‘ability’ grouping, reinforce particular views about students as learners: their learning capacity, motivation, likely degree of engagement,
possible trajectory of progress, and so on. These structures can replicate and reinforce existing inequalities to do with class and/or ethnicity.
That said, mixed groups do not, on their own, provide a solution. Fixed ‘ability’ thinking persists in relation to them too, and as it does despite changes to language and terminology. In the current system, students will be seen and constructed through the lens of ‘ability’ unless teachers actively and consistently refuse to do this—an arduous and challenging task—and are supported in that refusal by the ethos, structures and approaches of the school. In an education system so profoundly constrained by accountability measures, not least the requirement to maximise test scores, such support is hard to come by.
The EEF Grouping Study has been brandished by those who advocate an education system segregated by ‘ability’, and the authors of the research have not pushed back against this. Teachers who want to see all their students thrive in the maths classroom, and everywhere else, should be aware of the report’s shortcomings, the ideological misuse to which it is already being put, and the continuing need to expose and counter the ‘ability’ discourse, its language and practices, as a prelude to moving beyond it in the cause of a genuinely comprehensive education.
Comments?We welcome replies, corrections and debate pieces. Contact us