To put it plainly, based on the reading of two prominent academics in England, the sample from English schools is light on the percentage of pupils with special needs. This echoes the problems of representativeness in the English sample in 2022 (more below) when the results were ‘asterisked’ with a serious warning after almost being fully excluded. There is no history of such gaming or bias in the other 3 UK nations.
Help Talking-up Scotland keep going
Crowdfunder at: https://www.crowdfunder.co.uk/p/help-talking-up-scotland-keep-going
Or by direct bank transfer to: JOHN WATSON ROBERTSON Sort Code – 08-91-04 Account – 12266421
The above from the Times Education Supplement‘s analysis of the Pisa test results today is being used attack the Scottish education system and of course the SNP Government.
The English results are subject to serious concerns.
Second, following on from the critiques of Professor Frijters retired, of LSE 2016-2019, and of John Jerrim Professor of Education and Social Statistics at UCL, above, the 2025 PISA sample for England is being claimed to be more representative than in several previous cycles (especially 2022).
A degree of scepticism is reasonable.
England appears to have met the OECD’s official participation thresholds this time: a final weighted school response rate of 90% (the standard is 85%) and a pupil response rate of 83% (the standard is 80%). The original school sample was 205 schools; after replacements the final sample was 183 schools and 5,619 pupils. England has a long record of weak PISA response rates and a tilt toward more willing, often higher-attaining, schools. A sudden jump from failing the OECD standards in 2022 to meeting them in 2025 does not, by itself, prove the sample is now clean.
A government that wants a good PISA story, plus a contractor (Pearson) whose job is to get schools over the line suggests we cannot rule out softer selection with more schools that are better organised, more compliant, or under more official encouragement taking part. https://www.pearson.com/uk/web/pisa25/en/for-school-coordinators/next-steps.html
So, the remaining exclusion rate, pupil non-response among disadvantaged and SEN pupils, and England’s history of gaming and low take-up are enough to treat the 2025 scores as a bit high relative to the true population
Third, and this is where things get political, John Jerrim (UCL) noted that England’s advantage over the rest of the UK is especially at the top of the distribution. England is not 25 PISA points ahead of Scotland at every ability level. A large part of the mean gap comes from England’s high attainers doing better than Scotland’s high attainers. The middle and the bottom are closer. https://www.tes.com/magazine/news/general/pisa-2025-scores-revealed-england-uk-ranking
Put simply, this means that the apparently better scores in English schools is mostly amongst the top fifth.
Some more reliable research:
Help Talking-up Scotland keep going
Crowdfunder at: https://www.crowdfunder.co.uk/p/help-talking-up-scotland-keep-going
Or by direct bank transfer to: JOHN WATSON ROBERTSON Sort Code – 08-91-04 Account – 12266421
Discover more from Talking-up Scotland
Subscribe to get the latest posts sent to your email.
I remember in the first few years of PISA and looking at the samples of students in the countries which were ‘top of the league’. These countries presented no students who had additional needs. Others presented mainly children outwith the two lowest socioeconomic deciles.
With regard to children from Scotland, a number of the questions were on topics which were no longer in the curricula in Scottish education.
Much is known of the concomitant factors which lead to higher outcomes. For example present only girls for examination, or present no children from single parent families, present only children whose mothers have a number of Higher grades or university degrees, etc.
An example of gaming the system related to the ‘derived grade procedure’ which used to be applied to results after the papers had been marked but before grades had been determined by the examiners. When schools present children for examination, they have to provide a list of all candidates in rank order according to the evidence the school has, such as preliminary examinations. If the ranking order from the school matches closely the ranking order produced by the actual exams, the school is deemed ‘concordant’. So, if, for example, in a group of ten consecutive student 8 attain the expected grade, the other two are automatically awarded the expected grade even though their actual performance on exam day was lower. In the private sector because of selection by affluence they have large numbers of children being presented compared to a local authority school in an area of low socioeconomic status. Because the numbers are large, its rank order is deemed statistically significant and those within the group whose marks were lower are automatically given the same grade as their peers. But in the low socioeconomic group the sample size is small and is not deemed statistically significant, so pupils in the group whose exam marks are less than expected are not automatically upgraded and have to submit evidence which, in most cases is rejected.
For example Posh Academy presents 24 pupils expected gain an A pass and 18 do so on the first making then the other four are upgraded to A. At Ragged Secondary 8 are expected to get an A, and 6 do, the other two do not get upgraded automatically because the sample size is smaller. Evidence for appeal is submitted, and, in almost all cases is deemed insufficient. Having graded such exams I confirm that most evidence submitted whether from Posh Academy or Ragged Secondary IS insufficient. But, in most cases Posh Academy does not need to submit evidence. So, there is an incentive to place a few pupils who are unlikely to to get an A in with certain A passes, on the expectation of automatic upgrading. Thus, the affluent schools can game the system to inflate their pass rates.
This was made clear in the diet of 2000 or 2001, when the Scottish Exam Board made an arse of things (to use the technical term!) The Detived Grade Procedure was unable to be applied, so, there were no automatic upgrades and the posh school pass rates were pretty much the same as those for the local schools. Cue outrage by the rightwing press and BBC Scotland!
LikeLiked by 2 people