The Herald today has:

I’ve lost track of how often I’ve felt the need to deal with Prof Paterson, Scotland’s only regular rentagobprof across Scottish and English MSM, always ready to tell everyone that education in Scotland is in decline and worse than elsewhere and, to my knowledge, only fan of Pisa data.

He is, to my knowledge, the only serious educationist publishing in peer-review journals who thinks the Pisa scores are of any value at all. I’ll come back to Pisa scores below but first and perhaps explaining his odd attachment to them:

Why is Paterson not an expert at all? This:

Professor of Educational Policy at Edinburgh University, Lindsay Paterson is a statistician with a background in agricultural and medical research using statistical methods. He has absolutely no experience of teaching in schools or in teacher education methods. He has no experience of school management or of research into the curriculum other than in counting assessment data. He’s just a jumped up bean-counter. Despite that, he’ll turn up on request, rent-a-gob, to say something bad about any Scottish Government decision that involves numbers.

So, to the Pisa scores he relies upon:

Here’s what an experienced, retired, Glasgow head teacher, has to say about Pisa scores:

By Alasdair Macdonald

The issue is with PISA. It has very little statistical rigour and offers nothing in the way of improving educational practice.

I was a secondary school head teacher at the time it and other ‘measures’ were introduced in education in Scotland in other places. I and my colleagues argued against it.

Educational provision, like everything else, ought to be continuously evaluated, partly to get evidence on whether it is achieving the aims set out for it and partly to get information on areas which require attention. However, there are, literally, hundreds of aspects to be measured and many of these aspects are not measurable by things like tests of arithmetical competence. Such tests are part of the picture but their results are not the full picture (insofar as a full picture can be obtained) Most of the data have to be set in a fairly long time context so that temporary fluctuations can be seen. Usually such temporary fluctuations are easily explainable, in the cases of individual schools.

Such data do, in fact exist. There is a great deal of it and it requires a great deal of care and experience to identify things which are of statistical significance. And things can be significantly ‘good’ as well as ‘poor’.

PISA is of no benefit in this evaluative context. It is an instrument of BLAME and, as such, is a tool of control and compliance over schools and teachers. For some politicians and their media mouthpieces it is a malign instrument which stifles the creativity which is an essential part of teaching.

Some of those in power see such creativity as presenting a threat to their power.

Creativity and the autonomy necessary to be creative can challenge power, but it is the way that people in all walks of life improve things. It is essential in medicine, science, engineering, the arts, social policy, etc.

PISA is a club to beat creativity out of teachers and schools.

Footnote:

Further reading suggested by the Editor – 

Interpreting PISA Results: It’s Poverty, Stupid (With a Bit of the iPhone)

https://fairtest.org/interpreting-pisa-results-its-poverty-stupid-with-a-bit-of-the-iphone/#:~:text=Does%20PISA%20measure%20anything%20of,the%20innovation%20economy%20and%20PISA.

As for Scotland performing ‘below’ England? Complete twaddle. See:

The England PISA sample consisted of 201 eligible schools having at least one pupil in this age range. In England, 3,852 pupils from original sample schools and 911 pupils from replacement schools participated. Pupils in participating schools that did not participate are not replaced. 159 agreed to participate, along with a further 32 replacement schools, but 16 schools withdrew before data collection. Data was therefore collected from 143 schools in the original sample and 32 replacement schools. Of this total, nine original sample schools and one replacement school were omitted from the response rate adjudication process, leaving a total of 134 schools from the original sample and 31 replacement schools.

chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://assets.publishing.service.gov.uk/media/656dc3321104cf0013fa742f/PISA_2022_England_National_Report.pdf

So, in England, in the end, 165 schools and 4 763 pupils took part.

The Scottish participation, from only a tenth of the pupil population and schools, was:

117 secondary schools participated in the survey, representing a school response rate of 96.4 per cent; exceeding the OECD’s minimum standard of 85 per cent. 3,257 students took part giving a weighted student participation rate of 79.4 per cent, slightly below the student participation technical standard of 80 per cent. https://www.gov.scot/publications/programme-international-student-assessment-pisa-2022-highlights-scotlands-results/

There are 357 secondary schools overall in Scotland so a sample of 32.8% was used.

Only 4 763 pupils in 165 schools took part in England. There are 3 458 secondary schools in England so only 4.7% of them took part. Were they then representative or just maybe, the best performing?

There are roughly 10 times as many 15 year-olds in England yet the sample-size was less than twice.

The Pisa results in England and Scotland are simply not comparable and had the English sample of schools been similarly representative and not so suspiciously selective, we can only guess that Scotland would have performed far better than England.