Oh dear, not PISA again?

By stewartb

Oh dear, not PISA again? This supposed hammer blow to the SNP government and to the credibility of Scotland’s school system is becoming tiresome – and the claims based solely on it are just plain wrong and for lots of reasons. The claims are IMHO often driven by a political agenda.

I venture the alarmist claims also run counter to most parents’/carers’ and school students’ direct experience of our schools. Could Scottish education be better? Of course – it would be daft to deny this for any education system as it would to deny it for any area of public policy in a democracy.

I wrote the following btl for Talking up Scotland on 25 June, 2021 to make good the media deficit on what the OECD itself – the organisation which delivers the PISA tests – wrote positively about Scotland’s schools system in 2012. Here are some bits of what I wrote then:

‘… the OECD report provides a much more measured, balanced account of Scotland’s PISA results – and surely being in an OECD report, we can take this account as more authoritative than many others? I offer this as two among many positives from across the OECD report.

The report explains that PISA’s new global competence module “aims to capture the capacity of 15-year-olds to examine local, global and intercultural issues, to understand and appreciate the perspectives and world views of others, to engage in open, appropriate and effective interactions with people from different cultures, and to act for collective well-being and sustainable development.”

It records that: “Scotland ranked among the top-performing countries in global competence.
– “Scotland was the fourth top-performing country, behind Singapore, Canada, and Hong Kong (China)”
– “Scotland was the third country with the largest proportion of students who scored at Level 5 (12%), behind Singapore (22%) and Canada (15%). This is significantly higher than the average of 4% of students.”

What does Level 5 mean? “At Level 5, the highest level of proficiency in global competence, students can analyse and understand multiple perspectives. They can examine and evaluate large amounts of information without much support provided in the unit’s scenario. Students can effectively explain situations that require complex thinking and extrapolation and can build models of the situation described in the stimulus.”

Elsewhere the report, in a section on ‘equity’, it explains that the extent of socio-economic disparities in academic performance indicates whether an education system helps promote equality of opportunities. It concludes:

“In Scotland, students’ socio-economic status had relatively little impact on their reading performance than other OECD countries.” It notes that in 2018, the socio-economic status as measured by the PISA index of economic, social and cultural status (ESCS) explained only 8.6% of the difference in performance between students from the most and least advantaged backgrounds in Scotland, a smaller impact on their performance in Scotland than on average across the OECD, where the ESCS explained 12% of the difference in performance.

Moreover, the impact of students’ socio-economic status on their PISA performance in maths and science was also smaller in Scotland than on average in the OECD area, explaining 7.9% of the performance difference in maths, compared to 13.8% on average, and 10.1% of the performance difference in science compared to 12.8% on average.

There’s lots more interesting insight and lots more positivity to be found in the report. Indeed there is much more material for positive media headlines that will – of course – never see the light of day!

Source: OECD (2021), Scotland’s Curriculum for Excellence: Into the Future, Implementing Education Policies, OECD Publishing, Paris, https://doi.org/10.1787/bf624417-en

When one examines the results of the last PISA assessment taking account of the OECD’s careful presentation of statistical significance one finds this.

On reading performance: there is NO statistical difference between the scores of England , NI and Scotland. All three, statistically, are ranked significantly higher than Wales.

On mathematics performance: England is ranked higher by a statistically significant margin. There is NO statistical difference between Scotland, NI and Wales.

But as today’s main blog post indicates, PISA is NOT the definitive, NOT the comprehensive, and certainly NOT a universally-endorsed gold standard!

But it has proved a convenient, endlessly-used hook on which to hang unwarranted ‘denigration’. Regrettably, voters who will never read the source material will not become better informed on the findings by relying on the public service broadcaster!

4 thoughts on “Oh dear, not PISA again?

  1. For the Brit Nats and their media, criticising Scotland with misleading stats is a……
    ……………………….. PISA piss.

    Liked by 2 people

  2. O/T.-
    “Boris did not “knowingly” lie to parliament”, says Grant Shapps.
    “He stepped into something”.

    Yup, and I can smell what he stepped into from here!

    Merde, as Papa Johnson might now say—like the native he is not.

    Liked by 1 person

  3. “On mathematics performance: England is ranked higher by a statistically significant margin.”

    Is it not the case that England’s higher ranking was found to be due to flawed data?

    https://www.theguardian.com/education/2021/apr/22/maths-scores-in-world-education-rankings-inflated-for-england-and-wales-study

    “the results and subsequent rankings were distorted because of low levels of participation in the test and a disproportionate underrepresentation of lower achieving students.”

    Liked by 5 people

Leave a Reply to patziwinter Cancel reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.