Taking the PISA

In the Herald today:

THE UK Government’s education secretary has been asked to give advice to Holyrood on teaching practices amid claims Scotland’s standards are “in freefall”. Nadim Zahawi said Scotland’s performance on the international PISA tables, a mechanism for ranking global education standards, has fallen and the situation was “concerning”. 

https://www.heraldscotland.com/politics/20159394.uk-education-secretary-agrees-give-holyrood-advice-claiming-scots-schools-in-freefall/

It’s been some time but TuS has posted several demolitions of these tables. You can see them all at: https://talkingupscotlandtwo.com/?s=Pisa

The best was this from stewartb in March 2020:

1) BBC SCOTLAND’S LATEST EXPERT

I was prompted to research this by a post on Talking Up Scotland (https://talkingupscotlandtwo.com/2020/03/01/gordon-brewer-offers-us-more-guff-on-scottish-education/). I wanted to find out more about Stephen Curran, the ‘expert’ guest of BBC Scotland on 29 February. And helpfully I found this in the Times Education Supplement (TES).

Source: https://www.tes.com/news/progressive-education-leading-scotland-crisis

Mr Curran argues here that the Scottish education system’s “reputation came crashing down when the … (PISA) ratings were published late last year.” He states that Scotland’s attainment in maths and science had taken “another dive” and moreover the PISA scores “… should be a wake-up call to everyone in the UK education system who advocates progressive education.” Keep “crashing down”,  “dive” and this threat from the ‘progressive’ in mind as you read on!

Mr Curran makes much of educational reforms introduced in England by Michael Gove. He appears to link England’s recent PISA scores to the perceived success of these reforms. And Mr Curran is concerned about us: “Scotland is a great country and I hope for its people that the politicians wake up sooner rather than later about the crisis facing it. And it is a crisis.”  Keep this ‘crisis’ and the impact of Gove’s reforms – based on PISA evidence – in mind as you read on!

And not to risk understating his ‘crisis’, Mr Curran ends with this: “The education of young people is the key to unlocking the future potential of a country and its economy. Get it right and everything else will fall in place – getting it wrong can only lead to stagnation and failure.” Some might take the view that Mr Curran has now quite evidently ‘jumped the shark’!

(As an aside, Mr Curran’s article does have some comic value. He tells us that Curriculum for Excellence: “… is a move away from the more traditional approaches that characterised Scottish education and produced such great minds as Adam Smith, Andrew Neil, Michael Gove, Gordon Brown, Duncan Bannatyne – to name but a few renowned people .…” Now just ponder for a moment –  this ‘expert’ is arguing that Adam Smith (born 1723) and Duncan Bannatyne (born 1949) experienced similar educational systems – now that would be one very conservative, ‘traditional’ system! – and that they can be grouped together based on their ‘renown’!!!)

Let’s now build a counter view on PISA step by step.

2) PISA – MANY QUESTIONS

This is from the considered writing of Henry Hepburn (4 December, 2019), news editor for the educational weekly newspaper, TES Scotland:

Source: https://www.tes.com/news/pisa-results-2018-what-happened-scotland

“Every three years, a new round of PISA scores is released into the world. While MANY QUESTION whether the Programme of International Student Assessment should have the level of influence that it does and how reliable the data is, there is no doubt that the results influence government policy. And when things go badly, that they become AMMUNITION to use against governments.” (my emphasis)  But Mr Hepburn’s ‘many question’ caution is putting it mildly!

3) LONGSTANDING INTERNATIONAL CONDEMNATION

Adverse criticism of PISA goes back some time. The Guardian on 6 May 2014 published an open letter to the OECD under this headline:

‘OECD and PISA tests are damaging education worldwide – academics: In this letter to Dr Andreas Schleicher, director of the OECD’s Programme for International Student Assessment, academics from around the world express deep concern about the impact of PISA tests and call for a halt to the next round of testing’

The link below has the full text of the letter: it is worth visiting to check out the list of ‘expert’ signatories – extensive and impressive: I suggest ‘weighing’ this against BBC Scotland’s guest Mr Curran’s credentials for declaring “yes it is a crisis” based on PISA scores.

Source: http://archive.is/DqNNE 

In a lengthy, damning critique, the letter states: “We assume that OECD’s PISA experts are motivated by a sincere desire to improve education. But we fail to understand how your organisation has become the global arbiter of the means and ends of education around the world. OECD’s narrow focus on standardised testing risks turning learning into drudgery and killing the joy of learning. As PISA has led many governments into an international competition for higher test scores, OECD has assumed the power to shape education policy around the world, with no debate about the necessity or limitations of OECD’s goals. We are deeply concerned that measuring a great diversity of educational traditions and cultures using a single, narrow, biased yardstick could, in the end, do irreparable harm to our schools and our students.”

The signatories also contend: “These developments (ones arising from PISA) are in overt conflict with widely accepted principles of good educational and democratic practice:

  • No reform of any consequence should be based on a single narrow measure of quality.
  • No reform of any consequence should ignore the important role of non-educational factors, among which a nation’s socio-economic inequality is paramount.”

4) PISA – REJECT IT OUTRIGHT

In 2016 the TES reported a proponent of this action.

Source: https://www.tes.com/news/ignore-pisa-entirely-argues-top-academic

‘Countries should “IGNORE” the world’s most influential education rankings because they fail to measure what matters, an expert on the impact of globalisation on education has claimed. The idea of nations competing to reach the top of the … (PISA) league tables makes as much sense as university students competing to see who can drink the most beer, according to Professor Yong Zhao, from the University of Oregon in the US. He told TES: “You’re maybe the best drinker but you’ve got to think, ‘Is it good for you and does it matter?’”

According to Zhao, PISA homogenises education systems: ’We need creativity, not uniformity’.

5) PISA – SO WHAT?

The following perspective appeared in Forbes magazine under the headline: ‘PISA Scores Are Out. Ours Are Not Great. So What?’

Source: https://www.forbes.com/sites/petergreene/2019/12/03/pisa-scores-are-out-ours-are-not-great-so-what/#1e2734aa5e84

It seems there is also lot of hand wringing in the US each time the PISA results are published. The author of the article (Peter Greene) argues: ‘PISA coverage tends to overlook one major question—why should anyone care about these scores? Where is the research showing a connection between PISA scores and a nation’s economic, political, or global success? What is the conclusion to the statement, “Because they get high PISA scores, the citizens of [insert nation here] enjoy exceptionally good______” ?’

The author asks: “Did US companies outsource work to India and China because of their citizens’ PISA scores, or because of low wages and loose regulation? Do we have the world’s most expensive health care system because of mediocre PISA scores? Which politicians have ridden to success on the PISA score platform pony? Are any geopolitical conflicts solved by whipping out the contending countries’ PISA scores for comparison? And is there a shred of evidence that raising PISA scores would improve life for US citizens (spoiler alert: no)?”

6) WHAT SCOTLAND’S TEACHER UNIONS ONCE SAID ABOUT PISA

Closer to home, once upon a time (4th December 2013), the teacher unions in Scotland expressed their views on the status and significance of PISA results.

Source: https://www.eis.org.uk/Education-Updates/PISA-Results

The EIS and the SSTA urged caution in how these data are viewed.  Whilst both unions believed that PISA data offered “some interesting areas for debate” they added “ITS WORTH IN MEASURING THE VALUE OF ANY COUNTRY’S EDUCATION SYSTEM IS LIMITED”. The unions’ 2013 statement goes on:

“The EIS and the SSTA urge the Scottish Government and Opposition Parties to GLANCE BRIEFLY at this data but continue to concentrate on the development of meaningful curriculum reform in Scotland aimed at the creation of a fair and just society.”

Commenting further, EIS General Secretary Larry Flanagan noted at that time, “While this (PISA) report seeks to measure performance in school education there are SIGNIFICANT QUESTIONS OVER THE TESTING METHODS used to gather data and the degree to which like is being compared with like. The circumstances in different countries vary widely, and individual nations take significantly different approaches to education in terms of how schools are organised, funded and run.  There is also concern that, in some countries, a heavy emphasis is placed on preparing pupils to perform well in these tests specifically to boost PISA rankings.”

The Acting General Secretary of the SSTA at the time commented, “These studies have become LITTLE MORE THAN AN INVITATION TO BEAT OURSELVES UP UNNECESSARILY. While the scale of the exercise is immense given the number of nations involved the scope is limited, the focus only on narrow performance outcomes in three areas merely encourages certain nations to concentrate on the rapid ascent of the league tables without reference to the wider exercise of good educational practice. Yet again we are reminded of the need to measure what is valuable not value what is measurable.”

So a question today for the EIS and SSTA leaders, what has changed regarding the status and significance of PISA results since 2013?  Well, on the latest PISA results, the EIS response included (just!) this:

Source: https://www.eis.org.uk/Latest-News/PISA19

“The analysis of Scotland’s data from the … (PISA), published today, has highlighted the value of Scotland’s comprehensive education system. Data from the PISA survey indicates a SOLID PERFORMANCE overall, with IMPROVEMENT in reading and with results in maths and science HOLDING RELATIVELY STEADY.”  So, has the EIS not noticed the crisis of which Mr Curran writes and broadcasts on BBC Scotland?

The recent EIS statement notes that the PISA survey also “indicates that the gap in performance between the most and least disadvantaged pupils was smaller in Scotland than across the average of other OECD countries – indicating the Scotland’s comprehensive system offers greater equality of opportunity to pupils from all backgrounds.” But, but what about the ‘crisis’ ….?

And finally from the EIS on this: ”It is clear, however, that Scottish Education is actively working on the key elements identified by the OECD as necessary for continued improvement – tackling the impact of deprivation on educational attainment, ensuring high standards of teaching, and being concerned about pupil well-being.”

7)  ABOUT THE LATEST PISA RESULTS

The TES reported the views of Mark Priestley, professor of education at the University of Stirling.

Source: https://www.tes.com/news/listen-teachers-gain-clarity-around-curriculum-aims

In a measured contribution to the debate over Scotland’s performance, firstly Professor Priestley is reported as stating: “.. we don’t have the evidence to say whether Curriculum for Excellence (CfE) is responsible for the decline or otherwise in PISA results,” This seems in complete contrast to the views of Mr Stephen Curran.

Professor Priestley is reported as adding: ‘Today’s PISA results may have indicated some “slight improvements” in reading, although “maths and science seem to be SITTING MUCH WHERE THEY WERE”. This can’t be right – there is a ‘crisis’, someone on BBC Scotland told me so!

Also in the TES:

Source: https://www.tes.com/news/are-scottish-and-english-education-closer-thought

Andreas Schleicher, head of the Directorate of Education and Skills at the OECD – top official for PISA –  is reported as stating that the differences in both education approaches and PISA scores across the UK are marginal when set against the international picture.

8) TEACHER UNION VIEWS IN ENGLAND ON PISA RESULTS

Given Curran’s support for a ‘traditional’ approach to education and his endorsement of, and claims of early success for, the Gove reforms in England, it is interesting to note comments on the latest PISA results by the main union in England, the NEU.

Source: https://neu.org.uk/press-releases/pisa-results

Kevin Courtney its Joint General Secretary has said: “The findings from PISA 2018 ….. REFLECT A EDUCATION SYSTEM BEFORE THE MARKET REFORMS introduced by former Education Secretary Michael Gove and pursued by subsequent Secretaries of State.”

He goes on: “Each time the survey is conducted, a larger number of countries take part and therefore THE GLOBAL RANKINGS DO NOT COMPARE LIKE WITH LIKE. In addition, each PISA series focuses on a different learning domain – in 2018 it is reading, in 2015 it was science. The ‘league table’ element of PISA should therefore be TREATED WITH ENORMOUS CAUTION since the rankings are comparing different subjects among a different number of countries.  The OECD also point out that many of the changes in countries’ scores (and therefore their apparent ranking) are NOT STATISTICALLY SIGNIFICANT. This includes two of the three scores given to the UK.”

9) AND DOES PISA/OECD SUPPORT A RETURN TO ‘TRADITIONAL’ EDUCATION ANYWAY?

In November 2019, the TES reported the NEU teaching union’s joint General Secretary saying this on the impact of educational reforms: “I think we have to be cautious in attributing any reform in any national state, four years later, to a position in a league table, because the position on the league table is not just about the curriculum that’s taught in school or the way children are assessed.”

Source: https://www.tes.com/news/will-tuesdays-pisa-results-be-goves-report-card

On PISA results, the General Secretary goes on: “It’s difficult to disentangle how much curriculum has played a part, how much exams and assessment have played a part, how much those parts of education policy actually played a part. I will say however that WHAT IS ABUNDANTLY CLEAR IS THAT GOVE’S DIRECTION OF TRAVEL IS COMPLETELY OPPOSED TO what the Organisation for Economic Cooperation and Development, which is behind Pisa, is saying about what effective education systems should look like.”

And this view is supported by the OECD’s review of Scotland’s Curriculum for Excellence in 2015. In this the OECD endorsed the progressive nature of the ongoing process of reform in Scotland’s schools whilst challenging aspects of its implementation.

See: http://www.oecd.org/education/school/Improving-Schools-in-Scotland-An-OECD-Perspective.pdf

According to the OECD: “The Curriculum for Excellence (CfE) is an important reform to put in place a coherent 3-18 curriculum. It privileges learning and a holistic understanding of what it means to be a young Scot growing up in today’s world.”

10) AND FINALLY …

I cannot resist recalling comments in 2016 by a former education spokesperson for the Scottish Conservative and Unionist Party following the publication of earlier PISA results. Liz Smith MSP was quoted as saying: “These shocking statistics are a damning indictment of A DECADE OF FAILURE under the SNP.”

Source: https://inews.co.uk/essentials/news/education/scottish-education-system-no-longer-world-leading/

But then in response to a UCAS report also in 2016 on university admissions, the same politician claimed this: “Despite having OUTSTANDING GRADES, many Scottish students are finding it increasingly difficult to get into universities because of the constraints applied via the capping policy.”

Source: http://www.bbc.co.uk/news/uk-scotland-38316717

So just to recap, the Tory position seemed to be that: (i) a decade of educational system failure – as evidenced by a set of PISA results, was (ii) delivering outstanding grades, and (iii) had led to an increasing demand for a university education – all at the same time!  Go figure! 

Let’s hope the new Tory spokesperson on education, Mr Jamie Greene MSP can make more coherent contributions to what is a necessarily ongoing debate over continuous improvement in Scotland’s school system.

9 thoughts on “Taking the PISA

  1. Clearly Education is your field of expertise, yet I’m puzzled by the premise…

    Hannah Rodger created this latest nonsense on the basis of what appears to have been an orchestrated exchange between the Liberal Democrat MP for Caithness, Sutherland and Easter Ross Jamie Stone, and the Minister who has no idea who arranged what over the Sue Gray enquiry and has no invitation to advise SG even if Jamie Stone is best mates with Stoned Jamie at Holyrood security.

    Perhaps because it’s PISA I’m leaning toward (I’ll get my coat) them opening yet another front on the Scotland is Shite into edumacat/educit/learnin’ as extension to their current Ferries, Rail, bus passes inventions which have been broadly met by

    Liked by 3 people

  2. Micky Gove?
    Of the nodding head, sparkly eyes and multiple personalities——in charge of English childrens education?
    A day in the life of a fantasist———

    1. “I endorse Boris for PM”.
    2. “No, don’t vote for Boris—he is not fit to be PM”.
    3. “Aha—-Boris will be a GREAT PM”.
    A tired and emotional Govey covers all bases.

    The fine Colombian marching powder is ubiquitous at Westminster.
    Not that it’s obvious, of course.

    Liked by 4 people

  3. Oh dear, not PISA again? This supposed hammer blow to the SNP government and to the credibility of Scotland’s school system is becoming tiresome – and the claims based solely on it are just plain wrong and for lots of reasons. The claims are IMHO often driven by a political agenda.

    I venture the alarmist claims also run counter to most parents’/carers’ and school students’ direct experience of our schools. Could Scottish education be better? Of course – it would be daft to deny this for any education system as it would to deny it for any area of public policy in a democracy.

    I wrote the following btl for Talking up Scotland on 25 June, 2021 to make good the media deficit on what the OECD itself – the organisation which delivers the PISA tests – wrote positively about Scotland’s schools system in 2012. Here are some bits of what I wrote then:

    ‘… the OECD report provides a much more measured, balanced account of Scotland’s PISA results – and surely being in an OECD report, we can take this account as more authoritative than many others? I offer this as two among many positives from across the OECD report.

    The report explains that PISA’s new global competence module “aims to capture the capacity of 15-year-olds to examine local, global and intercultural issues, to understand and appreciate the perspectives and world views of others, to engage in open, appropriate and effective interactions with people from different cultures, and to act for collective well-being and sustainable development.”

    It records that: “Scotland ranked among the top-performing countries in global competence.
    – “Scotland was the fourth top-performing country, behind Singapore, Canada, and Hong Kong (China)”
    – “Scotland was the third country with the largest proportion of students who scored at Level 5 (12%), behind Singapore (22%) and Canada (15%). This is significantly higher than the average of 4% of students.”

    What does Level 5 mean? “At Level 5, the highest level of proficiency in global competence, students can analyse and understand multiple perspectives. They can examine and evaluate large amounts of information without much support provided in the unit’s scenario. Students can effectively explain situations that require complex thinking and extrapolation and can build models of the situation described in the stimulus.”

    Elsewhere the report, in a section on ‘equity’, it explains that the extent of socio-economic disparities in academic performance indicates whether an education system helps promote equality of opportunities. It concludes:

    “In Scotland, students’ socio-economic status had relatively little impact on their reading performance than other OECD countries.” It notes that in 2018, the socio-economic status as measured by the PISA index of economic, social and cultural status (ESCS) explained only 8.6% of the difference in performance between students from the most and least advantaged backgrounds in Scotland, a smaller impact on their performance in Scotland than on average across the OECD, where the ESCS explained 12% of the difference in performance.

    Moreover, the impact of students’ socio-economic status on their PISA performance in maths and science was also smaller in Scotland than on average in the OECD area, explaining 7.9% of the performance difference in maths, compared to 13.8% on average, and 10.1% of the performance difference in science compared to 12.8% on average.

    There’s lots more interesting insight and lots more positivity to be found in the report. Indeed there is much more material for positive media headlines that will – of course – never see the light of day!

    Source: OECD (2021), Scotland’s Curriculum for Excellence: Into the Future, Implementing Education Policies, OECD Publishing, Paris, https://doi.org/10.1787/bf624417-en

    When one examines the results of the last PISA assessment taking account of the OECD’s careful presentation of statistical significance one finds this.

    On reading performance: there is NO statistical difference between the scores of England , NI and Scotland. All three, statistically, are ranked significantly higher than Wales.

    On mathematics performance: England is ranked higher by a statistically significant margin. There is NO statistical difference between Scotland, NI and Wales.

    But as today’s main blog post indicates, PISA is NOT the definitive, NOT the comprehensive, and certainly NOT a universally-endorsed gold standard!

    But it has proved a convenient, endlessly-used hook on which to hang unwarranted ‘denigration’. Regrettably, voters who will never read the source material will not become better informed on the findings by relying on the public service broadcaster!

    Liked by 3 people

    1. aargh! – ‘wrote positively about Scotland’s schools system in 2012’ – should be 2021 of course!

      Like

  4. The headline PISA scores are calculated using a computer algorithm something that is not widely acknowledged or in fact known. After the debacle of exam grades calculated using a computer algorithm during the pandemic you would think people would be a little more wary of marks/scores generated by this method but the PISA scores are too convenient a rod with which to beat the SG.

    As the article mentions and some of the posts above emphasise the OECD reports, eg 2015 and 2021, give a much fuller, nuanced and positive picture of Scottish education. But God forbid journalists and opposition politicians would acknowledge that.

    For information here are the PISA scores from 2000 onwards. Note the fall from 2000 to 2006 in Reading and Maths before the SNP came to power and the more or less stable position thereafter. This was something acknowledged in their 2015 report. Finally the science scores fell in 2015 & 2018 but so did the Global average which suggests the OECD may have tweaked the algorithm.

    Year, Maths, Reading, Science
    2000, 533, 526, 522
    2003, 524, 516, 514
    2006, 506, 499, 515
    ………………………………………….
    2009, 499, 500, 515
    2012, 498, 506, 513
    2015, 491, 493, 497
    2018, 489, 504, 490

    The Curriculum for Excellence was adopted by the Lab/LibDem coalition in 2004 and rolled out first to Primary schools and then into Secondary schools around 2010.

    Liked by 3 people

  5. Can I ask who has asked the English education secretary to give ‘advice to Holyrood ‘?
    It would be funny if it wasn’t so serious a threat to Scotland’s well established and very successful education system which is not devolved but independent of the English governments’ jurisdiction. Another attempt to undermine the actual competency of the Scottish government and the agenda is quite clearly to legitimise any planned take over of independent areas of the Scottish government such as education, health, law and infrastructure, using the English governments’ internal market act. With that comes assimilation, Scotland a region of Northern Britain.

    Scotland has two choices ahead, independence or direct rule by the English government.

    Liked by 2 people

  6. The only major achievement in and contribution too education in England by Gove was to vanish the title deeds for just about all school properties, which very like NHS properties have been bundled for sale to the private sector or more likely just handed over. The levelling down process is in full swing and in Scotland there are so many opportunities to make an unearned buck or 2.
    Like ArtyHetty I want the name of the presumptuous ill informed toerag who dared to invite more presumptuous ill informed toerags to interfere in our education system.

    Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.