A screenshot of a book shelf

Description automatically generated

BBC Wales have clearly got an early look at the Pisa results due on Tuesday and they’re bad, very bad:

“It’s not just that Wales has underperformed, it’s seen its performance decline”. That’s the stark assessment of Andreas Schleicher, head of education and skills at the Organisation for Economic Cooperation and Development (OECD). Mr Schleicher spoke to BBC Wales ahead of the publication of the latest set of Pisa results, due on Tuesday. Pisa tests are taken by 15-year-olds in dozens of countries and economies across the world, and Wales has a poor record.

https://www.bbc.co.uk/news/uk-wales-50592611

Presumably BBC Scotland haven’t managed to get a look or, heaven forbid, they’re better than expected. There are rumours that England will do well on reading this time.

Anyhow, it matters not. The Pisa results are useless. No serious academic uses them. Here’s a reminder of why:

  1. Some tests just suit some countries’ education systems. Doing well in one test doesn’t make that country’s overall education system better than that of lower rankers and in some cases, especially in East Asia (China, Korea, Singapore), it is evidence of them being worse in many ways.
  2. The PISA results are based on unreliable estimates with huge scope for error and thus, I quote, ‘useless.’
  3. Summarising a country’s education system in just three numbers is, I quote, ‘madness.’
  4. Comparing countries with radically different cultures and educational structures is meaningless.
  5. PISA does not measure curriculum knowledge just general skills, so the so-called successes of Finland and South Korea or the middle-rank ‘failure’ of Scotland are not based on the quality of their teachers, their schools or their curriculum.
  6. The Finnish system is not that successful in other ways that PISA does not test.
  7. The highly authoritarian, ‘industrial’ East Asian systems are a form of child abuse we surely do not want to see in Scotland.
  8. The East Asian systems are not at all successful in developing the creativity, originality and innovation needed for future success in developed societies.
  9. The East Asian systems, in most cases, brutally abandon children with learning difficulties.
  10. The Scottish system is highly successful in feeding its universities with ever more and better qualified students, it is inclusive, caring and explicitly promotes creativity, originality and innovation.

Some tests just suit some countries

The Finnish system, used to do badly and you could argue they now teach toward the tests to get good results with an emphasis on developing general problem-solving skills rather than curriculum knowledge which, perhaps, the Scottish schools do better? The East Asian schools, as we’ll see below, force-feed their children with highly intensive and long days with disturbing side-effects, but high scores in PISA and in other tests like TIMSS.

Unreliability

With PISA methods only a small number of pupils in each school answer the same set of questions, with these results then used to estimate students’ ‘latent’ ability. According to Professor Spiegelhaller of Cambridge University, this creates a huge scope for error and make the results useless.

Summarising

Dr Hugh Morrison of Queen’s University, Belfast says bluntly: ‘There are very few things you can summarise with a number and yet PISA claims to be able to capture a country’s education system in just three of them. It can’t be possible. It is madness.’

Comparisons

Given the huge range and diversity of educational practice, values, cultures, economies and parent behaviour across countries as alien to each other as, say, Scotland, China, Peru and Tunisia, making comparisons is pointless and perhaps damaging.

Finland and South Korea’s ‘Success’

Finland and South Korea seem quite different at first sight yet both are successful in PISA. Closer examination, however, reveals they have two identical features which make it easier for them to win at the game of PISA.

First, in both languages, words are written just as they are said unlike in other languages especially English. This means language acquisition happens quicker and more successfully across all pupil abilities than in other countries giving their systems more time, earlier, to build on these language skills to develop general problem-solving skills. Notice the English-speaking USA and UK are only middle achievers in PISA.

Second, both cultures are very homogeneous with few migrant groups with other first languages pulling down the scores as they struggle with a new language. It’s for this reason that the otherwise very similar Swedish system appears to do much less well because it has 10 times the immigration rate of Finland.

Why the Finnish System is not so successful

 Like South Korean children, Finnish pupils are at the bottom of the tables in assessments of happiness in school and rarely answer yes to ‘I am happy at school’. Perhaps the Finnish emphasis on group-work and general problem-solving is not so ‘child-centred’ as many in the UK think?

Also, Finland does much worse in the wider TIMSS tests of wider and valuable mathematics knowledge coming 95 points behind China though it is only 2 points behind in PISA.

Child Abuse 

The East Asian systems in South Korea and Shanghai/China are based on gruelling programmes with 13 hour days and only 5.5 hours sleeping time. Social time is not mentioned at all. Professor Zhao of Oregon University has described them as:

‘Glorifying educational authoritarianism and romanticising misery.’

In 2014, the New York Times described South Korea’s system as ‘an assault on children’ and suggested that South Korea:

‘..produces ranks of over-achieving students who pay a stiff price in health and happiness. The entire programme amounts to child abuse.’

Pupil suicide rates are high in East Asia and low in the UK.

Creativity, Originality and Innovation

 Developed and developing countries need innovators along with cadres of skilled workers who can replicate current standards. East Asian companies often have to buy-in Western innovations, staff and start-up companies from places like Scotland. In 2014, The Harvard Business Review said:

‘China’s examination hell; how can students so focused on test scores possibly become innovators?

 Even Finland’s success can be overstated. Its economic innovations have been in the context of only one company, Nokia, and it is now in decline.

Leaving ‘special’ children to languish and fail

According to teacher’s chat-room evidence (the only kind allowed out of these countries), very few East Asian schools make arrangements to support children with conditions such as ADHD or autism in their race for high test ratings and the status that comes with them and that our Labour and Tory politicians shamefully seem to believe.

Scotland

The Scottish system is not perfect. All educational systems are infinitely improvable. It is however, caring, inclusive, and successful with ever greater numbers of exam passes and, critically, for the future, has an explicit commitment to creativity from its government:

‘Creativity is very clearly at the heart of the philosophy of Curriculum for Excellence.’

 Scotland has a long and strong record of innovations, some of them sold to China recently.

Despite Scotland’s frankly more humane, more future-oriented system, it still does respectably well in international tests of little real value. Dips and rises in these, mean little or nothing at all.

Why doesn’t Reporting Scotland tell us any of this? They fail my test.