‘The SQA has done its job as fairly as it could’

https://www.sqa.org.uk/sqa/files_ccc/SQAAwardingMethodology2020Report.pdf

From Alasdair Macdonald

The SQA ‘Technical Report – National Qualifications 2020 Awarding — Methodology Report’ requires a fair bit of concentration and close examination of the data to comprehend it fully.

My feeling is that in these unique and unprecedented circumstances, the SQA has done its job as fairly as it could and has erred slightly on the side of leniency. I think that they have explored all the possibilities in some depth and have settled on a methodology which is the best that could have been done in the circumstances. The SQA staff involved have faced up to a very difficult task and have acted transparently and with a great deal of professional rigour and courage. They deserve our thanks and commendation.

The appeal system has still to take place and we will see how this is tackled.

I think the most striking statistic is the percentage of teacher estimates – in schools across the socio-economic spectrum – which indicate the top grade. This is more than 10% above the previous historic data, based on the examination system. Although the other ‘pass’ grades are slightly higher than the historic data, it is in the top grades that there is the greatest disparity. It is also noteworthy that in the historic data, there is evidence of teacher estimates of awards at the top grades being consistently greater than the actual top grades awarded. The historic data was based on the full range of moderation data being available.

Clearly many young people, through no fault of their own, have had their expectations raised and are understandably upset and feel aggrieved.

Although most of the adjustments in grading made by the SQA have been to reduce the grade, approximately 1 in 11 had the grading increased. Most of the reductions in grade were to the grade below and so most did, in fact, get a ‘pass’, albeit at a lower level than they had been led to expect.

Also, many students actually got the awards predicted, but one or two were at a lower level. For example, one of the students (who spoke very well), indicated she had been predicted to get 2A awards and 2 Bs, but attained 2As, 1B and 1C. This is a very fine achievement, but the ‘slippage’ of one grade has occurred for many students every year over many decades. Some of these, in the past have been restored on appeal.

It seems to me that the problem is rooted in how teachers arrived at their estimates and the procedures used within the subject departments to moderate the reliability of these, the moderation procedures within each school and the moderation procedures of the Councils’ Education Departments.

The Appeals system will require the subject teachers to provide further evidence to justify their original estimates. I hope for the young people that such evidence is persuasive in many cases, but, I fear, in most, it will not.

We have a set of results which is a little more generous than in previous years, but is not so different from previous years as to cast doubt of the validity of the awards for ALL students in this particular year.

Much of the reporting and outrage is focussed on the results for young people at schools serving areas of higher levels of multiple deprivation. Sadly, for societal and economic reasons beyond the ability of schools alone to deal with, the aggregate levels of attainment has tended to be lower than those forScotland as a whole. The ‘gap’ between such schools and others has been narrowing, fairly steadily over the years and, in 2020, that narrowing has continued.

I sat my Highers in 1965 and attained 5As and 1B. In these years, the school system was ‘selective’ and I was one of the 35% who had passed the ‘Qualy’ and attended a Senior Secondary. I was born and raised in an area of multiple deprivation and for a number of years my family were ‘on benefits’. I also taught for 39 years and the 7 secondary schools in which I taught were all in the lower half of Scottish schools ranked by levels of deprivation. 2 of those schools were in the 10% of ‘most deprived’. Every year, in everyone of those schools there were students who attained the highest awards and many others who defied very difficult circumstances to attain very good, although not the highest grades. If there had been ways of having endeavour and application and courage and fortitude recorded on certificates, there are several thousand young people, whose certificates I would have endorsed gladly. We did, of course, provide references for them, but these never had the cachet of SQA certificates.

I have also written examination questions for the SQA, marked examination papers and moderated samples of scripts for SQA. I presented students for examination in every one of those years. Most of my predictions – but not all – were fairly close to what was awarded and I had similar proportions of successful appeals to the national average. Sadly, I did have one year, when my predictions were out of kilter by a fair amount. But, over the course of my career, the general trend of attainment by my students was upwards. There were fluctuations.

Finally, in any school the variation in the attainments across departments can be wide – i.e. students attain poorer grades in a few departments compared to their attainments in other subjects. These differences are often statistically significant and appear year after year. Such variations are evident in schools right across the spectrum. However, in the more affluent areas they are off-set by parents being able to afford private tutoring. In less affluent areas schools can sometimes squeeze money from their budgets or obtain grants to provide some form of ‘supported study’ and many teachers voluntarily provide lunchtime or after school help. But, the problem is societal. It is also political.

Sadly much of the current hoo-haw is political not to change society, but to score cheap political points.

The SQA has provide voluminous information on what they did. Let us hear from the Conservatives, Labour, Lib Dems, Greens and others what they would have done in these unprecedented times.

Congratulations to the students for their awards and congratulations, too, to those who have been protesting. Many have made their case with dignity and eloquently.

14 thoughts on “‘The SQA has done its job as fairly as it could’”

  1. Standing back a little from all the contrived controversy of the moment I have to ask if the SQA may, in fact, have come upon a fairer way to assess school pupils’ attainments?

    Liked by 2 people

  2. Indeed Alasdair, read your synopsis earlier and meant to commend it’s honesty and balance, stark contrast to the shit-fest that passes for “journalism” these days.

    Liked by 2 people

    1. Well said Bob.

      The BBC’s Bill Whatford is continuing the “shit-fest” interviewing two contributors both attacking the SQA and Scot Gov. Labelled by Whatford as ” a fascinating discussion”

      Liked by 2 people

  3. “Sadly much of the current hoo-haw is political not to change society, but to score cheap political points.”

    This is such a key point.

    I have great sympathy with pupils who feel cheated by a system, but there is so many bad faith political (adult) actors who frankly aren’t helping.

    The problem (which I think you are touching upon) is that this years contingency is simply replicating as fairly as possible what usually happens. The exam system has been awarding poorer pupils with lower grades for years. One of the reasons for this is that in an exam system each pupil is competing against every other candidate sitting the qualification. There is no set pass mark, this varies each year depending on performance.

    This year with no exams but the grading structure having to be retained it has exposed the system at play. If young people and adults are seriously concerned about this situation, then it’s not going to be solved in a few months over a pandemic. We’ll need to devise a new qualification system.

    That will also include what qualifications are actually for. They’re a system of showing ‘ability’ to others. If you can get X grade in Y qualification you can do Z.

    Personally, I think for key competencies/qualifications a pass/fail would suffice (the old Standard Grade system of separate Credit/General/Foundation papers was maybe more effective at showing different standards of competency.

    Anyone know of any examples from around the world that offer a (non-pandemic) solution to the fundamental problem?

    Liked by 2 people

    1. Norm, the exam system is not competitive and has not been for the past 40 years.

      The system in which I was educated was a selective system and a pre-defined proportion – 35% – were deemed suitable for more advanced secondary school education, and an even smaller proportion of them (c7%) entered Higher Education. This system was competitive and was called — NORM referenced! It discriminated against young people from less affluent backgrounds and people like I who managed to get through were described as ‘getting a foot on the ladder of opportunity’ and of becoming ‘socially mobile’.

      It had been obvious for many decades prior to then that this system was tremendously squandering of talent. Both World Wars showed that millions of women and working class people had as much talent as those who attended ‘top’ schools (which of course buy privilege). Even amongst Conservative politicians it was recognised that the education system was wasteful of talent and hindering the social and economic development of society as a whole. During the 1950s there was a fair degree of consensus for educational which eventually produced Comprehensive schools for the 1970s where every child as a right had entitlement to the full range of the curriculum. Very quickly, large numbers of children who in previous generations would have been consigned to a very restricted Junior Secondary Curriculum, were now achieving very highly.

      The exam system was also changed to what is called ‘criterion referenced’, which defined standards to be attained and, if these were attained a pass was awarded. An example of this is the Driving Test. So, it was not competitive. The Standard Grades to which you refer were a good example of this.

      Because increasing numbers of ‘plebs’, like us, were proving ourselves at least as able as the offspring of the wealthy, the latter saw their pastures being ‘threatened’ and continually denigrated the system, with terms and ‘straw men’ like ‘dumbing down’ , ‘all shall have prizes’, etc. Sadly, there were always some teachers who were only too ready to shout about the ‘chaos’ in schools and the teacher unions were not averse to using this in pursuing wages and conditions and also to avoid accountability. (I was a teacher union member for my entire career!) Many of the most voluble critics of the system were on the extreme left, and we see some of them behind the current stooshie. They are examples of Lenin’s ‘useful idiots’ as we see from the alacrity with which the right wing media and politicians have adopted their rhetoric for short term gain.

      Liked by 1 person

  4. Thank you Alasdair for that summation. On a more general point, any time the collaborators believe they scent a weakness in Scottish Government policy, even though none exist, they, assisted by their fellow-travellers in the media, try to convince the public that they could have done better, despite never revealing their plans. Now a motion of No Confidence in our Education Secretary is being proposed by the Labour Party, backed of course by their co-conspirators, the Tories. John Swinney, and the S.Q.A, should stand firm, as neither I believe could have done any better, given the unprecedented circumstances under which they had to operate.

    Liked by 1 person

    1. Agreed Alex, Swinney and the SQA should not be demonised for this stooshie. What alternatives were available to them? Holding normal exams in lockdown clearly wasn’t an option, nor was holding everyone back for a year to complete the course and sit the exam (just imagine the outcry if that had happened!)

      Instead the solution chosen was the best option. The outcome shows credible increases in pass rates and a meaningful rise in passes for children from deprived backgrounds. Had the teachers’ original recommendations been adopted without moderation, the outcome would not have been credible and the backlash about “lowering of standards”, “cheapening qualifications” etc would have been just as deafening, febrile and misguided.

      Liked by 1 person

  5. Thank you, Alasdair.

    The hypocrisy of politicians, particularly Labour ones, on full display over this issue. On Reporting Scotland last night they interviewed a Labour Councillor from North Lanarkshire who was venting on the issue. This was the Council which in 2017 sacked 198 classroom assistants in a fit of pique because the SG was giving money directly to head teachers in NL schools to allow them to help pupils achieve their potential. The Council clearly wanted the money paid directly to them so that they could spend it/dole it out as they saw fit and not necessarily on Education.

    Hypocrisy thy name is Labour.

    Liked by 2 people

  6. Now that Nicola has apologised for supposedly getting this wrong and John Swinney seems to be intending to revert to the teachers’ estimated grades, will this mean 2020 results are skewed and unreliable?
    I’d be interested in your views of these developments in light of your undoubted knowledge, perhaps worthy of a separate blog?

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.