Why exam pass rates tell you next to nothing about the competence of politicians, SNP or otherwise

Why is this book here? See at the bottom.

Ed: I hope the headline makes some sense in the light of what seems like too useful a comment to be left ‘at the foot’. There is no functioning editorial process here. No staff and little time.

From reader Graham

The level at which a pass is awarded is not fixed but changes from year to year. In fact “pass level” is an almost fictitious entity. Many years ago I marked Standard Grade English. At the Examiners’ Meeting it was explained that the pass mark was modified each year to take account of externalities: ie the different cohort, the different exam which might be easier or harder than the previous year, different markers, different teachers and so on. (though they tended to stress different level of difficulty) So they fudged the “pass mark” so that more or less the same percentage passed each year. Of course, this is complete nonsense. It’s simply unknowable whether any one exam is more or less difficult than another, whether one cohort is more or less stupid than the previous cohort – but you have to do something, and make a load of silly assumptions.

This “adjustment” still occurs: “It would not be fair to get a higher grade simply because the exam was easier than intended, or a lower grade because it was more difficult than intended. If this happens, adjustments are made to the grade boundaries to ensure fairness for all candidates, and that the national standard of the course is maintained.” (https://www.sqa.org.uk/sqa/files_ccc/SQASettingTheGrade2018.pdf)

Therefore how it happens that the overall pass rate can improve from one year to the next, or, in this case crash so catastrophically that the entire SG must resign is beyond my simple understanding.

Of course, if the opposite had happened, and grades had improved “catastrophically” we would hear the BBC bellowing “Grade Inflation” and hearing calls for the entire SG to resign.

Footnote: TuSC is developing into a ‘public spherule’, a place were all can speak, of a quality that great German champion of democratic society, Jurgen Habermas would be proud of.

I love it but I’m beginning to worry that I’ll miss your contribution. If you think I have, forget your self-doubt and send me a reminder!

4 thoughts on “Why exam pass rates tell you next to nothing about the competence of politicians, SNP or otherwise

  1. What also has to be taken into account in recent exam cycles is that the length of the exams has been increased and the format of the papers, in some cases, changed to take account of the fact that unit assessments have been removed. These changes were phased in starting in 2017-18 and will be completed in 2019-20 when the Advanced Highers move to the new format. Exams have been extended by 30 mins and in some cases as much as 40 min.

    Liked by 1 person

    1. Intuitively it makes sense that the changes to the exam process – removal of units of assessment and greater weight/longer length associated with end of year exams – could have significant impact. As a non-specialist on education matters, I get more confused the more I dig into these matters. For example, see the following:

      Firstly from the main teaching union, the EIS: “.. the aim of last session’s industrial action was the removal of mandatory unit assessments in the interests of workload reduction ..”

      Source: https://www.eis.org.uk/Changes-To-Quals/SQA

      Then this from the NASUWT on the decision to end compulsory units of assessment: “The NASUWT welcomes the proposals as a first step in reducing workload … .” And: “The NASUWT also welcomes the decision to include Advanced Higher as part of the process of change.”

      Source: https://www.nasuwt.org.uk/uploads/assets/uploaded/31f0f103-5c21-4e5f-9dfa69092470b72d.pdf

      So far so clear – unions have variously taken industrial action, advocated and supported the ending of units of assessment in order to reduce the workload of their teaching members.

      But then there is this from the same NASUWT source: “The proposal to remove Unit Assessments, whilst broadly welcomed, seems to ignore the view of some teachers (particularly in Sciences) that these are still useful.“ So is the union not so fully supportive of their removal? What does it actually want?

      And finally this from the EIS: “The EIS is in favour of assessment models within SQA qualifications which are not wholly dependent on final examinations. Assessment models based solely on exams are known to be disadvantageous to students from poorer socio-economic backgrounds and are not the best means by which to assess the skills and knowledge learned in all cases.” So because of workload concerns has the union placed these professional, educational principles to the side?

      Source: https://www.eis.org.uk/Changes-To-Quals/EnglishFolio

      Confused? Come on BBC Scotland perform your public service role and help us all to be better informed – help us all see through the partisan guff – on these important matters!

      Liked by 1 person

      1. Stewart,
        A working party was set up in January 2016 to look at the issues of workload and National Qualifications. Membership was made up of the EIS, and SSTA unions, various parent groups and the usual acronym stew of education quangos. The teaching unions were adamant they wanted rid of unit assessments but were told that it could not be done overnight because it would impact the integrity of the exams. The teaching unions balloted their members on industrial action in pursuit of their objective of getting rid of UAs. I think the SSTA carried out some sort of work to rule.

        Over the summer of 2016 the various education bodies worked on the issue and put a series of recommendations that would allow scrapping the UAs to Mr Swinney which he accepted. He then announced this a a teachers’ conference in Glasgow that September. The teachers were delighted and hailed it as a victory for common sense. But within a short time they decided it did not address their workload issues and this was before it had even been phased in.

        From what I can make out the UAs were a bit like the Continuous Assessment system which most Universities now use. Under that system course work counts for 40% of the final mark and the exam for 60%. It takes the pressure off the exam to a certain extent, allows deeper learning of course content and allows students to develop transferrable skills such as researching a topic and the gathering and presentation of information. More than a few of the current teaching workforce will have studied at a University using such a system and know its benefits but still they set their face against it when it came to their pupils.

        …..

        September 2016 press release extract Scot Gov
        “”A strengthened final exam and externally assessed coursework will replace mandatory unit
        assessments for qualifications at National 5 and Higher level under proposals outlined by Deputy
        First Minister John Swinney today.
        The proposed reforms will significantly reduce teacher workload ensuring teachers have more time
        to teach by ending teacher-led unit assessment. Instead, certification will be entirely based on a
        strengthened final exam and, in some subjects, externally assessed coursework.
        The planned changes are the result of the work between the Scottish Government and partners on
        the Assessment and National Qualifications Group and will now be taken to the Curriculum for
        Excellence management board for further detailed discussion.””
        …………?…………………….
        From Sept 2016
        “”Scottish teaching union says the move to scrap unit assessments at National 5
        and Higher is a victory for ‘common sense’””
        “”Controversial assessments associated with school qualifications – criticised by both
        teachers and parents – will be scrapped, the education secretary has announced.””
        https://www.tes.com/news/school-news/breaking-news/controversial-exam￾assessments-be-scrapped-over-workload-concerns

        Like

  2. When the O Grade system was set up in the early 1960s it was intended that 35% would be awarded a ‘pass’. It was a filtering/selection mechanism, like the old “Qualifying Examination” which selected the 35% who would go on to attend Senior Secondary Schools. It was about putting people in their place.

    However, over time, it was increasingly difficult to set boundaries which identified the ‘top 35%’, because more pupils were scoring the kinds of marks which, historically, would have been deemed passes. Also, examiners had got better at writing questions which tested more rigorously, what the curriculum set out rather than students’ ability to meet the idiosyncrasies of the question writers. The questions became much more objective (NOT to be confused with ‘multiple choice’ tests).

    This marked a shift to ‘criterion referenced’ examinations, where learning outcomes were specified, i.e. students knew what they were trying to learn and teachers knew what they were to teach. Of course, it was not perfect, because it could lead to ‘teaching to the test’ and could be criticised in that education is not just about objectives that can be specified in behavioural terms (to use the terminology.) A good example of criterion referenced assessment is the Driving Test. Nevertheless, this is not a reason to reject it, because, on the whole, it led to improvements in attainment. It meant that other aspects which were not assessable by ‘pencil and paper’ methods began to be tested by appropriate methods, usually, by internal assessments in schools, with external moderating procedures to check teachers’ judgements. These internal assessments led to increases in the level of attainment, particularly amongst children from areas with a high Scottish Index of Multiple Deprivation (SIMD).

    This had two principle attackers, for entirely selfish group reasons. One was teachers, who began grousing about ‘workload’ and, as Ledgerwood has pointed out, this led to the ending of internal assessments in many subjects and longer final examinations. The other was from the families who used the private education sector and those who attended local authority schools in the affluent (low SIMD areas), because increased attainment by the plebs meant that their children were facing increasing competition for places in higher education.

    Historically, this was not a simple partisan party political issue, because many Tories supported wider access to education because they recognised that the bipartite system was dreadfully wasteful of talent. They supported comprehensive education, because they saw it as competitive (and it does indeed, introduce ‘contest mobility’ at the expense of the ‘sponsored mobility’ of private and selective education. And, a fair number of Labour politicians opposed the ending of selection, because they saw the senior secondaries and grammar schools as ways of getting more working class children into more influential positions in society, and, to some extent it did this, although it often led to these children becoming alienated from their communities. Now, the Tories are much more hostile to comprehensive education and Labour, since Blair and Brown have tended towards a sort of selection. However, it must be stated, unequivocally, that Blair and Brown invested in public education to a far greater extent than any other government and had admirable policies for early years and for inclusive education – these things are to their eternal credit.

    The big failure that Blair and Brown made was to adopt the Chris Woodhead/Ofsted/League Tables/academies approach set up under various of Mrs Thatcher’s Education Secretaries from Sir Keith Joseph onward. Of the Labour education secretaries only Estelle Morris had any real commitment to wider access and was the only one with real experience of public sector schools. Sadly, she was not fully supported and, eventually, had to resign having been worn down by the teacher unions – mainly NUT and NASUWT – and by the backseat driving from Downing St.

    Essentially, what the recent hoo-haw has been about is what Keith Joseph intended the ‘league tables’ to be – instruments of blame, control and dirigisme and, it is the dirigisme and control to which self proclaimed ‘radicals’ like Mr McEnaney are attracted. There have always been sections of the ‘left’ which have not been so different in mindset from the establishment. They think they can use the apparatus of the UK State to impose THEIR vision. The UK State was set up to preserve the privileges of the Establishment and, while it will adapt to suit contemporary circumstances, will always work to sustain the UK Establishment.

    Liked by 1 person

Leave a reply to Alasdair Macdonald Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.