SQE Marking and Standard Setting Policy

Last updated: April 2021

Kaplan SQE Limited (Kaplan SQE) has been appointed by the Solicitors Regulation Authority (“SRA”) as the sole provider of the Solicitors Qualifying Examination (“the Assessment”) and is the End Point Assessment Organisation (“EPAO”) for Solicitor Apprentices.

This document sets out the policy on marking SQE assessments and determining the pass mark.

1

SQE1

1.1

SQE1 consists of two exams - FLK1 and FLK2. Candidates must pass both FLK1 and FLK2 to pass SQE1. To pass FLK1 candidates must obtain the overall pass mark for FLK1. Then to pass FLK2, candidates must again obtain the overall pass mark for FLK2.

1.2

Each correct response will receive one mark but no marks will be deducted for incorrect responses. Multiple responses for one question will not be counted. The marks for FLK1 will be aggregated to derive the overall percentage mark for FLK1. Likewise, the marks for FLK2 will be aggregated to derive the overall percentage mark for FLK2. To determine whether a candidate has obtained the overall pass mark, the candidate's total percentage mark is rounded off to the nearest integer¹.

1.3

The pass mark will be determined by the Assessment Board (the Board) having reference to the Modified Angoff method, supplemented by statistical equating and other recognised methods² as appropriate. A correction for measurement error will be made in order to ensure that those who pass deserve to pass, on the basis that it provides sufficient assurance that those above the pass mark are competent to practise. Following this, the pass mark will be rounded off to the nearest integer³.

1.4

The Modified Angoff method involves a panel of qualified solicitors who are familiar with day one competence considering, for each question on the assessment, how many out of ten just competent Day One solicitors would answer the question correctly. In reaching this judgment the group will consider the level of competency of the Day One solicitor as defined in the Threshold Standard, as set out in the SQE1 Assessment Specification. Prior to the panel’s consideration of each question there will be training for panel members and a discussion of the application of the Threshold Standard. The judgements made by the panel members are averaged for each item and summated for the assessment to give a first indication of the cut score⁴.

1.5

Statistical equating involves a comparison between the difficulty of different assessments by looking at how candidates performed on common items and how they performed on different items. The results of the Angoff Panel are normally supplemented with a consideration by the Board of statistical equating

1.6

Having determined a cut score, the precision of the assessment as measured by the Standard Error of Measurement (SEm), will be considered. A decision will be made by the Board about the level of any correction which should be made for measurement error to arrive at a final pass mark for the assessment. This correction will be made in order to ensure that those who pass deserve to pass, on the basis that it provides sufficient assurance that those above the pass mark are competent to practise.

2

SQE2

2.1

SQE2 consists of one exam with 16 stations⁵.

2.2

An assessor who has been trained in playing the role of the client will assess candidates' performance in the client interview. The client interview will be assessed purely on skills, not on the application of the law. The attendance note and all other exercises will be marked by solicitors and will be marked on both skills and application of the law. All markers will have been trained and had their marking standardised.

2.3

Marking is based on global professional judgements rather than a ‘tick box’ or checklist approach. The starting point for these global professional judgements is the standard of competency of the assessment set out in the SQE2 Assessment Specification, namely that of the just competent Day One solicitor (Threshold Standard). Markers are trained to be flexible as to the approach taken by the candidate.

2.4

Each type of exercise (client interview and attendance note/legal analysis, advocacy, case and matter analysis, legal research, legal writing, legal drafting) has assessment criteria which are set out in the SQE2 Assessment Specification.

2.5

Marking is based on performance on each of the assessment criteria judged on a scale from A – F as follows:

• Superior performance: well above the competency requirements of the assessment
• Clearly satisfactory: clearly meets the competency requirements of the assessment
• Marginal pass: on balance, just meets the competency requirements of the assessment
• Marginal fail: on balance, just fails to meet the competency requirements of the assessment
• Clearly unsatisfactory: clearly does not meet the competency requirements of the assessment
• Poor performance: well below the competency requirements of the assessment

2.6

Marks for each candidate are placed on a numerical scale such that A= 5 marks and F = 0 marks. and these numerical scores for each station are then summated to arrive at the candidate’s total score.

2.7

Each station has a percentage score calculated from the marks awarded for application of law, and for skills, which are combined with equal weighting to provide a total score for each station. The final candidate percentage score is an average of the 16 station totals.

2.8

To calculate the score for the Client Interview and Attendance Note/Legal Analysis, the skills marks is calculated from the skills mark for the interview (which is marked only on skills) and the skills mark for the attendance note, weighted equally. The application of law mark comes solely from the attendance note/legal analysis. To calculate the total station score, skills and application of law are then weighted equally.

2.9

In order to pass SQE2 candidates must obtain the overall pass mark for SQE2. In calculating a candidate’s total score, the candidate's total percentage mark will be rounded off to the nearest integer⁶.

2.10

The pass mark for SQE2 will be set by the Board having reference to the borderline regression method supplemented by other recognised methods as appropriate⁷, as well as a correction for measurement error. Following this the pass mark will be rounded off to the nearest integer⁸.

2.11

In the borderline regression method, each exercise is marked as outlined above. Each examiner also provides an overall “standard-setting” grade of pass, marginal pass, marginal fail or fail. This standard-setting grade does not count as part of the candidate’s mark but is used to set the cut score for the exercise. Candidate scores are regressed against standard setting grades to arrive at a cut score for each exercise, and summated for the assessment.

2.12

Having arrived at a cut score, the precision of the assessment as measured by the Standard Error of Measurement (SEm) will be considered. A decision will be made about the level of any correction to the cut score that should be made to account for measurement error before arriving at a final pass mark for the assessment. This correction will be made to ensure that those who pass deserve to pass, on the basis that it provides sufficient assurance that those above the pass mark are competent to practise.

¹At XX.499 and below, the mark goes down to the nearest integer; at XX .500 and above it goes up to the nearest integer
²See for instance Hofstee, W.K.B. (1983). The case for compromise in educational selection and grading. In S. B. Anderson & J. S. Helmick (Eds.), On educational testing (pp. 109–127). San Francisco, CA: Jossey-Bas
³At XX.499 and below, the mark goes down to the nearest integer; at XX .500 and above it goes up to the nearest integer
⁴The cut score is the lowest possible score to pass the assessment. The cut score is normally calculated without reference to measurement error, which is considered after a cut score is arrived at (see para 1.3) to derive the pass mark
⁵The 16 different exercises that candidates undertake in SQE2 are referred to as stations
⁶At XX.499 and below the mark goes down to the nearest integer; at XX .500 and above it goes up to the nearest integer
⁷See for instance Hofstee, W.K.B. (1983). The case for compromise in educational selection and grading. In S. B. Anderson & J. S. Helmick (Eds.), On educational testing (pp. 109–127). San Francisco, CA: Jossey-Bass
⁸At XX.499 and below the mark goes down to the nearest integer; at XX .500 and above it goes up to the nearest integer