SQE Marking and Standard Setting Policy

Last updated: December 2024

Kaplan SQE Limited (Kaplan SQE) has been appointed by the Solicitors Regulation Authority Limited (“SRA”) as the sole provider of the Solicitors Qualifying Examination (“the Assessment”) and is the End Point Assessment Organisation (“EPAO”) for Solicitor Apprentices.

This document sets out the policy on marking SQE assessments and determining the pass mark.

1

SQE1

1.1

SQE1 consists of two assessments - FLK1 and FLK2. Candidates must pass both FLK1 and FLK2 to pass SQE1. To pass FLK1 the candidate’s score must be greater than or equal to the pass mark for FLK1. To pass FLK2 the candidate’s score must be greater than or equal to the pass mark for FLK2.

1.2

Each correctly answered question will receive one mark. No marks will be deducted for incorrectly answered questions. Multiple answers for one question will not be counted. The number of marks is added together to provide the candidate score for the relevant assessment.

1.3

The unrounded cut score¹, expressed as an unrounded number of correctly answered questions, will be determined by the Assessment Board (the Board). The Board will have reference to the standard determined by a Modified Angoff method (see 1.4) and other recognised standard setting methods², as appropriate, as well as statistical equating (see 1.6).

1.4

The Modified Angoff method involves a panel of qualified solicitors who are familiar with day one competence, considering, for each question on the assessment, how many out of ten just competent Day One solicitors would answer the question correctly. In reaching this judgment the panel will consider the level of competency of the Day One solicitor as defined in the Threshold Standard, as set out in the SQE1 Assessment Specification. After providing their initial judgments (scores) on each question, panel members have an opportunity for discussion and, if deemed appropriate by any panel member, to adjust their individual score. Prior to the panel’s consideration of the questions there will be training for panel members and a discussion of the application of the Threshold Standard.

1.5

The standard setting process (using the Modified Angoff method and any other recognised standard setting methods², as appropriate) is repeated periodically. In between times, the standard of assessments is maintained using statistical equating.

1.6

Statistical equating allows for a comparison of the difficulty of different assessments by considering how candidates perform on common questions. Calibrating the relationship between these common questions and other questions in each assessment and over time, ensures that standards are consistent across assessments.

1.7

Having determined the unrounded cut score, the precision of the assessment, as measured by the Standard Error of Measurement (SEm), will be considered. A decision will be made by the Board about the level of any correction which should be made for measurement error to arrive at a pass mark for the assessment. This correction will be made in order to ensure that those who pass deserve to pass, on the basis that it provides sufficient assurance that those achieving the pass mark are competent to practise.

1.8

Once any correction has been made to the unrounded cut score, the corrected cut score becomes the unrounded pass mark, which will be rounded to the nearest integer³ to provide the pass mark. The pass mark will be the number of correctly answered questions needed to pass the assessment.

1.9

When candidate scores and the pass mark have been confirmed, these are placed onto a scale between 0 and 500. The pass mark is always placed at 300 on the scale, with candidate scores above and below the pass mark being scaled relative to that pass mark. The candidate’s scaled score is then rounded to the nearest integer³. This process does not change any candidate’s overall outcome for the assessment. If the candidate’s number of correctly answered questions is greater than or equal to the pass mark, they will pass.

2

SQE2

2.1

SQE2 consists of one assessment with 16 stations⁴.

2.2

An examiner who has been trained in playing the role of the client will assess candidates' performance in the client interview. The client interview will be assessed purely on skills, not on the application of the law. The attendance note and all other exercises will be marked by solicitor examiners on both skills and application of the law. All examiners will have been trained and had their marking standardised. Consistency is ensured throughout the marking process via moderation of the examiners.

2.3

Marking is based on professional judgements rather than a ‘tick box’ or checklist approach. The starting point for these professional judgements is the standard of competency of the assessment set out in the SQE2 Assessment Specification, namely that of the just competent Day One solicitor (Threshold Standard). Examiners are trained to be flexible as to the approach taken by the candidate whilst still adhering to the expected standard of competency.

2.4

Each type of station⁴ has assessment criteria which are set out in the SQE2 Assessment Specification.

2.5

Marking is based on performance on each of the assessment criteria judged on a scale from A – F as follows:

• Superior performance: well above the competency requirements of the assessment
• Clearly satisfactory: clearly meets the competency requirements of the assessment
• Marginal pass: on balance, just meets the competency requirements of the assessment
• Marginal fail: on balance, just fails to meet the competency requirements of the assessment
• Clearly unsatisfactory: clearly does not meet the competency requirements of the assessment
• Poor performance: well below the competency requirements of the assessment

Marks for each of the assessment criteria are placed on a numerical scale such that A = 5 marks and F = 0 marks.

2.6

Each candidate has a percentage score calculated for each station⁴ from the marks awarded for application of law, and for skills, which are combined with equal weighting to provide a total score for each station. The total candidate percentage score is the average of the percentage scores across the 16 stations.

2.7

To calculate the score for the Client Interview and Attendance Note/Legal Analysis, the skills mark is calculated from the skills mark for the interview (which is marked only on skills) and the skills mark for the attendance note, weighted equally. The application of law mark comes solely from the attendance note/legal analysis. To calculate the total station score, skills and application of law are then weighted equally.

2.8

In calculating a candidate’s final score, the candidate's total percentage score will be rounded to the nearest integer³.

2.9

The unrounded percentage cut score¹ for SQE2 will be set by the Board having reference to the borderline regression method supplemented by other recognised methods as appropriate².

2.10

In the borderline regression method, each station⁴ is marked as outlined above. Each examiner also provides an overall “standard-setting” grade of pass, marginal pass, marginal fail or fail. This standard-setting grade does not count as part of the candidate’s mark but is used to set the cut score for the station. Candidate scores are regressed against standard setting grades to arrive at a cut score for each station. The station cut scores are then averaged to arrive at the cut score¹ for the assessment.

2.11

Having arrived at a cut score, the precision of the assessment as measured by the Standard Error of Measurement (SEm) will be considered. A decision will be made by the Board about the level of any correction which should be made for measurement error to arrive at a pass mark for the assessment. This correction will be made to ensure that those who pass deserve to pass, on the basis that it provides sufficient assurance that those achieving the pass mark are competent to practise.

2.12

Following this, the percentage pass mark will be rounded to the nearest integer³.

2.13

When candidate scores and the pass mark have been confirmed, these are placed onto a scale between 0 and 500. The pass mark is always placed at 300 on the scale, with candidate scores above and below the pass mark being scaled relative to that pass mark. The candidate’s scaled score is then rounded to the nearest integer³. This process does not change any candidate’s overall outcome for the assessment. If the candidate’s percentage score is greater than or equal to the pass mark, they will pass.

¹The cut score is the lowest possible score to pass the assessment. The cut score is normally calculated without reference to measurement error, which is considered after a cut score is arrived at (see para 1.3) to derive the pass mark
²See for instance Hofstee, W.K.B. (1983). The case for compromise in educational selection and grading. In S. B. Anderson & J. S. Helmick (Eds.), On educational testing (pp. 109–127). San Francisco, CA: Jossey-Bas
³At XX.499 and below, the mark goes down to the nearest integer; at XX .500 and above it goes up to the nearest integer
⁴The 16 different exercises that candidates undertake in SQE2 are referred to as stations. There are six types of station - client interview and attendance note/legal analysis, advocacy, case and matter analysis, legal research, legal writing, legal drafting.