Understanding Clerkship Grading 2019-20

Understanding clerkship
grading 2019-20
Valerie P. Jameson, MD.
Assistant Dean of Clinical Curriculum
Office of Medical Education
Why grade at all?
 Grading is the summative (ultimate) feedback to student performance.

 Am I performing at a high, low or acceptable level?
 Do I have the skills necessary to be a competent physician?
 Do I have the skills necessary to succeed in a highly competitive field?
 Grading allows outside entities (residencies) to know how a particular
student performance compares with his/her peers in that institution.
 Using national norms helps assure that the grading is meaningful
outside of the particular institution.
Grading in the pre-clinical years
 Grading in the basic science courses is mostly based on objective

data derived from Multiple Choice Question Testing.
 The grade is roughly based on the average of the scores from
several MCQT’s.
 You can argue about a few questions, but the score probably
reflects what a student would do on a similar test on any given day.
 Caveat - Measuring true performance is complicated by the
additional uneven skill of MCQT Test Taking ability.
Grading in the clinical years
 More complex
 Now students must also be graded on
 Clinical skills – patient interaction, communication, eye-hand coordination, reasoning
skills, team integration, self-perception, etc.
 Professionalism, ethical behavior
 While the opinion of an experienced faculty member in judging student
performance is considered “objective”, the reality is that subjectivity and
perception have an increased influence on feedback.
 While this may seem to allow for unfair variability, this is exactly how patients
and peers judge physicians in the real world. (Learning how people perceive
you is incredibly valuable knowledge and that’s what you get in your clinical
grade).
Grading in the clinical years
 Equally important in describing student performance is

knowledge and reasoning ability, otherwise known as
expertise.
 Patients come to physicians for caring and compassion,
but they expect ethical treatment and expertise as well.
 Expertise is more easily quantified through MCQT.
 This also allows for national comparison.
It’s vital that all three aspects contribute to the
summative clinical grade
 Students often excel in one aspect or the other.
 However, the good physician must be competent in all three areas.
 Grading therefore must fairly evaluate all three domains.
 This allows students to accurately identify their strengths and
weaknesses.
 It informs residencies which students would best fit their discipline.
 It satisfies the public that medical schools are producing competent
physicians.
How to achieve this goal
 Clinical evaluations must provide detailed evaluation of students’

performance and behavior in the clinical setting.
 Rather than have faculty assign a grade, faculty are simply asked to
choose descriptors of the student’s performance. These are assigned
numeric values which added together provide the grade.
 This way, students achieving the same descriptors across the
hospital, city or state will receive the same grade.
 Sites with significantly smaller student numbers may have faculty
submit feedback to the clerkship director who then compiles the final
evaluation using the same rubric.
Measuring expertise
 All UT clerkships use the National Board of Medical Examiners Subject exams (known as
shelf exams).
 Student performance is reported with a raw score(mean equated percent correct score)
which resembles a percentage correct but is actually a complexly derived number based on
the performance of students answering these questions on previous exams.
 Each raw score is reported with the national percentile performance of students in a
previous year.
 Each discipline has a different set of raw scores and percentiles for their specific exams
dictated by the performance of students in their respective disciplines.
Choosing passing and honors scores
 The NBME gathers information from clerkship directors across the

nation to determine acceptable minimum scores for passing and
honors designation on shelf exams.
 These numbers are reported in ranges.
 In an attempt to offset the asymmetric power of the MCQT when
grading, UT clerkship directors have typically chosen to use a lower
level of these ranges for determining passing and honors.
NBME- nationally recommended minimum
passing score for the Family Medicine Shelf Exam
Minimum passing score for the shelf exam and
the clerkship.
 Each clerkship chooses their own minimum passing and honors scores as well as weights for
each component of their grades which they can explain in detail during their clerkship
orientation.
 The minimum scores are consistent across the state for each clerkship and for the entire
academic year. These percentages and scores are reassessed each year before block 5/6 and
kept the same throughout the academic year.
 For example, the Family Medicine clerkship has chosen a raw score of 61, which is the 5th
percentile for the national student performance across the 2018-19 academic year.
 Although the NBME recalculates these parameters every few years, the clerkships should
maintain the same percentile for passing.
 So, if a 59 raw score was passing last year and a 61 is passing this year, the percentile is the
same in order to remain consistent year to year.
Minimum honors score for the shelf exam
Minimum honors score for the shelf exam
 The Family Medicine clerkship has chosen a raw score of 79, which
is the 63rd percentile for the national student performance across the
2018-19 academic year.
 79 is the lowest raw score acceptable to achieve honors in the
national recommended range of scores.
 In this way, Family Medicine is able to assure that a student
receiving an A for the clerkship has achieved a better than average
level of expertise in MCQT using nationally derived and accepted
standards.
Clerkship minimums (academic year 2018-19)
 Clerkship Min Pass Min Honors(A) for the Course

 Medicine 59(5%ile) 79 (66%ile)
 Pediatrics 62(5%ile) 80 (63%ile)
 Neurology 62(2%ile) 82 (62%ile)
 Family Medicine 61(5%ile) 79 (63%ile)
 Psychiatry 69(6%ile) 84 (63%ile)
 Surgery 60(7%ile) 81 (83%ile)
 OB-Gyn 64(5%ile) 80 (58%ile)
Weighting clinical and exam scores
 The competent student excels in all three areas – patient care, ethics
and expertise.
 However, if all student clinical evaluations are glowing, then the
shelf exam, by default, becomes the discriminating factor.
 This is unacceptable.
 We must strive to make the clinical evaluation discriminating and to
weight it sufficiently to counterbalance the ruthless objectivity of
the shelf score.
Correct weighting
 Potential outcomes
 Super nice, hard working student but average performance on the shelf – Clerkship Grade B
 Never available and only putting in the minimal work on the ward in order to be off studying for
the shelf which they blow out of the water – Clerkship Grade B
 Excellent performance on the wards, intimately integrated into the team, barely makes above the
cutoff for honors on the shelf – Clerkship Grade A
 The student must excel in both areas to be able to qualify for Honors in the Clerkship.
 So just because a student achieved the minimum honors score on the shelf, it does not guarantee
that he/she will get an A for the clerkship.
Why is this approach necessary?
Why grade at all?
 Because of a decided lack of grade discrimination across the country, residencies are unable
to use GPA or Dean’s letters (MSPE’s) to determine which students would be suitable for
their field.
 As a consequence, competitive residencies have resorted to using the USMLE Step 1 score
as a means of winnowing down the massive numbers of student applying for their positions.
 The Step 1 was never designed to be used in this manner and is not statistically valid in
identifying the student most likely to be an exceptional physician.
 It is unconscionable to allow one day’s performance on a MCQT to determine the career
options for the awesome students we have in our college of medicine!
 Most importantly, the only way for a student to improve his/her skills is by receiving valid,
useful, timely, specific, discriminating feedback.
Beta testing
 The Internal Medicine and Pediatrics Clerkships are beta testing a variation of this process
for the academic year 2018-19.
 They have chosen a slightly higher raw score within the minimum honors score range to
determine honors for the shelf exams.
 However, they are using last year’s minimum honors score to identify the students who are
eligible to achieve an A for the rotation.
 We will study this method to determine if it is helpful in emphasizing the effect of the
clinical grade when computing the composite grade, basically making it possible for a
student to get an A for the rotation even if he/she does not get an honors grade on the shelf.
What does a grade mean?
 Until recently, the Dean’s Letter, MSPE, simply provided a GPA along with the student
transcripts.
 Lots of A’s looked pretty good.
 Now, the MSPE must include the grade distribution for each clerkship.
 Residency Program directors will know that getting an A when 80% of the class did as well
is meaningless.
 Getting an A when only a third of the class did as well is discriminating.
 Our job is to assure that that assignment is fair and valid.
Lastly, apples and oranges
 The NBME reports the raw scores (mean equated percent correct score).
 To use them in the calculation with clinical scores, oral and slide exams, it is essential that
the raw score is converted to the same format we use for grading.
 Clerkship Grading Scale
 A 89.5-100
B 78.5-89.49
C 67.5-78.49
F < 67.49
So, in Family medicine
 If 61 is used as the lowest passing score, then it is converted to 67.5.

 If 79 is used as the minimum honors shelf score, then it is converted to 89.5.
 All the other grades are adjusted accordingly which results in a number higher than the raw
score and is used in the calculation of the grade. (using all apples instead of a mixture of
fruit.)
 This makes that calculation numerically valid, even at the expense of a bit of confusion
when discussing how the grades are derived.
 Remember that specific numbers on the shelf are used to determine the minimum passing
score and minimum honors score. The adjusted shelf score is used to calculate the final
composite grade, but there is no set NBME score for determining between a B or C final
grade.
UTHSC is not alone
 Med schools across the nation are struggling with the same issues.
 OME takes student grading very seriously and is always on the
lookout for ways to improve.
 Significant changes can only be made at the start of a new academic
year, although smaller improvements or correction of errors may
occur at any time to assure fairness for the students. Mid-year
adjustments will only be made in the students’ favor.
This powerpoint is in response to feedback
 Thanks to all students who take the time to give us constructive

feedback.
 Just like students, we welcome constructive advice that informs us
of issues and provides suggestions for improvement.
 UT COM students are truly our greatest resource.
 (I admit, I could be biased.)

Understanding Clerkship Grading 2019-20

Uploaded by

Copyright:

Available Formats

Understanding Clerkship Grading 2019-20

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Understanding Clerkship Grading 2019-20

Uploaded by

Copyright:

Available Formats

Understanding clerkship

 Grading is the summative (ultimate) feedback to student performance.

 Grading in the basic science courses is mostly based on objective

 Equally important in describing student performance is

 Clinical evaluations must provide detailed evaluation of students’

 The NBME gathers information from clerkship directors across the

 Clerkship Min Pass Min Honors(A) for the Course

 If 61 is used as the lowest passing score, then it is converted to 67.5.

 Thanks to all students who take the time to give us constructive

You might also like