Investigation of standardized patient ratings of humanistic competence on a medical licensure examination using Many-Facet Rasch Measurement and generalizability theory

Xiuyuan Zhang, William L. Roberts

Research output: Contribution to journalArticlepeer-review

5 Scopus citations

Abstract

Humanistic doctor-patient interaction has been measured for eight years using the Global Patient Assessment (GPA) tool in the national osteopathic clinical skills medical licensure examination. Standardized patients (SPs) apply the GPA tool to rate examinees' competence on doctor-patient communication, interpersonal skills, and professionalism. Many-Facet Rasch Measurement was employed to evaluate the overall functioning of the GPA rating scale and to estimate measurement errors associated with characteristics of SP raters and medical case presentations. Generalizability theory was applied to investigate variance components corresponding to each facet of interest. For the 2010-2011 testing cycle, 50,090 SP ratings were analyzed. Although SP raters varied in leniency/stringency of rating, SPs differentiated the six GPA aspects in difficulty, and utilized a reasonable range of the 9-point scale. Reliability indices resulted in sufficient examinee separation, 0.94, from the Rasch model and sufficient dependability from the generalizability analysis for raw scores, 0.83, and transformed Rasch scores, 0.97. Results indicate that medical students' humanistic competence can be reliably measured through the GPA tool in the simulated environment. These measurement models supplement other means of observation and quality control with valuable information about the psychometric quality of SP ratings of humanistic competence. © 2012 Springer Science+Business Media Dordrecht.
Original languageEnglish (US)
Pages (from-to)929-944
Number of pages16
JournalAdvances in Health Sciences Education
Volume18
Issue number5
DOIs
StatePublished - Dec 1 2013
Externally publishedYes

ASJC Scopus subject areas

  • Education
  • General Medicine

Fingerprint

Dive into the research topics of 'Investigation of standardized patient ratings of humanistic competence on a medical licensure examination using Many-Facet Rasch Measurement and generalizability theory'. Together they form a unique fingerprint.

Cite this