Faculty Bibliography

BACKGROUND: To establish international standards for medical schools, an appropriate panel of experts must decide on performance standards. A pilot test of such standards was set in the context of a multidimensional (multiple-choice question examination, objective structured clinical examination, faculty observation) examination at 8 leading schools in China. METHODS: A group of 16 medical education leaders from a broad array of countries met over a 3-day period. These individuals considered competency domains, examination items, and the percentage of students who could fall below a cut-off score if the school was still to be considered as meeting competencies. This 2-step process started with a discussion of the borderline school and the relative difficulty of a borderline school in achieving acceptable standards in a given competency domain. Committee members then estimated the percentage of students falling below the standard that is tolerable at a borderline school and were allowed to revise their ratings after viewing pilot data. RESULTS: Tolerable failure rates ranged from 10% to 26% across competency domains and examination types. As with other standard-setting exercises, standard deviations from initial to final estimates of the tolerable failure rates fell, but the cut-off scores did not change significantly. Final, but not initial cut-off scores were correlated with student failure rates (r = 0.59, P = 0.03). DISCUSSION: This paper describes a method to set school-level outcome standards at an international level based on prior established standard-setting methods. Further refinement of this process and validation using other examinations in other countries will be needed to achieve accurate international standards.

BACKGROUND: Evidence supporting professionalism as a critical measure of competence in medical education is limited. In this case-control study, we investigated the association of disciplinary action against practicing physicians with prior unprofessional behavior in medical school. We also examined the specific types of behavior that are most predictive of disciplinary action against practicing physicians with unprofessional behavior in medical school. METHODS: The study included 235 graduates of three medical schools who were disciplined by one of 40 state medical boards between 1990 and 2003 (case physicians). The 469 control physicians were matched with the case physicians according to medical school and graduation year. Predictor variables from medical school included the presence or absence of narratives describing unprofessional behavior, grades, standardized-test scores, and demographic characteristics. Narratives were assigned an overall rating for unprofessional behavior. Those that met the threshold for unprofessional behavior were further classified among eight types of behavior and assigned a severity rating (moderate to severe). RESULTS: Disciplinary action by a medical board was strongly associated with prior unprofessional behavior in medical school (odds ratio, 3.0; 95 percent confidence interval, 1.9 to 4.8), for a population attributable risk of disciplinary action of 26 percent. The types of unprofessional behavior most strongly linked with disciplinary action were severe irresponsibility (odds ratio, 8.5; 95 percent confidence interval, 1.8 to 40.1) and severely diminished capacity for self-improvement (odds ratio, 3.1; 95 percent confidence interval, 1.2 to 8.2). Disciplinary action by a medical board was also associated with low scores on the Medical College Admission Test and poor grades in the first two years of medical school (1 percent and 7 percent population attributable risk, respectively), but the association with these variables was less strong than that with unprofessional behavior. CONCLUSIONS: In this case-control study, disciplinary action among practicing physicians by medical boards was strongly associated with unprofessional behavior in medical school. Students with the strongest association were those who were described as irresponsible or as having diminished ability to improve their behavior. Professionalism should have a central role in medical academics and throughout one's medical career.

BACKGROUND: Medical students have unique information about peers' professionalism but are reluctant to share it through peer assessment. METHOD: Students (231 of 375; 62%) in one school replied to a survey about whether various characteristics of peer assessment (e.g., who receives the assessment, its anonymity, implications for the classmate) would prevent or encourage their participation. RESULTS: Sixty-six percent of the students agreed that there should be peer assessment of professionalism as long as the assessment reflected their preferences for how the assessment should take place. Some of their preferences included reporting unprofessional behavior to an impartial counselor, a 100% anonymous process, and having the classmate receive corrective instruction. Students across year levels generally agreed about the characteristics of peer assessment. Men and women disagreed about some characteristics. CONCLUSION: Most students are willing to participate in peer assessment as long as their preferences are taken into consideration.

BACKGROUND: Although peer assessment holds promise for assessing professionalism, reluctance and refusal to participate have been noted among learners and practicing physicians. Understanding the perspectives of potential participants may therefore be important in designing and implementing effective peer assessment. OBJECTIVE: To identify factors that, according to students themselves, will encourage or discourage participation in peer assessment. DESIGN: A qualitative study using grounded theory to interpret views shared during 16 focus groups that were conducted by leaders using a semi-structured guide. PARTICIPANTS: Sixty-one students in Years 1, 3, and 4 in 2 mid-western public medical schools. RESULTS: Three themes students say would promote or discourage peer assessment emerged: personal struggles with peer assessment, characteristics of the assessment system itself, and the environment in which the system operates. Students struggle with reporting an unprofessional peer lest they bring harm to the peer, themselves, or their clinic team or work group. Who receives the assessment and gives the peer feedback and whether it is formative or summative and anonymous, signed, or confidential are important system characteristics. Students' views of characteristics promoting peer assessment were not unanimous. Receptivity to peer reports and close positive relationships among students and between students and faculty mark an environment conducive to peer assessment, students say. CONCLUSIONS: The study lays a foundation for creating acceptable peer assessment systems among students by soliciting their views. Merely introducing an assessment tool will not result in students' willingness to assess each other.

OBJECTIVES: Over time, quarantine has become a classic public health intervention and has been used repeatedly when newly emerging infectious diseases have threatened to spread throughout a population. Here, we weigh the economic costs and benefits associated with implementing widespread quarantine in Toronto during the SARS outbreaks of 2003. METHODS: We compared the costs of two outbreak scenarios: in Scenario A, SARS is able to transmit itself throughout a population without any significant public health interventions. In Scenario B, quarantine is implemented early on in an attempt to contain the virus. By evaluating these situations, we can investigate whether or not the use of quarantine is justified by being either cost-saving, life saving, or both. RESULTS: Our results indicate that quarantine is effective in containing newly emerging infectious diseases, and also cost saving when compared to not implementing a widespread containment mechanism. CONCLUSIONS: This paper illustrates that it is not only in our humanitarian interest for public health and healthcare officials to remain aggressive in their response to newly emerging infections, but also in our collective economic interest. Despite somewhat daunting initial costs, quarantine saves both lives and money.

Increasing physician and patient mobility has led to a move toward internationalization of standards for physician competence. The Institute for International Medical Education proposed a set of outcome-based standards for student performance, which were then measured using three assessment tools in eight leading schools in China: a 150-item multiple-choice examination, a 15-station OSCE and a 16-item faculty observation form. The purpose of this study was to empanel a group of experts to determine whether international student-level performance standards could be set. The IIME convened an international panel of experts in student education with specialty and geographic diversity. The group was split into two, with each sub-group establishing standards independently. After a discussion of the borderline student, the sub-groups established minimally acceptable cut-off scores for performance on the multiple-choice examination (Angoff and Hofstee methods), the OSCE station and global rating performance (modified Angoff method and holistic criterion reference), and faculty observation domains (holistic criterion reference). Panelists within each group set very similar standards for performance. In addition, the two independent parallel panels generated nearly identical performance standards. Cut-off scores changed little before and after being shown pilot data but standard deviations diminished. International experts agreed on a minimum set of competences for medical student performance. In addition, they were able to set consistent performance standards with multiple examination types. This provides an initial basis against which to compare physician performance internationally.

BACKGROUND: No universally accepted method to measure professionalism exists. We developed an instrument to measure specific aspects of professionalism in surgical residents. METHODS: Professionalism was deconstructed into 15 domains. Behavioral descriptors were determined for extreme and selected intermediate anchors. It became evident that residents could "go too far" in some professional behaviors. Therefore, although a 7-point continuous ordinal scale forms the framework, a score of 7 does not necessarily indicate the ideal. This characteristic minimizes the problem of inflated ratings. RESULTS: The instrument was utilized by attending faculty to evaluate residents and also by residents as a self-evaluation. Calculated from ordinal values, mean (SD) of the ratings across domains for faculty evaluations of residents was 4.95 (0.38) while mean for self-evaluations was 4.95 (0.39). Reliability was high (coefficient alpha 0.85). CONCLUSIONS: This instrument provides a means to measure professional behaviors during surgical residency. Repeated use will be required to thoroughly establish validity and reliability.