Faculty Bibliography

BACKGROUND: Standardized Patients (SPs) are actors trained to portray health care patients during the training and assessment of health care providers. This paper describes the methods and costs associated with using SPs to evaluate the skills of telephone counselors working on a clinical trial that evaluated a telephone smoking cessation program tailored for smokers using Department of Veterans Affairs mental health clinics. FINDINGS: Conducting the SP exercises required five main steps: (1) Write a SP case description detailing patient demographics, demeanor, clinical symptoms and history, and instructions on how to respond to counseling, (2) Identify, select and train actors to portray the SP cases; (3) Conduct audio-taped counseling encounters between the SPs and counselors, (4) Rate the counselors on their core counseling competencies, (5) Provide feedback to counselors. The SPs and study supervisors reported that the checklist was easy to use when rating the counselors. Counselors reported that the SP encounters were realistic and helpful for practicing their clinical work and for building self-efficacy for working with real patients. The labor costs of developing two SP cases and training two SP actors was approximately $1,475. The per-session labor cost of conducting a 1-hour counseling session between one SP and one counselor was approximately $314. CONCLUSIONS: Using SPs to train telephone counselors working on a clinical trial was feasible and offered training benefits beyond those provided by didactic instruction and role plays. Our research group is now routinely using SPs for the training of incoming telephone counselors.

BACKGROUND: While unannounced standardized patients (USPs) have been used to assess physicians' clinical skills in the ambulatory setting, they can also provide valuable information on patients' experience of the health care setting beyond the physician encounter. This paper explores the use of USPs as a methodology for evaluating patient-centered care in the health care system. METHODS: USPs were trained to complete a behaviorally-anchored assessment of core dimensions of patient-centered care delivered within the clinical microsystem, including: 1) Medical assistants' safe practices, quality of care, and responsiveness to patients; 2) ease of clinic navigation; and 3) the patient-centeredness of care provided by the physician. Descriptive data is provided on these three levels of patient-centeredness within the targeted clinical microsystem. Chi-square analyses were used to signal whether variations by teams within the clinical microsystem were likely to be due to chance or might reflect true differences in patient-centeredness of specific teams. RESULTS: Sixty USP visits to 11 Primary Care teams were performed over an eight-month period (mean 5 visits/team; range 2-8). No medical assistants reported detecting an USP during the study period. USPs found the clinic easy to navigate and that teams were functioning well in 60% of visits. In 30% to 47% of visits, the physicians could have been more patient-centered. Medical assistants' patient safety measures were poor: patient identity was confirmed in only 5% of visits and no USPs observed medical assistants wash their hands. Quality of care was relatively high for vital signs (e.g. blood pressure, weight and height), but low for depression screening, occurring in only 15% of visits. In most visits, medical assistants greeted the patient in a timely fashion but took time to fully explain matters in less than half of the visits and rarely introduced themselves. Physicians tried to help patients navigate the system in 62% of visits. CONCLUSIONS: USP assessment captured actionable, critical, behaviorally-specific information on team and system performance in an urban community clinic. This methodology provides unique insight into the patient-centeredness and quality of care in medical settings.

BACKGROUND: Assessment of obstetrics-gynecology residents' ability to integrate clinical judgment, interpersonal skills, and technical ability in a uniform fashion is required to document achievement of benchmarks of competency. An observed structured clinical examination that incorporates simulation and bench models uses direct observation of performance to generate formative feedback and standardized evaluation. METHODS: The Test of Integrated Professional Skills (TIPS) is a 5-station performance-based assessment that uses standardized patients and complex scenarios involving ultrasonography, procedural skills, and evidence-based medicine. Standardized patients and faculty rated residents by using behaviorally anchored checklists. Mean scores reflecting performance in TIPS were compared across competency domains and by developmental level (using analysis of variance) and then compared to standard faculty clinical evaluations (using Spearman rho). Participating faculty and residents were also asked to evaluate the usefulness of the TIPS. RESULTS: Twenty-four residents participated in the TIPS. Checklist items used to assess competency were sufficiently reliable, with Cronbach alpha estimates from 0.69 to 0.82. Performance improved with level of training, with wide variation in performance. Standard faculty evaluations did not correlate with TIPS performance. Several residents who were rated as average or above average by faculty performed poorly on the TIPS (> 1 SD below the mean). Both faculty and residents found the TIPS format useful, providing meaningful evaluation and opportunity for feedback. CONCLUSIONS: A simulation-based observed structured clinical examination facilitates observation of a range of skills, including competencies that are difficult to observe and measure in a standardized way. Debriefing with faculty provides an important interface for identification of performance gaps and individualization of learning plans.

BACKGROUND: When surgeons decide to become surgeons has important implications. If the decision is made prior to or early in medical school, surgical education can be more focused on surgical diseases and resident skills. METHODS: To determine when surgeons - compared with their nonsurgical colleagues - decide on their medical path, residents in surgery, internal medicine, obstetrics and gynecology, pediatrics, psychiatry, and emergency medicine were surveyed. Timing of residency choice, demographic data, personal goals, and reason for residency choice were queried. RESULTS: A total of 234 residents responded (53 surgical residents). Sixty-two percent of surgeons reported that they were "fairly certain" of surgery before medical school, 13% decided during their preclinical years, and 25% decided during their clerkship years. This compares with an aggregate 40%, 7%, and 54%, respectively, for the other 5 residency specialties. These differences were statistically significant (P = .001). When the 234 residents were asked about their primary motivation for choosing their field, 51% pointed to expected job satisfaction and 44% to intellectual curiosity, and only 3% mentioned lifestyle, prestige, or income. CONCLUSIONS: General surgery residents decide on surgery earlier than residents in other programs. This may be advantageous, resulting in fast-tracking of these medical students in acquiring surgical knowledge, undertaking surgical research, and early identification for surgical residency programs. Surgical training in the era of the 80-hour work week could be enhanced if medical students bring much deeper knowledge of surgery to their first day of residency.

PURPOSE: Describe the relationship between medical students' self-assessment and goal-setting (SAGS) skills and development of interviewing skills during the first-year doctoring course. METHOD: 157 first-year medical students completed three two-case standardized patient (SP) interviews. After each of the first two, students viewed videotapes of their interview, completed a SAGS worksheet, and reviewed a selected tape segment in a seminar. SAGS was categorized into good and poor quality and interviewing skills were rated by trained raters. RESULTS: SAGS improved over time (37% good week 1 vs. 61% good week 10). Baseline SAGS and interviewing skills were not associated. Initial SAGS quality was associated with change in interviewing skills - those with poor-quality SAGS demonstrated a decrease and those with good-quality SAGS demonstrated an increase in scores by 17 weeks (ANOVA F=4.16, p=0.024). For students whose SAGS skills were good at both week 1 and 10, interviewing skills declined in weeks 1-10 and then increased significantly at week 17. For those whose SAGS remained 'poor' in weeks 1-10, interviewing skills declined in weeks 10-17. CONCLUSIONS: In general, the quality of students' SAGS improved over time. Poor baseline SAGS skills and failure to improve were associated with a decrease in interviewing skills at 17 weeks. For students with better SAGS, interviewing skills increased at week 17. Improvement in SAGS skills was not associated with improved interviewing skills. Understanding structured self-assessment skills helps identify student characteristics that influence progressive mastery of communication skills and therefore may inform curriculum and remediation tailoring.