Faculty Bibliography

BACKGROUND: Food insecurity (FI) occurs when individuals or households have limited or uncertain access to adequate food. People with FI may find it particularly difficult to eat a balanced diet and manage weight. Individuals who report FI are 1.4 times more likely to have obesity. U.S. Veterans have a greater prevalence of FI compared with the general population (24% vs. 19%). As part of a pilot randomized controlled trial to test the impact of an obesity intervention for Veterans, we explored the prevalence of FI in our sample and assessed the impact of FI on weight outcomes. METHODS: Veterans with a Body Mass Index of greater than or equal to 30 or between 25 and 29.99 with at least one comorbidity were recruited by phone for a pilot study. Participants randomized to the intervention received a technology-assisted health coaching session and follow-up coaching calls; control participants received a VA "healthy living messages" pamphlet. At baseline and 3 months, participants had weight measurements and completed surveys. To measure FI, we used a 6-item Household Food Security Scale (FI = 2 or more affirmative responses). We assessed the impact of FI on variables independent of arm assignment in participants who returned for a 3-month measurement visit (Wilcoxon rank sum tests and Fisher's Exact Test). We also analyzed the role of FI as a moderator between arm and study variables using ANOVA. RESULTS: Out of 31 Veterans who enrolled in the study (Mean age = 53.48, 63% male, mean BMI = 31.72), 10 demonstrated FI at baseline and 25 participants completed 3 month measurements. Independent of the intervention, those with FI (n = 10) lost .01 kg of weight and those without FI (n = 15) lost 1.69 kg at 3-months but the difference was not significant (p = 0.2). Similarly, 0/10 Veterans with FI achieved 2.5% weight loss, compared to 4/15 of those without FI (p = 0.12). Those with FI in the control group gained weight when compared to those without FI (1.04 kg, SD = 1.41 vs. -1.75, SD = 3.96) and those with FI in the intervention lost the same amount of weight as those without FI (-1.59 kg, SD = 0.83, vs. -1.59, SD = 2.27; p = 0.24 for the interaction (not significant)). CONCLUSIONS: FI in Veterans enrolled in a weight management study is high. Participants with FI in the control arm may have gained weight when compared to non-FI participants, while intervention arm participants with FI lost similar amounts of weight to those without FI. This suggests that the intervention may reduce the negative impact of FI. While this pilot study was not powered to show a significant difference, we have amended health coaching protocols to better address FI in our intervention (e.g. screening for FI and linking participants with Supplemental Nutrition Assistance Program (SNAP) benefits and similar resources) hoping to maximize its effectiveness

BACKGROUND: The shift toward a Patient Centered Medical Home has redefined healthcare delivery to be a patient centered affair. While this is beneficial, it also calls for patients to be more activated in the doctor's office and responsible in their self-directed care outside of the office. For patients with chronic diseases, the burden of illness is even higher and requires significantly more effort in disease selfmanagement. We developed 4 hour Patient Empowerment Program (PEP) to bridge that gap through a program training patients with diabetes in the skills necessary to communicate effectively with providers and engage in shared decision making (SDM). Previously, we reported improved diabetes self-care behaviors at 6 months post intervention based on standardized questionnaires. This study examined participants' perspectives on PEP from 1-2 years post intervention. METHODS: 71 patients with type 2 diabetes mellitus were recruited from 2 urban safety-net hospitals to participate in PEP. 33 patients completed the intervention and 28 patients completed a 6-month follow-up assessment. Participants were predominantly low-income, racial minorities, with limited health literacy (Newest Vital Sign M= 2.21, SD = 1.67). We reached 22 of those 28 participants, at 11 to 20months post intervention (M= 16, SD = 2.31). They were interviewed via telephone, with structured open-ended questions asking them to reflect on what they took away from the classes, and whether they'd behaved differently during doctor visits or cared for their diabetes differently since the classes. A qualitative analysis was made of these responses, using Dedoose software to assist in analysis. RESULTS: All 22 patients recalled PEP and could identify key concepts from it. Participants referenced the doctor-patient relationship, from "how to improve relationship with my doctor" to "practicing role of doctor and patient in interaction and relationship". 86% of participants identified changes in their behavior since PEP- 36% identifying that they share more information with their doctor now than they did before, 18% reporting that they ask more questions, and another 18% reporting that they are more proactive in their diabetes care. When asked about their diabetes self-management, prevalent themes were better diet/food choices, exercise, and adherence to medication. 3 patients reported losing weight since the classes. CONCLUSIONS: Despite follow up occurring almost a year later for some, and almost 2 years for others, the message of PEP was clear and compelling. Participant's perspective on their role as a patient changed from a passive recipient of healthcare to a more engaged and activated one. They felt empowered to participate in SDM with their doctors and more comfortable speaking up for their preferences. While further validation is necessary, PEP offers an important way to prepare patients to become true partners with their providers

BACKGROUND: It is critically important for medical schools to understand how well prepared their graduates are for residency and yet we do not have a full understanding of how well competencies, assessed in medical school, transfer to residency. This study explores how communication skills measured in a high-stakes, rigorous, comprehensive OSCE in the 3rd year of medical school are related to performance in a similar OSCE in residency and to Residency Program Directors' ratings of intern competence. METHODS: We analyzed communication skills from three time points in a longitudinal cohort of NYU graduates who entered our Internal Medicine Residency (n = 42). 39 provided consent for their GME-UME data to be compiled into a longitudinal, de-identified educational research database through an IRB-approved Registry. Communication skills were measured using a behaviorally anchored 15-item checklist across the 8-station, pass/fail, MS3 OSCE and then midway through PGY2 of residency in a 6- station OSCE (score =% of items rated well done). SPs also provided an overall rating of communication skills (not recommend, with reservations, recommend, highly recommend). In between, at the end of intern year, residents were also rated by their Program Directors on communication skills (and other competencies) using a 4-pt scale. RESULTS: OSCE communication performance assessed in medical school was modestly associated with performance in residency (r = .26, p = .07) but not with Program Directors' ratings of residents' communication skills as interns (r = .11, p = .28). Number of cases in which medical students were "not recommended" for their communication skills was negatively associated with residency OSCE communication scores (r = -.33, p = .05) and positively associated with number of "not recommends" (r = .46, p = .01) but not with Directors' ratings of interns (r = -.08, p = .49). Number of not recommends independently explained more variance in subsequent residency communication scores than did medical school performance (9% vs. 5%). While average OSCE communication scores improved from medical school to residency (65 to 71%), those with 2 or more "not recommends" improved significantly more than those with 1 or no not recommends. Overall, most learners' (21/39) communication scores improved substantially; less than a quarter (7) decreased; and about a quarter (11) were stable. CONCLUSIONS: While communication scores from medical school are associated with similarly measured scores in residency, SPs' decisions to "not recommend" students appear to serve as an independent indicator of future skill deficits. Patterns of change, however, are not necessarily straightforward: students with the most "not recommends" improved the most. The ability to track competency assessments longitudinally is essential for understanding the transition from medical school to residency and future research will benefit from larger sample sizes and the inclusion of learner characteristics that may explain developmental patterns

BACKGROUND: At Bellevue Hospital Center (BHC), we have a robust Unannounced Standardized Patient (USP) program, where trained actors portraying real patients in the clinical setting, incognito, assess the residents' skills following their visit. We sought to determine the relationship between USP ratings of residents' skills and clinical outcomes among the residents' continuity patient panels to define educationally sensitive patient outcomes. METHODS: We assembled a retrospective cohort of PGY 2 internal medicine residents with at least 2 USP visits between 7/1/14-6/30/15 and ambulatory care patient panels at BHC. The two outcome variables were the percentage of hypertensive patients in the residents' panel with blood pressure (BP) <140/90, and the average of the most recent glycosylated hemoglobin (HbA1C) result among the residents' patients with diabetes. The predictor variables included mean USP ratings of residents' clinical skills and mean faculty rating of the residents' clinic notes (scored for quality on a 0 to 3 scale). USPs used a behaviorally anchored checklist (not done, partly done, well done) for the following domains: communication, case specific assessment, patient education, physical examination, professionalism, management plan, patient satisfaction, and patient activation measure. We tested the correlations between USP scores with BP and HbA1C control, and then developed multivariate, linear regression models of USP scores on BP and HbA1C scores, respectively, each controlling for Avg. Chronic health score (ACHS, derived by scoring different clinical conditions by acuity and used to determine if the panel is getting sicker over time) and total number of patients in the panel (TNPP) because these variables were correlated with both the outcome and predictor variables. RESULTS: 29 PGY 2 residents had a mean of 2.5 (SD 1.0) USP visits during the study period. Residents' patient panels size varied (median 124, range 62- 171) and mean patient age was 48 years (SD 1.4). Patient Activation scores were correlated with Average Chronic Health Score (r = .482, p = .008) and Panel Average last A1c (r = -.311, p = .10). Patient activation scores explained 16% variance in the mean panel last HgA1c, (adjusted R2 .137, p = .08). Case specific Assessment & Patient Education skills across USP cases explained 21.5% of the variance and the Average Chart Note Score explained 14.4% of the variance in % of Hypertension controlled (adjusted R2.378, p < .009). CONCLUSIONS: This exploratory study suggests that learnable resident clinical skills are associated with quality of care indicators for HTN and DM control. In particular, being able to activate patients, assess and educate them and write high quality notes are pathways to quality care. Next steps are to confirm these findings in a larger dataset. Doing so will help align medical education with patient safety and care quality and provide guidance for educational and clinical research aimed at improving the health of populations served

BACKGROUND: Effective smoking cessation counseling improves smokers' health and quality of life. As part of our assessment program, an Unannounced Standardized Patient (USP) case was developed to measure residents' performance in a routine visit with a smoker. METHODS: The USP was a 40 year-old male new patient presenting with heartburn. He began smoking up to two packs/day at 22 years old; at the time of the visit, he reports having cut down to one pack/day and quitting cold turkey twice in the past only to return to smoking. If the resident engages him, he discusses his relationship with smoking and the possibility of quitting. TheUSP received 6 hours of character and checklist training to ensure standardized portrayal and evaluation. Data was collected using 2 forms of assessment: a post-visit USP checklist and a systematic review of the EMR (lab orders, prescriptions, and referrals). The 170- item USP checklist measured communication, patient education, assessment skills, and case-specific items. Each response option included descriptive behavioral anchors and was rated as not done, partly done, or well done. RESULTS: Data was examined from 73 USP visits from 2009-2015. Mean visit length = 37 min, SD = 15 min (range: 15 to 95 min). Overall communication scores ranged from 17 to 100% with an average of 62% (Cronbach's alpha = 0.75). All residents documented History of Tobacco Use or Tobacco Use Disorder in the EMR, and the majority (82%) prescribed smoking cessation medication. There was variation in the sophistication of smoking cessation-counseling approach. Whilemost residents (78%) discussed the risks of smoking and/or the benefits to quitting, significantly fewer (48%) explored the patient's view of the pros and cons of his smoking (p = 0.00). Residents who prescribed smoking cessation medication and discussed risks/benefits to smoking/quitting (N = 31) were compared to residents who did the same but also invited the patient to discuss his personal pros and cons of smoking (N = 29). Groups were not significantly different by PGY or gender. Patients who were asked to discuss their pros/cons rated the resident higher on patient activation questions (0-2 point scale), including "Helped you understand the importance of quitting smoking" (1.38 vs 0.90, p = 0.00), "Made you want to change your smoking" (1.10 vs 0.52, p = 0.00), and "Made you feel like you would be able to quit smoking" (1.07 vs 0.35, p = 0.00). There were no significant differences in labs ordered, referrals to a smoking cessation program, or quality of documentation. CONCLUSIONS: While all residents ask about tobacco use and most appropriately prescribe medication, fewer than half demonstrate the skills known to motivate patients to quit smoking. Curricula needs to reinforce the importance of a patient discussing their personal relationship with smoking in order to feel activated and willing to engage in cessation

BACKGROUND: Safe, high quality outpatient care often depends on the degree to which patients understand their situation and how to follow through on physician recommendations. However, we do not know enough about how often physicians focus on ensuring that their patients have achieved these understandings by the end of the visit and whether such end-of-visit practices are associated with physicians' communication, patient education and activating skills. METHODS: Two Unannounced Standardized Patient cases (highly trained actors who present as real patients) were delivered to 71 internal medicine residents in two clinics: one required the physician to identify a patient's depression and engage him in follow-up care, and the other required the physician to recognize a patient's failure to use her asthma medicine correctly and educate her in using it properly. End-of-visit practices were: reviewing the plan; asking if further questions; giving information about follow-up care and further contact; and helping the patient navigate the system in order to follow through on next steps. Each was assessed by the SP as not done, partly done, or well done. SPs also rated physicians' communication skills, patient activating skills, and case-specific education skills. Summary scores were calculated as% of items well done. RESULTS: Close to three-quarters of the physicians reviewed the plan with the patient and invited further questions in the depression case and slightly more than half did so in the asthma case (56 and 60%). Patients were given complete information about follow-up care and how to navigate the system in just under half of depression visits (49 and 47%) and just over half of asthma visits (58 and 58%). On average, residents were rated as performing 61% of these 8 items well (SD 28%) across both cases. Primary care residents performed significantly better than categorical internal medicine residents (67%vs 47%, p = .004). There were no differences by physician gender. End of visit scores were significantly positively correlated with both general and casespecific clinical skills, and after controlling for the variance contributed by the program (R2 = 12%, p = .004), case-specific education scores explained 10% of the variance in end of visit score (p = .005), patient activating skills 10% of the variance (p = .002) and communication skills 13% of the variance (p = .001). With all variables in the model, only the general communication domain of patient education and counseling was independently associated with end of visit scores (Std Beta = .35, p = .015). CONCLUSIONS: Had these patients been real patients, in one-quarter to onehalf of the visits, the patient would have left not fully understanding the plan or how to follow-through on care. Resident physicians with more effective communication and patient activating skills tended to provide safer end-of-visit care, suggesting that these may reflect an outpatient safety orientation or skillset

BACKGROUND: At NYU, as part of a comprehensive objective structured clinical skills exam, experienced medical educators judge clinical knowledge, decision-making, and clinical reasoning skills of trainees based on their patient notes. Despite being rubric-driven, this task requires tremendous time and effort to establish consistent scoring, delaying and limiting individualized feedback. We conducted pilot machine learning text classification studies to establish if accurate automated scoring of clinical notes is possible. METHODS: As a use case, we tested 100 student written clinical notes from7 standardized patient cases (Vision Loss, Tel Diarrhea, Difficulty Sleeping, Shoulder Pain, Failure To Thrive, Abdominal, Pain, Palpitations) that had been scored for quality of clinical reasoning by faculty on a 1-4 scale. In order to assess performance of NLP strategies to categorize students in meaningful groups we dichotomized students based on their faculty given scores by case into "failing" (score of 1, 5-18 students per case) and "passing" (score 2,3,4). We treated each task as a binary classification task in a text classification pipeline. First, we treated each note as a bag of tokens and weight each token with term frequency-inverse document frequency (TFIDF) a numerical statistic that reflects howimportant aword is to a document. We then applied 3 different classification algorithms (random forests, support vector machines, and Bayesian logistic regression) and measured discriminatory performance using Area Under Curve (AUC) in a cross validation evaluation design. RESULTS: TFDIF performed with AUCs between 0.669 and 0.905. Logistic regression provided the highestAUC in four cases: Difficulty Sleeping (0.905), Shoulder Pain (0.618), Failure To Thrive (0.717) and Abdominal Pain (0.892). As we observed the highest AUCs in Difficulty Sleeping and Abdominal Pain cases, we have begun to refine the algorithm for these two cases by identifying the importance features that lead faculty to give students to a higher grade and improve the accuracy of NLP based scoring. Promising features include the presence and sequence of certainwords in the problem representation, sentence length in the management section, ranking of the differential diagnosis, sequence between key words (e.g. rule out appendicitis), and evidence of "thinkingness" or what many call semantic qualifiers. CONCLUSIONS: With additional effort to build targeted case specific classifiers for clinical content and reasoning, a validated machine-learning model may achieve partial or full automation of grading of the notes. This work, which builds on decades of clinical decision-making and critical reasoning research, may provide medical trainees with more and potentially better feedback; facilitating learning of clinical reasoning, freeing faculty to coach this process, and in the long run impacting healthcare quality and patient safety

BACKGROUND: The AAMC has identified 13 Entrustable Professional Activities (EPAs) that all entering residents should be expected to perform on day 1 of residency without direct supervision regardless of specialty choice. We developed an immersive, Night on call (NOC) simulation to understand the measure of entrustment of all 13 Core EPAs from the perspective of patients, nurses, attendings, and peers. METHODS: NOC is a 4-hour simulation, during which a medical student rotates through a series of authentic clinical coverage scenarios including: 4 standardized patient (SP) cases with varying degrees of complexity, each of which require first answering a call from a standardized nurse, (SN), then evaluating a SP with the SN in the room, making immediate management decisions and writing a coverage note; a phone call to an attending (Attn, an experienced clinician) to orally present (OP), and discuss the case, formulation of a clinical question and finding a best answer using digital library resources (EBM), a test of ability to recognize a pre-entrustable peer, and a handoff of 4 cases to a peer (HOff, portrayed by an senior medical student). Competency assessments were based on validated tools where available. Each rater provided an entrustment judgment. This included 9 raters providing a total of 16 entrustment judgments: 4 SPs and 3 SNs (1 rating competency and 1 rating communication each), 1 Attn based on OP, 1 peer rating based on the HOff (1 item each). Raters were trained in both case portrayal and rating reliability. This study is IRB approved. After exploring the relationships among competency measures and entrustment judgements, to test the hypothesis that NOC measures trustworthiness of our near graduates, we conducted a one-factor (entrustment) confirmatory factor analysis (CFA) with the 16-entrustment items allowing the ratings from the same raters and between raters on the same case to correlate. The CFA was conducted with a means and variance adjusted weighted-least squares estimation (WLSMV) to take the ordinal distributions of the entrustment items into account. RESULTS: 73 medical students (39 women; Age 26.5 (+2.6) years) completed NOC. The one-factor CFA model fit the data (chi² = 155.27, df = 112, p < .001, CFI = 0.97, TLI = 0.97, RMSEA = 0.07, p > 0.05). All but 2 of the 16 factor loadings were greater than 0.3, (Attn factor loading = 0.23 and the SP ratings from the first clinical case of NOC sequence (0.21)). CONCLUSIONS: A single-factor model with 16measures fit the entrustment framework within an ecologically valid simulated workplace suggesting that an individual student's clinical trustworthiness is measurable across discrete work activities. This work provides an assessment framework for the educational handoff from medical school to residency to ensure quality of care and patient safety

NEEDS AND OBJECTIVES: To address rising education costs, physician shortages, and the need for educational reform, several medical schools have developed accelerated 3-year MD programs. In 2013, NYU School of Medicine began its new 3-year MD program with guaranteed acceptance into residency upon graduation. Using the AAMC's 13 Core Entrustable Professional Activities for Entry into Residency (CEPAER) framework, we designed an immersive 4-hour simulated "Night on Call" (NOC) experience to compare performance of our first graduating cohort of fifteen 3-year MD students (3A), with third (3T) and fourth year (4T) students in the traditional 4-year MD program. SETTING AND PARTICIPANTS: 73 medical students (39 women, age 26.5 (+2.6) years; 36 '3T', 12 '3A', 25 '4T') completed an IRB-approvedNOC at our simulation center 4 weeks prior to the end of their third or final year of medical school. DESCRIPTION: We developed NOC to measure competence and entrustment across all 13 CEPAERs from the perspective of patients, nurses, and attendings. During the simulation, a medical student rotated through a series of 8 clinical coverage scenarios including: 4 standardized patient (SP) cases with varying degrees of complexity, each of which require answering a call from a standardized nurse (SN), evaluating an SP with the SN in the room, making immediate management decisions and writing a coverage note; a phone call to an experienced clinician to orally present (OP) the case; formulation of a clinical question and finding the most appropriate evidence-based medicine (EBM) answer using digital library resources; a clinical vignette (CV) to test ability to recognize a pre-entrustable peer; and a handoff (HO) of 4 cases to a peer (a senior medical student). CEPAERs assessments based on validated tools included communication, physical exam, patient education and interprofessional teamwork skills assessed by an SP and SN, and clinical reasoning based on notes, OP, EBM, CV, HO. Each rater also provided an entrustment judgment. EVALUATION: Although overall student performance improved across cases and some interesting individual performance patterns emerged, there were no significant differences across the three groups in the core competency and entrustment measures evaluated across various NOC activities. DISCUSSION/REFLECTION/LESSONS LEARNED: The 13 CEPAERs are meant to define what students should be expected to perform (without direct supervision) prior to entering residency. Our results, based on multiple rater perspectives, suggest that our cohort of 3A students is as prepared for residency as their 4T counterparts

BACKGROUND: Training residents to effectively practice value-based care is challenging. We hypothesized that residents with better communication skills would order fewer unnecessary tests and prescribe more appropriate care. We used a USP case of a patient with uncontrolled asthma to examine the relationship between value-based care and communication skills. METHODS: A 25 year-old female USP presented as a new patient to a medicine resident's clinic, reporting asthma since childhood with worsening symptoms over the past few months. At the time of the visit, she was using her albuterol inhaler multiple times daily, without any additional asthma treatment, and was unsure whether she was using it properly. Data was collected using two forms of assessment: a post-visit USP checklist and a systematic review of the corresponding clinic note to examine treatment recommendations including referrals and quality of documentation. The USP checklist measured communication, patient education, and assessment skills. Each response option included descriptive behavioral anchors and was rated as not done, partly done, or well done. Domain scores were calculated as percent items rated well done. RESULTS: 141 USP visits were made from 2009 to 2016 with a mean visit length = 88 min, SD= 28 min (range: 40 to 180 min). Almost all residents (92%) evaluated the patient's asthma with a pulmonary examination. The most common treatment prescribed was albuterol and an inhaled steroid, with or without a spacer (79%). The majority of residents (53%) did not order any additional studies; 21% ordered one study, and 26% ordered two or more studies. Study orders fell into one of three categories: gold (appropriate/recommended: PFTs, flu shot, HIV), grey (pulmonary consult, HCG), or inappropriate (TSH, A1C). Across the 141 visits, 129 studies were ordered; 46% were gold, 5% were grey, and 49% were inappropriate. The most common study ordered was a PFT (31%). 87% of single study orders were gold, but 92% of multiple orders included at least one inappropriate study. Residents who did not order any studies had significantly higher patient education and counseling skills than residents who ordered one or more studies (54% vs 34%, p = 0.00) and were more likely to explain how to correctly use an inhaler than residents who ordered one or more studies (48% vs 27%, p = 0.01). These residents also had significantly higher management and treatment skills (61% vs 39%, p = 0.00) and overall communication skills (68% vs 55%, p = 0.01). There were no significant differences between groups in medications prescribed or in quality of documentation. CONCLUSIONS: Effective communication skills may contribute to valuebased care through appropriate patient education and ordering of fewer inappropriate studies. Rigorous curricula and assessment of resident's patient education skills should be in place to help both patients and health care system achieve value-based care