Faculty Bibliography

BACKGROUND: As healthcare increases demands, primary care physicians need evidenced-based, patient-centered care coordination, effective use of information technology, interdisciplinary team functioning and shared decision-making skills more so in underserved areas. In 2008, we documented 20 years of the NYU/Bellevue Primary Care Internal Medicine Residency Program (NYUBPC) on readiness for practice1. In light of the recent primary care changes we assessed our recent training of Primary Care Residents in high quality, person-centered, systems-savvy, team-based care for the underserved. Specifically we aimed to: 1. Assess the NYUBPCP impact on graduate career choices, values and style 2. Elicit reflections that illustrate complexities in educating primary care physicians METHODS: We surveyed 56 graduates of the NYUBPCP from 2007-2014. The 44 question survey included 12 open-ended questions about career path, current practice, preparedness for practice and specifics about how aspects of training provided necessary skills and knowledge. Responses were unidentified. We received 37 responses, (66%). RESULTS: 36 respondents currently provide clinical care, with about 40% of their time spent in a primary care setting (S.D. 32%). On a 4- point scale 85% either agreed or strongly agreed with Primary Care as a career choice. 74% felt prepared for the challenges of a primary care practice, rating clinical experiences with underserved communities, and the psychosocial, clinical epidemiology and health policy focus as essential aspects of training. All but 4 provide care to medically underserved populations. They valued the community of peers and colleagues that the NYUBPCP provided. While 53% rated their clinical site as hectic/chaotic (4 or 5 on a 5 point scale), only 6% reported persistently feeling burnout. 19% reported at least one symptom of burnout. Qualitative analyses revealed overlapping themes in alumni perceptions of how residency influenced current practice, aspects of training that were difficult to implement and expectations for the future directions of primary care. Responses demonstrated a mismatch between the "purity" of primary care practice graduates strove to achieve after residency and the actuality of a practice influenced by external factors (e.g. time pressures, reimbursement issues and metric achievements). Some found it difficult to be involved with research or advocacy while in full-time clinical practice. Graduates believed the future of primary care lies in a team-based approach. CONCLUSIONS: A training program emphasizing rigorous curriculum, committed role modeling, care of the underserved, and strong residency community for support continues to document high rates of retention in primary care. They are well adapted entering physicians with the skills and attitudes necessary to succeed in primary care and become educators of the next generation

BACKGROUND: The shift toward a Patient Centered Medical Home has redefined healthcare delivery to be a patient centered affair. While this is beneficial, it also calls for patients to be more activated in the doctor's office and responsible in their self-directed care outside of the office. For patients with chronic diseases, the burden of illness is even higher and requires significantly more effort in disease selfmanagement. We developed 4 hour Patient Empowerment Program (PEP) to bridge that gap through a program training patients with diabetes in the skills necessary to communicate effectively with providers and engage in shared decision making (SDM). Previously, we reported improved diabetes self-care behaviors at 6 months post intervention based on standardized questionnaires. This study examined participants' perspectives on PEP from 1-2 years post intervention. METHODS: 71 patients with type 2 diabetes mellitus were recruited from 2 urban safety-net hospitals to participate in PEP. 33 patients completed the intervention and 28 patients completed a 6-month follow-up assessment. Participants were predominantly low-income, racial minorities, with limited health literacy (Newest Vital Sign M= 2.21, SD = 1.67). We reached 22 of those 28 participants, at 11 to 20months post intervention (M= 16, SD = 2.31). They were interviewed via telephone, with structured open-ended questions asking them to reflect on what they took away from the classes, and whether they'd behaved differently during doctor visits or cared for their diabetes differently since the classes. A qualitative analysis was made of these responses, using Dedoose software to assist in analysis. RESULTS: All 22 patients recalled PEP and could identify key concepts from it. Participants referenced the doctor-patient relationship, from "how to improve relationship with my doctor" to "practicing role of doctor and patient in interaction and relationship". 86% of participants identified changes in their behavior since PEP- 36% identifying that they share more information with their doctor now than they did before, 18% reporting that they ask more questions, and another 18% reporting that they are more proactive in their diabetes care. When asked about their diabetes self-management, prevalent themes were better diet/food choices, exercise, and adherence to medication. 3 patients reported losing weight since the classes. CONCLUSIONS: Despite follow up occurring almost a year later for some, and almost 2 years for others, the message of PEP was clear and compelling. Participant's perspective on their role as a patient changed from a passive recipient of healthcare to a more engaged and activated one. They felt empowered to participate in SDM with their doctors and more comfortable speaking up for their preferences. While further validation is necessary, PEP offers an important way to prepare patients to become true partners with their providers

BACKGROUND: It is critically important for medical schools to understand how well prepared their graduates are for residency and yet we do not have a full understanding of how well competencies, assessed in medical school, transfer to residency. This study explores how communication skills measured in a high-stakes, rigorous, comprehensive OSCE in the 3rd year of medical school are related to performance in a similar OSCE in residency and to Residency Program Directors' ratings of intern competence. METHODS: We analyzed communication skills from three time points in a longitudinal cohort of NYU graduates who entered our Internal Medicine Residency (n = 42). 39 provided consent for their GME-UME data to be compiled into a longitudinal, de-identified educational research database through an IRB-approved Registry. Communication skills were measured using a behaviorally anchored 15-item checklist across the 8-station, pass/fail, MS3 OSCE and then midway through PGY2 of residency in a 6- station OSCE (score =% of items rated well done). SPs also provided an overall rating of communication skills (not recommend, with reservations, recommend, highly recommend). In between, at the end of intern year, residents were also rated by their Program Directors on communication skills (and other competencies) using a 4-pt scale. RESULTS: OSCE communication performance assessed in medical school was modestly associated with performance in residency (r = .26, p = .07) but not with Program Directors' ratings of residents' communication skills as interns (r = .11, p = .28). Number of cases in which medical students were "not recommended" for their communication skills was negatively associated with residency OSCE communication scores (r = -.33, p = .05) and positively associated with number of "not recommends" (r = .46, p = .01) but not with Directors' ratings of interns (r = -.08, p = .49). Number of not recommends independently explained more variance in subsequent residency communication scores than did medical school performance (9% vs. 5%). While average OSCE communication scores improved from medical school to residency (65 to 71%), those with 2 or more "not recommends" improved significantly more than those with 1 or no not recommends. Overall, most learners' (21/39) communication scores improved substantially; less than a quarter (7) decreased; and about a quarter (11) were stable. CONCLUSIONS: While communication scores from medical school are associated with similarly measured scores in residency, SPs' decisions to "not recommend" students appear to serve as an independent indicator of future skill deficits. Patterns of change, however, are not necessarily straightforward: students with the most "not recommends" improved the most. The ability to track competency assessments longitudinally is essential for understanding the transition from medical school to residency and future research will benefit from larger sample sizes and the inclusion of learner characteristics that may explain developmental patterns

BACKGROUND: Patient experience is an important quality indicator, and healthcare organizations spend considerable resources assessing patient satisfaction. Yet a view of patient experience gleaned from patient satisfaction measures tends to show high levels of reported satisfaction, with little variation. Unannounced standardized patients (USPs) have been used to assess providers' clinical skills, but can also provide other information about the healthcare encounter. This study examined the concordance between USP and patient reports of care at the same site. METHODS: Data was gathered at Bellevue Hospital Primary Care Clinic, a city safety-net hospital. USPs assess internal medicine residents training there, and complete a behaviorally anchored checklist of resident skills and interactions with other staff, wait times, ease of clinic navigation, and perceptions of team functioning. Data from 155 USP visits from July 2015-Oct 2016 was used in this study. Independently, as part of team-training efforts in the Primary Care Clinic, patient satisfaction surveys were collected, addressing similar issues. At the end of a clinic visit, research assistants unrelated to patient care asked patients to complete a 30-item survey. 118 surveys were completed between July-November 2016. 11 items appeared on both scales (though worded slightly different) and were used in this comparison. These included questions about clerical (CA) and patient care associates (PCA), and providers (MDs, NPs, PAs), provision of information, team functioning and clinic environment. Of the 11 items, 4 had the same response choices. 7 had differing numbers of responses (eg 4 vs 3 point Likert scales), evenly distributed across patient and USP scales. For each of these items, we collapsed items so to maximize positive ratings (eg. on a 4 point scale from poor to excellent, "good" and "excellent" were combined rather than "good" and "fair"). Chi-square analyses were computed to examine group differences. RESULTS: On chi-square analyses, 9 of the 11 items significantly differed between the USP and patient groups, with patients more likely to have positive ratings. These included rating PCAs as friendlier (x² = 8.67(1,206), p = .003) and providers better at answering questions (x² = 11.75 (2,265), p = .003); reporting that they received sufficient/clear instructions about medication refills and follow-up (x² = 29.5(2,264), p = .0001); finding the clinic atmosphere calmer than did USPs (x² = 10.5 (2,265), p=.005) and noting that the team functioned better (x² = 7.31(2,268), p = .026). There were no significant differences in willingness to recommend the clinic or on clarity of CAs' communication. CONCLUSIONS: Results of this study document the differing perspectives of patients and USPs. Consistent with previous work, patients in our study tended to rate most items higher than did the USPs. USPs provide a different, and likely a more critical look at the clinical setting and this information can enhance efforts to improve patient experience. (Table Presented)

NEEDS AND OBJECTIVES: Team-based primary care (PC) is seen as the best way to provide proactive, patient-centered quality care. However, developing these team-based skills is difficult in the ever-shifting, stressful healthcare environment. We sought to develop effective training to enhance team functioning at an urban safety-net hospital, with the goal of clinical transformation (e.g. improving clinic flow, enhancing care for patients with diabetes). SETTING AND PARTICIPANTS: Team training intervention at Bellevue Hospital's Adult Ambulatory Care Center, flagship of the NYC Health & Hospitals (H +H), serving poor, diverse patients with complex medical and social needs. There are 4 adult PC teams, each with 8 attending physicians, 20 residents, 1-2 physician assistants, 2 nurses, 5 patient care associates (PCA), and 2 clerical associates (CA), all caring for a panel of ~7,500 patients. To date, we have completed a training cycle for one team, with 26 members: 10 providers (7 MDs, 2 PAs, and 1 NP), 4 RNs, 5 PCAs, 3 CAs, and 4 residents participating. We are scheduled to complete training of a second team in February 2017, with the other 2 to follow. DESCRIPTION: We partnered with a parallel NYC H + H effort, enabling a seamless NYU-HRSA/NYC H + H program with increased time allotted. This includes 4 three-hour workshops co-led by NYC H + H and NYU-HRSA faculty. Each workshop blends activating, team-building exercises for teams; mini-lectures on topics like roles and responsibilities, communication skills, huddles, and experiential activities using the team's patient data. This is reinforced with seven, 30-min biweekly meetings to follow up on teamidentified topics and facilitate team members' quality improvement projects. EVALUATION: A 31-item (each item rated 0-3), retrospective pre/post survey was administered to trainees after training, addressing individual skills and attitudes (16 items) and team functioning (15 items)14 of 26 participants (54%) in team 1 completed the survey, and Team 2 participants will complete the survey in Feb. 2017. Training resulted in increased rating of individual skills t = 4.86, p < .0001) and team functioning (t = 4.02, p = .003). Additional metrics, including tracking teams' QI efforts and assessing patient experience (e.g. Unannounced Standardized Patient reports) and administrative and panel level data, are ongoing. DISCUSSION/REFLECTION/LESSONS LEARNED: Implementation of successful team training in an under-resourced, urban primary care setting is challenging. It demands flexibility, tailoring to participants' concerns; and responding to changing clinical and administrative circumstances. Essential to success was partnering with team members to guide the training

BACKGROUND: Effective smoking cessation counseling improves smokers' health and quality of life. As part of our assessment program, an Unannounced Standardized Patient (USP) case was developed to measure residents' performance in a routine visit with a smoker. METHODS: The USP was a 40 year-old male new patient presenting with heartburn. He began smoking up to two packs/day at 22 years old; at the time of the visit, he reports having cut down to one pack/day and quitting cold turkey twice in the past only to return to smoking. If the resident engages him, he discusses his relationship with smoking and the possibility of quitting. TheUSP received 6 hours of character and checklist training to ensure standardized portrayal and evaluation. Data was collected using 2 forms of assessment: a post-visit USP checklist and a systematic review of the EMR (lab orders, prescriptions, and referrals). The 170- item USP checklist measured communication, patient education, assessment skills, and case-specific items. Each response option included descriptive behavioral anchors and was rated as not done, partly done, or well done. RESULTS: Data was examined from 73 USP visits from 2009-2015. Mean visit length = 37 min, SD = 15 min (range: 15 to 95 min). Overall communication scores ranged from 17 to 100% with an average of 62% (Cronbach's alpha = 0.75). All residents documented History of Tobacco Use or Tobacco Use Disorder in the EMR, and the majority (82%) prescribed smoking cessation medication. There was variation in the sophistication of smoking cessation-counseling approach. Whilemost residents (78%) discussed the risks of smoking and/or the benefits to quitting, significantly fewer (48%) explored the patient's view of the pros and cons of his smoking (p = 0.00). Residents who prescribed smoking cessation medication and discussed risks/benefits to smoking/quitting (N = 31) were compared to residents who did the same but also invited the patient to discuss his personal pros and cons of smoking (N = 29). Groups were not significantly different by PGY or gender. Patients who were asked to discuss their pros/cons rated the resident higher on patient activation questions (0-2 point scale), including "Helped you understand the importance of quitting smoking" (1.38 vs 0.90, p = 0.00), "Made you want to change your smoking" (1.10 vs 0.52, p = 0.00), and "Made you feel like you would be able to quit smoking" (1.07 vs 0.35, p = 0.00). There were no significant differences in labs ordered, referrals to a smoking cessation program, or quality of documentation. CONCLUSIONS: While all residents ask about tobacco use and most appropriately prescribe medication, fewer than half demonstrate the skills known to motivate patients to quit smoking. Curricula needs to reinforce the importance of a patient discussing their personal relationship with smoking in order to feel activated and willing to engage in cessation

NEEDS AND OBJECTIVES: Standardized Patients (SPs) are integral to health care professions (HCPs) training. We must understand this workforce, make effective use of SPs' skills, and ensure they accurately portray cases and rate learners. To be authentic, simulation should reflect the demographics of the population served, while providing exposure to less commonly seen patients. We created an SP database to facilitate our work with SPs; review their demographic characteristics; and align information on SP performance -to better serve our educational mission. SETTING AND PARTICIPANTS: NYSIM (Simulation Center for NYU Langone and the City University of NY) serves hundreds of HCP training programs for learners at all levels. While sharing common resources, many programs independently recruit and train SPs. DESCRIPTION: We fielded a web-based survey for SPs and staff to populate the database. Survey items were iteratively reviewed by staff and SPs to ensure items elicited key information. Questions included basic demographics; SP experience/training; other professional background; and relevant physical findings (eg scars, cardiac findings). SPs also uploaded a headshot and resume. Staff separately input information about SPs' work on cases and programs; information about case portrayals; types of cases for which the SP is best suited, and other relevant information. EVALUATION: To date, we have 232 SP surveys, representing the majority of SPs at NYSIM. Demographics included gender (43% male, 56% female, 1% transgender), age range (x = 34.9 years, range teen to 75+) and selfidentified race (71% Caucasian, 17% African-American, 25.6% Asian/South Asian, 3.5% Middle Eastern, 3% Native American/Pacific Islander and 9% other). 22%are bilingual, with over 20 languages represented. SPs had a broad range of SP experience (x = 2.8 years, S.D. = 1.8, range 0-20 years). Almost all SPs were trained in basic case portrayal, with others being trained in aspects of the physical exam, emotional issues, giving feedback, and high stakes rating. SPs bring other skills to their work, including teaching (75%) or healthcare (12%). Survey information helps educators recruit SPs and identify (re)training needs. SPs who perform high stakes exams or are Unannounced Standardized Patients are shielded from general recruitment in order to maintain their anonymity. DISCUSSION/REFLECTION/LESSONS LEARNED: An SP database is useful for a high volume simulation center. Information in a searchable SP data base allows programs to understand the potential pool and expertise of SPs, and to track learners' exposure to specific SPs (this is relevant as our internal data reveal that SPs with more experience tend rate towards the middle of the scale). The demographic characteristics of our SPs broadly match the profile of our healthcare systems, and tracking the data allows us to maintain a good fit between SPs and our environment

BACKGROUND: Safe, high quality outpatient care often depends on the degree to which patients understand their situation and how to follow through on physician recommendations. However, we do not know enough about how often physicians focus on ensuring that their patients have achieved these understandings by the end of the visit and whether such end-of-visit practices are associated with physicians' communication, patient education and activating skills. METHODS: Two Unannounced Standardized Patient cases (highly trained actors who present as real patients) were delivered to 71 internal medicine residents in two clinics: one required the physician to identify a patient's depression and engage him in follow-up care, and the other required the physician to recognize a patient's failure to use her asthma medicine correctly and educate her in using it properly. End-of-visit practices were: reviewing the plan; asking if further questions; giving information about follow-up care and further contact; and helping the patient navigate the system in order to follow through on next steps. Each was assessed by the SP as not done, partly done, or well done. SPs also rated physicians' communication skills, patient activating skills, and case-specific education skills. Summary scores were calculated as% of items well done. RESULTS: Close to three-quarters of the physicians reviewed the plan with the patient and invited further questions in the depression case and slightly more than half did so in the asthma case (56 and 60%). Patients were given complete information about follow-up care and how to navigate the system in just under half of depression visits (49 and 47%) and just over half of asthma visits (58 and 58%). On average, residents were rated as performing 61% of these 8 items well (SD 28%) across both cases. Primary care residents performed significantly better than categorical internal medicine residents (67%vs 47%, p = .004). There were no differences by physician gender. End of visit scores were significantly positively correlated with both general and casespecific clinical skills, and after controlling for the variance contributed by the program (R2 = 12%, p = .004), case-specific education scores explained 10% of the variance in end of visit score (p = .005), patient activating skills 10% of the variance (p = .002) and communication skills 13% of the variance (p = .001). With all variables in the model, only the general communication domain of patient education and counseling was independently associated with end of visit scores (Std Beta = .35, p = .015). CONCLUSIONS: Had these patients been real patients, in one-quarter to onehalf of the visits, the patient would have left not fully understanding the plan or how to follow-through on care. Resident physicians with more effective communication and patient activating skills tended to provide safer end-of-visit care, suggesting that these may reflect an outpatient safety orientation or skillset

BACKGROUND: The AAMC has identified 13 Entrustable Professional Activities (EPAs) that all entering residents should be expected to perform on day 1 of residency without direct supervision regardless of specialty choice. We developed an immersive, Night on call (NOC) simulation to understand the measure of entrustment of all 13 Core EPAs from the perspective of patients, nurses, attendings, and peers. METHODS: NOC is a 4-hour simulation, during which a medical student rotates through a series of authentic clinical coverage scenarios including: 4 standardized patient (SP) cases with varying degrees of complexity, each of which require first answering a call from a standardized nurse, (SN), then evaluating a SP with the SN in the room, making immediate management decisions and writing a coverage note; a phone call to an attending (Attn, an experienced clinician) to orally present (OP), and discuss the case, formulation of a clinical question and finding a best answer using digital library resources (EBM), a test of ability to recognize a pre-entrustable peer, and a handoff of 4 cases to a peer (HOff, portrayed by an senior medical student). Competency assessments were based on validated tools where available. Each rater provided an entrustment judgment. This included 9 raters providing a total of 16 entrustment judgments: 4 SPs and 3 SNs (1 rating competency and 1 rating communication each), 1 Attn based on OP, 1 peer rating based on the HOff (1 item each). Raters were trained in both case portrayal and rating reliability. This study is IRB approved. After exploring the relationships among competency measures and entrustment judgements, to test the hypothesis that NOC measures trustworthiness of our near graduates, we conducted a one-factor (entrustment) confirmatory factor analysis (CFA) with the 16-entrustment items allowing the ratings from the same raters and between raters on the same case to correlate. The CFA was conducted with a means and variance adjusted weighted-least squares estimation (WLSMV) to take the ordinal distributions of the entrustment items into account. RESULTS: 73 medical students (39 women; Age 26.5 (+2.6) years) completed NOC. The one-factor CFA model fit the data (chi² = 155.27, df = 112, p < .001, CFI = 0.97, TLI = 0.97, RMSEA = 0.07, p > 0.05). All but 2 of the 16 factor loadings were greater than 0.3, (Attn factor loading = 0.23 and the SP ratings from the first clinical case of NOC sequence (0.21)). CONCLUSIONS: A single-factor model with 16measures fit the entrustment framework within an ecologically valid simulated workplace suggesting that an individual student's clinical trustworthiness is measurable across discrete work activities. This work provides an assessment framework for the educational handoff from medical school to residency to ensure quality of care and patient safety

NEEDS AND OBJECTIVES: Skillful research staff members are critical to productive translational research teams and yet their ongoing professional development is rarely formally addressed. Through the Strategic Teamwork for Effective Practice-Mentor Development Program (STEP-MDP), we aimed to both create a community of practice (COP) for research staff and build the skills needed to enhance research team performance. SETTING AND PARTICIPANTS: We selected 16 participants of 32 stafflevel applicants from among the NYU Schools of Medicine, SocialWork and Nursing for the first STEP-MDP cohort. Participants included research assistants, coordinators, managers and directors. DESCRIPTION: We delivered 3, two-hour workshops, scheduled 3 weeks apart, focused on team communication, identifying team areas for improvement, and mentorship/coaching skills. Peer-Coaching Teams (PCTs) were created by paring participants at the same position level, and PCTs worked together at each session to explore and practice learned skills. Sessions featured brief didactics, group learning and exercises based on participants' real issues. A variety of active learning techniques such as brainstorming, role-playing, problem solving, and peer coaching were used. Practical core readings, worksheets and summary cards were provided. PCTs met between sessions to practice coaching skills, and troubleshoot problems. EVALUATION: Participants (N = 16) completed a 37-item (4 point scale) retrospective pre/post self-assessment of team behaviors and skills, and a STEP-MDP evaluation survey at the end. We saw pre-post improvements in each of 5 self-assessment domains: Communication (4 items, Pre-mean 2.66, Post mean 3.36, p = <.001), Leadership (8 items, Pre-mean 2.76, Post mean 3.55, p = <.001), Empowerment and Motivation (12 items, Pre-mean 2.86, Post mean 3.51, p = <.001), Coaching (6 items, Pre-mean 2.40, Post mean 3.58, p = <.001), and Community (3 items, Pre-mean 2.33, Post mean 3.76, p = <.001). On average, PCTs met twice (range 2-4 times) between workshop sessions. One commented on the value of working with peers in PCTs, having no one in a similar position within his immediate work environment. Participants' written comments strongly endorsed the value of the workshops for their work, with the coaching skills session seen as the most valuable. Some participants worry that skills will decrease over time without continued reinforcement. All but one participant reported that they planned to continue with the PCT. DISCUSSION/REFLECTION/LESSONS LEARNED: The number of applicants to our program suggests a need and motivation for staff to participate in the STEP-MDP. Participants' reported improved skills and sense of community. To maintain the COP and address worry about degradation of skills we are planning to remind PCTs to meet once a month and will follow up with them 3 and 6 months post intervention to evaluate their continued development. This spring we will enroll a 2nd cohort. We believe developing these core teamwork skills will lead to more collaborative, efficient, and innovative research