Faculty Bibliography

OBJECTIVES: To examine internal medicine and emergency medicine healthcare provider perceptions of usefulness of specific clinical prediction rules. SETTING: The study took place in two academic medical centres. A web-based survey was distributed and completed by participants between 1 January and 31 May 2013. PARTICIPANTS: Medical doctors, doctors of osteopathy or nurse practitioners employed in the internal medicine or emergency medicine departments at either institution. PRIMARY AND SECONDARY OUTCOME MEASURES: The primary outcome was to identify the clinical prediction rules perceived as most useful by healthcare providers specialising in internal medicine and emergency medicine. Secondary outcomes included comparing usefulness scores of specific clinical prediction rules based on provider specialty, and evaluating associations between usefulness scores and perceived characteristics of these clinical prediction rules. RESULTS: Of the 401 healthcare providers asked to participate, a total of 263 (66%), completed the survey. The CHADS2 score was chosen by most internal medicine providers (72%), and Pulmonary Embolism Rule-Out Criteria (PERC) score by most emergency medicine providers (45%), as one of the top three most useful from a list of 24 clinical prediction rules. Emergency medicine providers rated their top three significantly more positively, compared with internal medicine providers, as having a better fit into their workflow (p=0.004), helping more with decision-making (p=0.037), better fitting into their thought process when diagnosing patients (p=0.001) and overall, on a 10-point scale, more useful (p=0.009). For all providers, the perceived qualities of useful at point of care, helps with decision making, saves time diagnosing, fits into thought process, and should be the standard of clinical care correlated highly (>/=0.65) with overall 10-point usefulness scores. CONCLUSIONS: Healthcare providers describe clear preferences for certain clinical prediction rules, based on medical specialty.

BACKGROUND: Medication nonadherence is widespread, but there are few efficient means of detecting medication nonadherence at the point of care. Visit-to-visit variability in clinical biomarkers has shown inconsistent efficiency to predict medication adherence. OBJECTIVE: To examine the performance of visit-to-visit variability (VVV) of hemoglobin A1c to predict nonadherence to antidiabetic medications. METHODS: In this cross-sectional study using a clinical and administrative database, adult members of a managed care plan at a safety-net medical center from 2008 to 2012 were included if they had >/= 3 noninsulin antidiabetic prescription fills within the same class and >/= 3 A1c measurements between the first and last prescription fills. The independent variable was VVV of A1c (within-subject standard deviation of A1c), and the dependent variable was medication adherence (defined by medication possession ratio) determined from pharmacy claims. Unadjusted and adjusted multivariate logistic regression models were created to examine the relationship between VVV of A1c and medication nonadherence. Receiver-operating characteristic (ROC) curves assessed the performance of the adjusted model at discriminating adherence from nonadherence. RESULTS: Among 632 eligible subjects, mean A1c was 7.7% +/- 1.3%, and 83% of the sample was nonadherent to antidiabetic medications. Increasing quintiles of VVV of A1c and medication nonadherence were both associated with increased within-subject mean A1c and younger subject age. The logistic regression model (adjusted for age, sex, race/ethnicity, within-subject mean A1c, number of A1c measurements, number of days between the first and last antidiabetic medication prescription fills, and rate of primary care visits during the study period) showed a nonsignificant association of VVV of A1c and medication nonadherence (OR = 1.19, 95% CI = 0.42-3.38 for the highest quintile of VVV). Adding VVV of A1c to a model including age, sex, and race only modestly improved the C-statistic of the ROC curve from 0.6786 to 0.7064. CONCLUSIONS: VVV of A1c is not a robust predictor of antidiabetic medication nonadherence. Further innovation is needed to develop novel methods of detecting nonadherence.

INTRODUCTION: The promise of Clinical Decision Support (CDS) has always been to transform patient care and improve patient outcomes through the delivery of timely and appropriate recommendations that are patient specific and, more often than not, are appropriately actionable. However, the users of CDS-providers-are frequently bombarded with inappropriate and inapplicable CDS that often are not informational, not integrated into the workflow, not patient specific, and that may present out of date and irrelevant recommendations. METHODS: The integrated clinical prediction rule (iCPR) project was a randomized clinical trial (RCT) conducted to determine if a novel form of CDS, i.e., clinical prediction rules (CPRs), could be efficiently integrated into workflow and result in changes in outcomes (e.g., antibiotic ordering) when embedded within a commercial electronic health record (EHR). We use the lessons learned from the iCPR project to illustrate a framework for constructing usable, useful, and effective actionable CDS while employing off-the-shelf functionality in a production system. Innovations that make up the framework combine the following: (1) active and actionable decision support, (2) multiple rounds of usability testing with iterative development for user acceptance, (3) numerous context sensitive triggers, (4) dedicated training and support for users of the CDS tool for user adoption, and (5) support from clinical and administrative leadership. We define "context sensitive triggers" as being workflow events (i.e., context) that result in a CDS intervention. DISCUSSION: Success of the framework can be measured by CDS adoption (i.e., intervention is being used), acceptance (compliance with recommendations), and clinical outcomes (where appropriate). This framework may have broader implications for the deployment of Health Information Technology (HIT). RESULTS AND CONCLUSION: iCPR was well adopted(57.4% of users) and accepted (42.7% of users). Usability testing identified and fixed many issues before the iCPR RCT. The level of leadership support and clinical guidance for iCPR was key in establishing a culture of acceptance for both the tool and its recommendations contributing to adoption and acceptance. The dedicated training and support lead to the majority of the residents reporting a high level of comfort with both iCPR tools strep pharyngitis (64.4 percent) and pneumonia (62.7 percent) as well as a high likelihood of using the tools in the future. A surprising framework addition resulted from usability testing: context sensitive triggers.

Translating research findings into practice promises to standardise care. Translation includes the integration of evidence-based guidelines at the point of care, discerning the best methods to disseminate research findings and models to sustain the implementation of best practices.By applying usability testing to clinical decision support(CDS) design, overall adoption rates of 60% can be realised.What has not been examined is how long adoption rates are sustained and the characteristics associated with long-term use. We conducted secondary analysis to decipher the factors impacting sustained use of CD Stools. This study was a secondary data analysis from a clinical trial conducted at an academic institution in New York City. Study data was identified patients electronic health records (EHR). The trial was to test the implementation of an integrated clinical prediction rule(iCPR) into the EHR. The primary outcome variable was iCPR tool acceptance of the tool. iCPR tool completion and iCPR smartest completion were additional outcome variables of interest. The secondary aim was to examine user characteristics associated with iCPR tool use in later time periods. Characteristics of interest included age, resident year, use of electronic health records (yes/no) and use of best practice alerts (BPA) (yes/no). Generalised linear mixed models (GLiMM) were used to compare iCPR use over time for each outcome of interest: namely, iCPR acceptance, iCPR completion and iCPR smartset completion.GLiMM was also used to examine resident characteristics associated with iCPR tool use in later time periods; specifically, intermediate and long-term (ie, 90+days). The tool was accepted, on average, 82.18% in the first 90 days (short-term period). The use decreases to 56.07% and 45.61% in intermediate and long-term time periods, respectively. There was a significant association between iCPR tool completion and time periods(p<0.0001). There was no significant difference in iCPR tool completion between resident encounters in the intermediate and long-term periods (p<0.6627). There was a significant association between iCPR smartset completion and time periods (p<0.0021). There were no significant associations between iCPR smartest completion and any of the four predictors of interest. We examined the frequencies of components of the iCPR tool being accepted over time by individual clinicians. Rates of adoption of the different components of the tool decreased substantially over time. The data suggest that over time and prolonged exposure to CDS tools, providers are less likely to utilise the tool. It is not clear if it is fatigue with the CDS tool, acquired knowledge of the clinical prediction rule, or gained clinical experience and gestalt that are influencing adoption rates. Further analysis of individual adoption rates over time and the impact it has on clinical outcomes should be conducted.

BACKGROUND: Evidence-based solutions for changing health behaviors exist but problems with feasibility, sustainability, and dissemination limit their impact on population-based behavior change and maintenance. OBJECTIVE: Our goal was to overcome the limitations of an established behavior change program by using the inherent capabilities of smartphones and wireless sensors to develop a next generation mobile health (mHealth) intervention that has the potential to be more feasible. METHODS: In response to the clinical need and the growing capabilities of smartphones, our study team decided to develop a behavioral hypertension reduction mHealth system inspired by Dietary Approaches to Stop Hypertension (DASH), a lifestyle modification program. We outline the key design and development decisions that molded the project including decisions about behavior change best practices, coaching features, platform, multimedia content, wireless devices, data security, integration of systems, rapid prototyping, usability, funding mechanisms, and how all of these issues intersect with clinical research and behavioral trials. RESULTS: Over the 12 months, our study team faced many challenges to developing our prototype intervention. We describe 10 lessons learned that will ultimately stimulate more effective and sustainable approaches. CONCLUSIONS: The experiences presented in this case study can be used as a reference for others developing mHealth behavioral intervention development projects by highlighting the benefits and challenges facing mHealth research.

PURPOSE: Usability testing can be used to evaluate human-computer interaction (HCI) and communication in shared decision making (SDM) for patient-provider behavioral change and behavioral contracting. Traditional evaluations of usability using scripted or mock patient scenarios with think-aloud protocol analysis provide a way to identify HCI issues. In this paper we describe the application of these methods in the evaluation of the Avoiding Diabetes Thru Action Plan Targeting (ADAPT) tool, and test the usability of the tool to support the ADAPT framework for integrated care counseling of pre-diabetes. The think-aloud protocol analysis typically does not provide an assessment of how patient-provider interactions are effected in "live" clinical workflow or whether a tool is successful. Therefore, "Near-live" clinical simulations involving applied simulation methods were used to compliment the think-aloud results. This complementary usability technique was used to test the end-user HCI and tool performance by more closely mimicking the clinical workflow and capturing interaction sequences along with assessing the functionality of computer module prototypes on clinician workflow. We expected this method to further complement and provide different usability findings as compared to think-aloud analysis. Together, this mixed method evaluation provided comprehensive and realistic feedback for iterative refinement of the ADAPT system prior to implementation. METHODS: The study employed two phases of testing of a new interactive ADAPT tool that embedded an evidence-based shared goal setting component into primary care workflow for dealing with pre-diabetes counseling within a commercial physician office electronic health record (EHR). Phase I applied usability testing that involved "think-aloud" protocol analysis of eight primary care providers interacting with several scripted clinical scenarios. Phase II used "near-live" clinical simulations of five providers interacting with standardized trained patient actors enacting the clinical scenario of counseling for pre-diabetes, each of whom had a pedometer that recorded the number of steps taken over a week. In both phases, all sessions were audio-taped and motion screen-capture software was activated for onscreen recordings. Transcripts were coded using iterative qualitative content analysis methods. RESULTS: In Phase I, the impact of the components and layout of ADAPT on user's Navigation, Understandability, and Workflow were associated with the largest volume of negative comments (i.e. approximately 80% of end-user commentary), while Usability and Content of ADAPT were representative of more positive than negative user commentary. The heuristic category of Usability had a positive-to-negative comment ratio of 2.1, reflecting positive perception of the usability of the tool, its functionality, and overall co-productive utilization of ADAPT. However, there were mixed perceptions about content (i.e., how the information was displayed, organized and described in the tool). In Phase II, the duration of patient encounters was approximately 10 min with all of the Patient Instructions (prescriptions) and behavioral contracting being activated at the end of each visit. Upon activation, providers accepted the pathway prescribed by the tool 100% of the time and completed all the fields in the tool in the simulation cases. Only 14% of encounter time was spent using the functionality of the ADAPT tool in terms of keystrokes and entering relevant data. The rest of the time was spent on communication and dialog to populate the patient instructions. In all cases, the interaction sequence of reviewing and discussing exercise and diet of the patient was linked to the functionality of the ADAPT tool in terms of monitoring, response-efficacy, self-efficacy, and negotiation in the patient-provider dialog. There was a change from one-way dialog to two-way dialog and negotiation that ended in a behavioral contract. This change demonstrated the tool's sequence, which supported recording current exercise and diet followed by a diet and exercise goal setting procedure to reduce the risk of diabetes onset. CONCLUSIONS: This study demonstrated that "think-aloud" protocol analysis with "near-live" clinical simulations provided a successful usability evaluation of a new primary care pre-diabetes shared goal setting tool. Each phase of the study provided complementary observations on problems with the new onscreen tool and was used to show the influence of the ADAPT framework on the usability, workflow integration, and communication between the patient and provider. The think-aloud tests with the provider showed the tool can be used according to the ADAPT framework (exercise-to-diet behavior change and tool utilization), while the clinical simulations revealed the ADAPT framework to realistically support patient-provider communication to obtain behavioral change contract. SDM interactions and mechanisms affecting protocol-based care can be more completely captured by combining "near-live" clinical simulations with traditional "think-aloud analysis" which augments clinician utilization. More analysis is required to verify if the rich communication actions found in Phase II compliment clinical workflows.

BACKGROUND: Effective communication of risks and benefits to patients is critical for shared decision making. PURPOSE: To review the comparative effectiveness of methods of communicating probabilistic information to patients that maximize their cognitive and behavioral outcomes. DATA SOURCES: PubMed (1966 to March 2014) and CINAHL, EMBASE, and the Cochrane Central Register of Controlled Trials (1966 to December 2011) using several keywords and structured terms. STUDY SELECTION: Prospective or cross-sectional studies that recruited patients or healthy volunteers and compared any method of communicating probabilistic information with another method. DATA EXTRACTION: Two independent reviewers extracted study characteristics and assessed risk of bias. DATA SYNTHESIS: Eighty-four articles, representing 91 unique studies, evaluated various methods of numerical and visual risk display across several risk scenarios and with diverse outcome measures. Studies showed that visual aids (icon arrays and bar graphs) improved patients' understanding and satisfaction. Presentations including absolute risk reductions were better than those including relative risk reductions for maximizing accuracy and seemed less likely than presentations with relative risk reductions to influence decisions to accept therapy. The presentation of numbers needed to treat reduced understanding. Comparative effects of presentations of frequencies (such as 1 in 5) versus event rates (percentages, such as 20%) were inconclusive. LIMITATION: Most studies were small and highly variable in terms of setting, context, and methods of administering interventions. CONCLUSION: Visual aids and absolute risk formats can improve patients' understanding of probabilistic information, whereas numbers needed to treat can lessen their understanding. Due to study heterogeneity, the superiority of any single method for conveying probabilistic information is not established, but there are several good options to help clinicians communicate with patients. PRIMARY FUNDING SOURCE: None.

BACKGROUND: Statins reduce the risk of coronary heart disease (CHD) in individuals with a history of CHD or risk equivalents. A 10-year CHD risk >20% is considered a risk equivalent but is frequently not detected. Statin use and low-density lipoprotein cholesterol (LDL-C) control were examined among participants with CHD or risk equivalents in the nationwide Reasons for Geographic and Racial Differences in Stroke study (n = 8812). METHODS: Participants were categorized into 4 mutually exclusive groups: (1) history of CHD (n = 4025); (2) no history of CHD but with a history of stroke and/or abdominal aortic aneurysm (AAA) (n = 946); (3) no history of CHD or stroke/AAA but with diabetes mellitus (n = 3134); or (4) no history of the conditions in (1) through (3) but with 10-year Framingham CHD risk score (FRS) >20% calculated using the third Adult Treatment Panel point scoring system (n = 707). RESULTS: Statins were used by 58.4% of those in the CHD group and 41.7%, 40.4% and 20.1% of those in the stroke/AAA, diabetes mellitus and FRS >20% groups, respectively. Among those taking statins, 65.1% had LDL-C <100 mg/dL, with no difference between the CHD, stroke/AAA, or diabetes mellitus groups. However, compared with those in the CHD group, LDL-C <100 mg/dL was less common among participants in the FRS >20% group (multivariable adjusted prevalence ratio: 0.72; 95% confidence interval: 0.62-0.85). Results were similar using the 2013 American College of Cardiology/American Heart Association cholesterol treatment guideline. CONCLUSIONS: These data suggest that many people with high CHD risk, especially those with an FRS >20%, do not receive guideline-concordant lipid-lowering therapy and do not achieve an LDL-C <100 mg/dL.

OBJECTIVE: This study assessed the baseline knowledge, perceptions, attitudes and behaviors of prediabetes patients in order to tailor a new technology-enhanced primary care-based lifestyle modification intervention. METHODS: Patients with a diagnosis of prediabetes were enrolled in a randomized, controlled pilot study, Avoiding Diabetes Thru Action Plan Targeting (ADAPT), a technology-based intervention to promote action plan discussions around patient-selected behavior change goals. RESULTS: A total of 54 adults (82% female) were enrolled in the pilot study. Most (89%) had comorbid conditions and mean BMI was 36. Participants exhibited high risk of diabetes knowledge (knowledge score 20 on a 32 point scale) and high levels of willingness to make changes to decrease diabetes risk. Number of daily steps was inversely correlated with perceived physical activity (r=-0.35082, p<0.001). Poorer scores on diet quality were inversely correlated with BMI. CONCLUSION: Participants in this sample demonstrated requisite levels of knowledge, self-efficacy, motivation and risk perception for effective behavior change. These data suggest that primary care-based prediabetes interventions can move beyond educational goals and focus on enhancing patients' ability to select, plan and enact action plans.