Faculty Bibliography

BACKGROUND:Residents receive infrequent feedback on their clinical reasoning (CR) documentation. While machine learning (ML) and natural language processing (NLP) have been used to assess CR documentation in standardized cases, no studies have described similar use in the clinical environment. OBJECTIVE:The authors developed and validated using Kane's framework a ML model for automated assessment of CR documentation quality in residents' admission notes. DESIGN, PARTICIPANTS, MAIN MEASURES/UNASSIGNED:Internal medicine residents' and subspecialty fellows' admission notes at one medical center from July 2014 to March 2020 were extracted from the electronic health record. Using a validated CR documentation rubric, the authors rated 414 notes for the ML development dataset. Notes were truncated to isolate the relevant portion; an NLP software (cTAKES) extracted disease/disorder named entities and human review generated CR terms. The final model had three input variables and classified notes as demonstrating low- or high-quality CR documentation. The ML model was applied to a retrospective dataset (9591 notes) for human validation and data analysis. Reliability between human and ML ratings was assessed on 205 of these notes with Cohen's kappa. CR documentation quality by post-graduate year (PGY) was evaluated by the Mantel-Haenszel test of trend. KEY RESULTS/RESULTS:The top-performing logistic regression model had an area under the receiver operating characteristic curve of 0.88, a positive predictive value of 0.68, and an accuracy of 0.79. Cohen's kappa was 0.67. Of the 9591 notes, 31.1% demonstrated high-quality CR documentation; quality increased from 27.0% (PGY1) to 31.0% (PGY2) to 39.0% (PGY3) (p < .001 for trend). Validity evidence was collected in each domain of Kane's framework (scoring, generalization, extrapolation, and implications). CONCLUSIONS:The authors developed and validated a high-performing ML model that classifies CR documentation quality in resident admission notes in the clinical environment-a novel application of ML and NLP with many potential use cases.

PROBLEM/OBJECTIVE:Internal medicine training programs operate under the assumption that the three-year residency training period is sufficient for trainees to achieve the depth and breadth of clinical experience necessary for independent practice; however, the medical conditions to which residents are exposed in clinical practice are not easily measured. As a result, residents' clinical educational experiences are poorly understood. APPROACH/METHODS:A crosswalk tool (a repository of international classification of diseases [ICD]-10 codes linked to medical content areas) was developed to query routinely collected inpatient principal diagnosis codes and translate them into an educationally meaningful taxonomy. This tool provides a robust characterization of residents' inpatient clinical experiences. OUTCOMES/RESULTS:This pilot study has provided proof of principle that the crosswalk tool can effectively map one year of resident-attributed diagnosis codes to both the broad content category level (for example "Cardiovascular Disease") and to the more specific condition category level (for example "Myocardial Disease"). The authors uncovered content areas in their training program that are overrepresented and some that are underrepresented relative to material on the American Board of Internal Medicine (ABIM) Certification Exam. NEXT STEPS/UNASSIGNED:The crosswalk tool introduced here translated residents' patient care activities into discrete, measurable educational content and enabled one internal medicine residency training program to characterize residents' inpatient educational experience with a high degree of resolution. Leaders of other programs seeking to profile the clinical exposure of their trainees may adopt this strategy. Such clinical content mapping drives innovation in the experiential curriculum, enables comparison across practice sites, and lays the groundwork to test associations between individual clinical exposure and competency-based outcomes, which, in turn, will allow medical educators to draw conclusions regarding how clinical experience reflects clinical competency.

BACKGROUND:Residents and fellows receive little feedback on their clinical reasoning documentation. Barriers include lack of a shared mental model and variability in the reliability and validity of existing assessment tools. Of the existing tools, the IDEA assessment tool includes a robust assessment of clinical reasoning documentation focusing on four elements (interpretive summary, differential diagnosis, explanation of reasoning for lead and alternative diagnoses) but lacks descriptive anchors threatening its reliability. OBJECTIVE:Our goal was to develop a valid and reliable assessment tool for clinical reasoning documentation building off the IDEA assessment tool. DESIGN, PARTICIPANTS, AND MAIN MEASURES/UNASSIGNED:The Revised-IDEA assessment tool was developed by four clinician educators through iterative review of admission notes written by medicine residents and fellows and subsequently piloted with additional faculty to ensure response process validity. A random sample of 252 notes from July 2014 to June 2017 written by 30 trainees across several chief complaints was rated. Three raters rated 20% of the notes to demonstrate internal structure validity. A quality cut-off score was determined using Hofstee standard setting. KEY RESULTS/RESULTS:The Revised-IDEA assessment tool includes the same four domains as the IDEA assessment tool with more detailed descriptive prompts, new Likert scale anchors, and a score range of 0-10. Intraclass correlation was high for the notes rated by three raters, 0.84 (95% CI 0.74-0.90). Scores ≥6 were determined to demonstrate high-quality clinical reasoning documentation. Only 53% of notes (134/252) were high-quality. CONCLUSIONS:The Revised-IDEA assessment tool is reliable and easy to use for feedback on clinical reasoning documentation in resident and fellow admission notes with descriptive anchors that facilitate a shared mental model for feedback.

Purpose: To carry out a large international validation of how dose prediction quality translates to plan quality in a radiotherapy knowledge-based planning (KBP) process.
Method(s): We collected dose predictions for head-and-neck cancer radiotherapy from 21 different research groups internationally who participated in the OpenKBP Grand Challenge. Each research group used the same training dataset (n=200) and validation dataset (n=40) to develop their methods. These methods predicted dose on a testing dataset (n=100), and those 2100 unique dose predictions were input to a previously published plan optimization method to generate 2100 treatment plans. The predictions and plans were compared to the ground truth dose via: (1)error, the mean absolute voxel-by-voxel difference in dose; and (2) quality, the mean and maximum deviation across 23 dose-volume histogram (DVH) criteria.
Result(s): The range in median prediction error among the top 20 methods was 2.3Gy to 12.0Gy, which was 6.8Gy wider than the range in median plan error of 2.1Gy to 5.0Gy. One method also achieved significantly lower prediction error (P<0.05; one-sided Wilcoxon test) than all the other methods, however, it generated plans with error that was not significantly lower than 28.6% of the other methods. Additionally, predicted dose was consistently lower quality than plan dose. Half (n=1050) of all predictions and plans had an average deviation that was 0.1Gy worse and 0.8Gy better than the ground truth dose, respectively. Similarly, half of all predictions had a maximum deviation that was 3.7Gy worse than the ground truth dose, which was 1.0Gy worse than half of all plans.
Conclusion(s): Many dose prediction methods can achieve low error, however, optimization often improves upon the predictions and eliminates significant differences between prediction methods. Thus, it is critical that we improve the optimization stage in KBP to get better utility out of the existing high-quality dose prediction methods

BACKGROUND: Clinical reasoning (CR) is a core component of medical training, yet residents often receive little feedback on their CR documentation. Here we describe the process of developing a machine learning (ML) algorithm for feedback on CR documentation to increase the frequency and quality of feedback in this domain.
METHOD(S): To create this algorithm, note quality first had to be rated by gold standard human rating. We selected the IDEA Assessment Tool-a note rating instrument across four domains (I=Interpretive summary, D=Differential diagnosis, E=Explanation of reasoning, A=Alternative diagnoses explained) that uses a 3-point Likert scale without descriptive anchors. To develop descriptive anchors we conducted an iterative process reviewing notes from the EHR written by medicine residents and validated the Revised-IDEA Assessment Tool using Messick's framework- content validity, response process, relation to other variables, internal structure, and consequences. Using the Hofstee standard setting method, cutoffs for high quality clinical reasoning for the IDEA and DEA scores were set. We then created a dataset of expertrated notes to create the ML algorithm. First, a natural language processing software was applied to the set of notes that enabled recognition and automatic encoding of clinical information as a diagnosis or disease (D's), a sign or symptom (E or A), or semantic qualifier (e.g. most likely). Input variables to the ML algorithm included counts of D's, E/A's, semantic qualifiers, and proximity of semantic qualifiers to disease/ diagnosis. ML output focused on DEA quality and was binarized to low or high quality CR. Finally, 200 notes were randomly selected for human validation review comparing ML output to human rated DEA score.
RESULT(S): The IDEA and DEA scores ranged from 0-10 and 0-6, respectively. IDEA score of >= 6.5 and a DEA score of >= 3 was deemed high quality. 252 notes were rated to create the dataset and 20% were rated by 3 raters with high intraclass correlation 0.84 (95% CI 0.74-0.90). 120 of these notes comprised the testing set for ML model development. The logistic regression model was the best performing model with an AUC 0.87 and a positive predictive value (PPV) of 0.65. 48 (40%) of the notes were high quality. There was substantial interrater reliability between ML output and human rating on the 200 note validation set with a Cohen's Kappa 0.64.
CONCLUSION(S): We have developed a ML algorithm for feedback on CR documentation that we hypothesize will increase the frequency and quality of feedback in this domain. We have subsequently developed a dashboard that will display the output of the ML model. Next steps will be to provide internal medicine residents' feedback on their CR documentation using this dashboard and assess the impact this has on their documentation quality. LEARNING OBJECTIVE #1: Describe the importance of high quality documentation of clinical reasoning. LEARNING OBJECTIVE #2: Identify machine learning as a novel assessment tool for feedback on clinical reasoning documentation

BACKGROUND:There is growing evidence that the accumulation of protein- bound uremic retention solutes, such as indoxyl sulfate, p-cresyl sulfate and kynurenic acid, play a role in the accelerated cardiovascular disease seen in patients undergoing chronic hemodialysis. Protein-binding, presumably to albumin, renders these solutes poor-dialyzable. We previously observed that the free fraction of indoxyl sulfate was markedly reduced at the end of hemodialysis. We hypothesized that solute binding might be pH-dependent and attributed the changes in free solute concentration to the higher serum pH observed at the end of standard hemodialysis with dialysis buffer bicarbonate concentration greater than 35 mmol/L. We observed that acidification of uremic plasma to pH 6 in vitro greatly increased the proportion of freeIS. METHODS:We tested our hypothesis by reducing the dialysate bicarbonate buffer concentration to 25 mmol/L for the initial half of the hemodialysis treatment ("isohydric dialysis"). Eight stable hemodialysis patients underwent "isohydric dialysis" for 90 minutes and then were switched to standard buffer (bicarbonate = 37mmol/L). A second dialysis, 2 days later, employed standard buffer throughout. RESULTS:We found a clearcut separation of blood pH and bicarbonate concentrations after 90 minutes of "isohydric dialysis" (pH = 7.37, bicarbonate = 22.4 mmol/L) and standard dialysis (pH = 7.49, bicarbonate = 29.0 mmol/L). Binding affinity varied widely among the 10 uremic retention solutes analyzed. Kynurenic acid (0.05 free), p-cresyl sulfate (0.12 free) and indoxyl sulfate (0.13 free) demonstrated the greatest degree of binding. Three solutes (indoxyl glucuronide, p-cresyl glucuronide, and phenyl glucuronide) were virtually unbound. Analysis of free and bound concentrations of uremic retention solutes confirmed our prediction that binding of solute is affected by pH. However, in a mixed models analysis, we found that the reduction in total uremic solute concentration during dialysis accounted for a greater proportion of the variation in free concentration, presumably an effect of saturation binding to albumin, than did the relatively small change in pH produced by isohydric dialysis. The effect of pH on binding appeared to be restricted to those solutes most highly protein-bound. The solutes most tightly bound exhibited the lowest dialyzer clearances. An increase in dialyzer clearance during isohydric and standard dialyses was statistically significant only for kynurenic acid. CONCLUSION/CONCLUSIONS:These findings provide evidence that the binding of uremic retention solutes is influenced by pH. The effect of reducing buffer bicarbonate concentration ("isohydric dialysis:"), though significant, was small but may be taken to suggest that further modification of dialysis technique that would expose blood to a greater decrease in pH would lead to a greater increase the free fraction of solute and enhance the efficacy of hemodialysis in the removal of highly protein-bound uremic retention solutes.

[This corrects the article DOI: 10.1371/journal.pone.0192770.].

Diospyrobezoars are a subtype of phytobezoars caused by excessive consumption of persimmons, which contain large amounts of tannins. In contrast to phytobezoars, diospyrobezoars have a harder consistency than other bezoars, making them more difficult to break up both chemically and endoscopically. We have previously reported successful dissolution of phytobezoars with diet soda and cellulase. A review of the literature found low efficacy of soda in dissolving diospyrobezoars compared to other phytobezoars. We report a case of successful dissolution of a diospyrobezoar after a failed attempt with diet soda alone.

Object In patients with tuberous sclerosis complex (TSC), the tuber-to-brain proportion (TBP) is a marker of seizure severity and cognitive function. However, few studies have quantified the TBP. Furthermore, authors of these studies have measured the TBP at only a single time point, despite the fact that tuber cells were found to express proliferation markers, suggesting that they may be dynamic lesions. Authors of the present study used a semi-automated tuber segmentation program to determine whether the TBP changes over time. Methods Axial FLAIR MR images were retrospectively identified for patients with TSC who had undergone imaging at the authors' institution between February 1998 and June 2009. Using FireVoxel software, the TBP was measured for each patient at a minimum interval of 2 years. Results Twelve patients meeting the study inclusion criteria were identified. The mean TBP was 1.88% (range 0.38%-3.70%). Eight patients demonstrated minimal changes and 3 patients demonstrated small increases in TBP. The remaining patient exhibited a decrease of 1.00%, which correlated with a visible decrease in the size of 2 cerebellar lesions. Conclusions Semi-automated brain segmentation is a valuable tool in the longitudinal study of tubers. A subset of patients with TSC, particularly those with cerebellar lesions, may exhibit changes in the TBP over time.