Faculty Bibliography

OBJECTIVES/OBJECTIVE:To develop a method for investigating co-authorship patterns and author team characteristics associated with the publications in high-impact journals through the integration of public MEDLINE data and institutional scientific profile data. METHODS:For all current researchers at Columbia University Medical Center, we extracted their publications from MEDLINE authored between years 2007 and 2011 and associated journal impact factors, along with author academic ranks and departmental affiliations obtained from Columbia University Scientific Profiles (CUSP). Chi-square tests were performed on co-authorship patterns, with Bonferroni correction for multiple comparisons, to identify team composition characteristics associated with publication impact factors. We also developed co-authorship networks for the 25 most prolific departments between years 2002 and 2011 and counted the internal and external authors, inter-connectivity, and centrality of each department. RESULTS:Papers with at least one author from a basic science department are significantly more likely to appear in high-impact journals than papers authored by those from clinical departments alone. Inclusion of at least one professor on the author list is strongly associated with publication in high-impact journals, as is inclusion of at least one research scientist. Departmental and disciplinary differences in the ratios of within- to outside-department collaboration and overall network cohesion are also observed. CONCLUSIONS:Enrichment of co-authorship patterns with author scientific profiles helps uncover associations between author team characteristics and appearance in high-impact journals. These results may offer implications for mentoring junior biomedical researchers to publish on high-impact journals, as well as for evaluating academic progress across disciplines in modern academic medical centers.

OBJECTIVE:Publications are a key data source for investigator profiles and research networking systems. We developed ReCiter, an algorithm that automatically extracts bibliographies from PubMed using institutional information about the target investigators. METHODS:ReCiter executes a broad query against PubMed, groups the results into clusters that appear to constitute distinct author identities and selects the cluster that best matches the target investigator. Using information about investigators from one of our institutions, we compared ReCiter results to queries based on author name and institution and to citations extracted manually from the Scopus database. Five judges created a gold standard using citations of a random sample of 200 investigators. RESULTS:About half of the 10,471 potential investigators had no matching citations in PubMed, and about 45% had fewer than 70 citations. Interrater agreement (Fleiss' kappa) for the gold standard was 0.81. Scopus achieved the best recall (sensitivity) of 0.81, while name-based queries had 0.78 and ReCiter had 0.69. ReCiter attained the best precision (positive predictive value) of 0.93 while Scopus had 0.85 and name-based queries had 0.31. DISCUSSION/CONCLUSIONS:ReCiter accesses the most current citation data, uses limited computational resources and minimizes manual entry by investigators. Generation of bibliographies using named-based queries will not yield high accuracy. Proprietary databases can perform well but requite manual effort. Automated generation with higher recall is possible but requires additional knowledge about investigators.

OBJECTIVE:To summarize literature describing approaches aimed at automatically identifying patients with a common phenotype. MATERIALS AND METHODS/METHODS:We performed a review of studies describing systems or reporting techniques developed for identifying cohorts of patients with specific phenotypes. Every full text article published in (1) Journal of American Medical Informatics Association, (2) Journal of Biomedical Informatics, (3) Proceedings of the Annual American Medical Informatics Association Symposium, and (4) Proceedings of Clinical Research Informatics Conference within the past 3 years was assessed for inclusion in the review. Only articles using automated techniques were included. RESULTS:Ninety-seven articles met our inclusion criteria. Forty-six used natural language processing (NLP)-based techniques, 24 described rule-based systems, 41 used statistical analyses, data mining, or machine learning techniques, while 22 described hybrid systems. Nine articles described the architecture of large-scale systems developed for determining cohort eligibility of patients. DISCUSSION/CONCLUSIONS:We observe that there is a rise in the number of studies associated with cohort identification using electronic medical records. Statistical analyses or machine learning, followed by NLP techniques, are gaining popularity over the years in comparison with rule-based systems. CONCLUSIONS:There are a variety of approaches for classifying patients into a particular phenotype. Different techniques and data sources are used, and good performance is reported on datasets at respective institutions. However, no system makes comprehensive use of electronic medical records addressing all of their known weaknesses.

BACKGROUND:The United States Food and Drug Administration (FDA) has proposed creating a unique device identification (UDI) system for medical devices to facilitate postmarket surveillance, quality improvement, and other applications. Although a small number of health care institutions have implemented initiatives comparable with the proposed UDI system by capturing data in electronic health record (EHR) systems, it is unknown whether institutions with fewer resources will be able to similarly implement UDI. OBJECTIVE AND METHODS/OBJECTIVE:This paper calls attention to organizational, workflow, and technological challenges in UDI system implementation by drawing from the literature on EHR and clinical research systems implementation. FINDINGS/RESULTS:Organizational challenges for UDI system implementation include coordinating multiple stakeholders to define UDI attributes and characteristics for use in EHRs, guiding organizational change within individual institutions for integrating UDI with EHRs, and guiding organizational change for reusing UDI data captured in EHRs. Workflow challenges include capturing UDI data in EHRs using keyboard entry and barcode scanning. Technological challenges involve interfacing UDI data between EHRs and surgical information systems, transforming UDI and related patient data from EHRs for research, and applying data standards to UDI within and beyond EHRs. DISCUSSION AND CONCLUSIONS/CONCLUSIONS:We provide recommendations for regulations, organizational sharing, and professional society engagement to raise awareness of and overcome UDI system implementation challenges. Implementation of the UDI system will require integration of people, process, and technology to achieve benefits envisioned by FDA, including improved postmarket device surveillance and quality of care.

The successful adoption by clinicians of evidence-based clinical practice guidelines (CPGs) contained in clinical information systems requires efficient translation of free-text guidelines into computable formats. Natural language processing (NLP) has the potential to improve the efficiency of such translation. However, it is laborious to develop NLP to structure free-text CPGs using existing formal knowledge representations (KR). In response to this challenge, this vision paper discusses the value and feasibility of supporting symbiosis in text-based knowledge acquisition (KA) and KR. We compare two ontologies: (1) an ontology manually created by domain experts for CPG eligibility criteria and (2) an upper-level ontology derived from a semantic pattern-based approach for automatic KA from CPG eligibility criteria text. Then we discuss the strengths and limitations of interweaving KA and NLP for KR purposes and important considerations for achieving the symbiosis of KR and NLP for structuring CPGs to achieve evidence-based clinical practice.

OBJECTIVES/OBJECTIVE:Public and private organizations are implementing systems for query-based health information exchange (HIE), the electronic aggregation of patient data from multiple institutions. However, existing studies of query-based HIE system usage have addressed a limited number of settings. Our goal was to quantify the breadth and depth of usage of a query-based HIE system implemented across multiple communities with diverse care settings and patient populations. METHODS:We performed a cross-sectional study in three communities in New York State using system access log files from January 2009 to May 2011 to measure usage patterns of a query-based HIE web portal system with respect to practice sites, users, patients, and data. RESULTS:System access occurred from 60% (n=200) of practice sites registered to use the system in Community A, 59% (n=156) in Community B, and 82% (n=28) in Community C. In Communities A and B, users were primarily non-clinical staff in outpatient settings, while in Community C inpatient physicians were the main users. Across communities, proportions of patients whose data were accessed varied with 5% (n=11,263) in Community A, 60% (n=212,586) in Community B, and 1% (n=1107) in Community C. In Community B, users updated patient consent through the HIE portal, whereas in the other communities, users updated patient consent through a separate system. Across communities, users most frequently accessed only patient summary data displayed by default followed by detailed laboratory and radiology data. CONCLUSIONS:This study is among the first to illustrate large-scale usage of a query-based HIE system implemented across multiple communities. Patient summary data displayed by default may be an important feature of query-based HIE systems. User role, practice site type, and patient consent workflow may affect patterns of query-based HIE web portal system usage in the communities studied and elsewhere.

Solutions are employed to support clinical research trial tasks in community-based practice settings. Using the IT Implementation Framework (ITIF), an integrative framework intended to guide the synthesis of theoretical perspectives for planning multi-level interventions to enhance IT use, we sought to understand the barriers and facilitators to clinical research in community-based practice settings preliminary to implementing new informatics solutions for improving clinical research infrastructure. The studies were conducted in practices within the Columbia University Clinical Trials Network. A mixed-method approach, including surveys, interviews, time-motion studies, and observations was used. The data collected, which incorporates predisposing, enabling, and reinforcing factors in IT use, were analyzed according to each phase of ITIF. Themes identified in the first phase of ITIF were 1) processes and tools to support clinical trial research and 2) clinical research peripheral to patient care processes. Not all of the problems under these themes were found to be amenable to IT solutions. Using the multi-level orientation of the ITIF, we set forth strategies beyond IT solutions that can have an impact on reengineering clinical research tasks in practice-based settings. Developing strategies to target enabling and reinforcing factors, which focus on organizational factors, and the motivation of the practice at large to use IT solutions to integrate clinical research tasks with patient care processes, is most challenging. The ITIF should be used to consider both IT and non-IT solutions concurrently for reengineering of clinical research in community-based practice settings.

OBJECTIVE:To develop a semantic representation for clinical research eligibility criteria to automate semistructured information extraction from eligibility criteria text. MATERIALS AND METHODS/METHODS:An analysis pipeline called eligibility criteria extraction and representation (EliXR) was developed that integrates syntactic parsing and tree pattern mining to discover common semantic patterns in 1000 eligibility criteria randomly selected from http://ClinicalTrials.gov. The semantic patterns were aggregated and enriched with unified medical language systems semantic knowledge to form a semantic representation for clinical research eligibility criteria. RESULTS:The authors arrived at 175 semantic patterns, which form 12 semantic role labels connected by their frequent semantic relations in a semantic network. EVALUATION/RESULTS:Three raters independently annotated all the sentence segments (N=396) for 79 test eligibility criteria using the 12 top-level semantic role labels. Eight-six per cent (339) of the sentence segments were unanimously labelled correctly and 13.8% (55) were correctly labelled by two raters. The Fleiss' κ was 0.88, indicating a nearly perfect interrater agreement. CONCLUSION/CONCLUSIONS:This study present a semi-automated data-driven approach to developing a semantic network that aligns well with the top-level information structure in clinical research eligibility criteria text and demonstrates the feasibility of using the resulting semantic role labels to generate semistructured eligibility criteria with nearly perfect interrater reliability.

CONTEXT/BACKGROUND:Public health services and systems research (PHSSR) focuses on the structure, organization, and legal basis of domestic public health activities and their effect on population health. An accurate description of the field is needed to empower funding agencies and other stakeholders to coordinate PHSSR activities and to foster the development of the field. The purpose of the study is to characterize the emerging community of researchers engaged in PHSSR. This study (1) describes dynamics of this growing community and (2) identifies research themes, subgroups within the field, and collaboration among groups. EVIDENCE ACQUISITION/METHODS:Coauthorship network visualization of selected research publications in the MEDLINE bibliographic database between 1988 and May 2010. EVIDENCE SYNTHESIS/RESULTS:PHSSR has emerged gradually with noticeable growth after 1994 and after 2004. The network of PHSSR research has a core-periphery structure. The core includes highly collaborative researchers focusing on topics pertaining directly to PHSSR, such as workforce, quality improvement and performance, law, and information infrastructure. The periphery consists of groups publishing either on general health services research topics or on epidemiologic and clinical topics. CONCLUSIONS:Although a nucleus group of productive and engaged individuals participate in PHSSR, most also publish broadly on health services research and population health. This trend suggests that this emerging field cannot yet support a singular focus on PHSSR. Lack of funding sources and defined career paths likely contribute to this pattern. An overview of collaboration in PHSSR is an important step in advancing a coordinated research agenda and attracting sustainable funding streams for this field.

This paper describes a usability evaluation study of an innovative first generation system (Data Dig) designed to retrieve phenotypic data from the large SFARI data set of 2700 families each of which has one child affected with autism spectrum disorder. The usability methods included a cognitive walkthrough and usability testing. Although the subjects were able to learn to use the system, more than 50 usability problems of varying severity were noted. The problems with the greatest frequency resulted from users being unable to understand meanings of variables, filter categories correctly, use the Boolean filter, and correctly interpret the feedback provided by the system. Subjects had difficulty forming a mental model of the organizational system underlying the database. This precluded them from making informed navigation choices while formulating queries. Clinical research informatics is a new and immensely promising discipline. However in its nascent stage, it lacks a stable interaction paradigm to support a range of users on pertinent tasks. This presents great opportunity for researchers to further this science by harnessing the powers of user-centered iterative design.