Faculty Bibliography

Solutions are employed to support clinical research trial tasks in community-based practice settings. Using the IT Implementation Framework (ITIF), an integrative framework intended to guide the synthesis of theoretical perspectives for planning multi-level interventions to enhance IT use, we sought to understand the barriers and facilitators to clinical research in community-based practice settings preliminary to implementing new informatics solutions for improving clinical research infrastructure. The studies were conducted in practices within the Columbia University Clinical Trials Network. A mixed-method approach, including surveys, interviews, time-motion studies, and observations was used. The data collected, which incorporates predisposing, enabling, and reinforcing factors in IT use, were analyzed according to each phase of ITIF. Themes identified in the first phase of ITIF were 1) processes and tools to support clinical trial research and 2) clinical research peripheral to patient care processes. Not all of the problems under these themes were found to be amenable to IT solutions. Using the multi-level orientation of the ITIF, we set forth strategies beyond IT solutions that can have an impact on reengineering clinical research tasks in practice-based settings. Developing strategies to target enabling and reinforcing factors, which focus on organizational factors, and the motivation of the practice at large to use IT solutions to integrate clinical research tasks with patient care processes, is most challenging. The ITIF should be used to consider both IT and non-IT solutions concurrently for reengineering of clinical research in community-based practice settings.

Temporal constraints are present in 38% of clinical research eligibility criteria and are crucial for screening patients. However, eligibility criteria are often written as free text, which is not amenable for computer processing. In this paper, we present an ontology-based approach to extracting temporal information from clinical research eligibility criteria. We generated temporal labels using a frame-based temporal ontology. We manually annotated 150 free-text eligibility criteria using the temporal labels and trained a parser using Conditional Random Fields (CRFs) to automatically extract temporal expressions from eligibility criteria. An evaluation of an additional 60 randomly selected eligibility criteria using manual review achieved an overall precision of 83%, a recall of 79%, and an F-score of 80%. We illustrate the application of temporal extraction with the use cases of question answering and free-text criteria querying.

CONTEXT/BACKGROUND:Public health services and systems research (PHSSR) focuses on the structure, organization, and legal basis of domestic public health activities and their effect on population health. An accurate description of the field is needed to empower funding agencies and other stakeholders to coordinate PHSSR activities and to foster the development of the field. The purpose of the study is to characterize the emerging community of researchers engaged in PHSSR. This study (1) describes dynamics of this growing community and (2) identifies research themes, subgroups within the field, and collaboration among groups. EVIDENCE ACQUISITION/METHODS:Coauthorship network visualization of selected research publications in the MEDLINE bibliographic database between 1988 and May 2010. EVIDENCE SYNTHESIS/RESULTS:PHSSR has emerged gradually with noticeable growth after 1994 and after 2004. The network of PHSSR research has a core-periphery structure. The core includes highly collaborative researchers focusing on topics pertaining directly to PHSSR, such as workforce, quality improvement and performance, law, and information infrastructure. The periphery consists of groups publishing either on general health services research topics or on epidemiologic and clinical topics. CONCLUSIONS:Although a nucleus group of productive and engaged individuals participate in PHSSR, most also publish broadly on health services research and population health. This trend suggests that this emerging field cannot yet support a singular focus on PHSSR. Lack of funding sources and defined career paths likely contribute to this pattern. An overview of collaboration in PHSSR is an important step in advancing a coordinated research agenda and attracting sustainable funding streams for this field.

OBJECTIVE:To develop a semantic representation for clinical research eligibility criteria to automate semistructured information extraction from eligibility criteria text. MATERIALS AND METHODS/METHODS:An analysis pipeline called eligibility criteria extraction and representation (EliXR) was developed that integrates syntactic parsing and tree pattern mining to discover common semantic patterns in 1000 eligibility criteria randomly selected from http://ClinicalTrials.gov. The semantic patterns were aggregated and enriched with unified medical language systems semantic knowledge to form a semantic representation for clinical research eligibility criteria. RESULTS:The authors arrived at 175 semantic patterns, which form 12 semantic role labels connected by their frequent semantic relations in a semantic network. EVALUATION/RESULTS:Three raters independently annotated all the sentence segments (N=396) for 79 test eligibility criteria using the 12 top-level semantic role labels. Eight-six per cent (339) of the sentence segments were unanimously labelled correctly and 13.8% (55) were correctly labelled by two raters. The Fleiss' κ was 0.88, indicating a nearly perfect interrater agreement. CONCLUSION/CONCLUSIONS:This study present a semi-automated data-driven approach to developing a semantic network that aligns well with the top-level information structure in clinical research eligibility criteria text and demonstrates the feasibility of using the resulting semantic role labels to generate semistructured eligibility criteria with nearly perfect interrater reliability.

This paper describes a usability evaluation study of an innovative first generation system (Data Dig) designed to retrieve phenotypic data from the large SFARI data set of 2700 families each of which has one child affected with autism spectrum disorder. The usability methods included a cognitive walkthrough and usability testing. Although the subjects were able to learn to use the system, more than 50 usability problems of varying severity were noted. The problems with the greatest frequency resulted from users being unable to understand meanings of variables, filter categories correctly, use the Boolean filter, and correctly interpret the feedback provided by the system. Subjects had difficulty forming a mental model of the organizational system underlying the database. This precluded them from making informed navigation choices while formulating queries. Clinical research informatics is a new and immensely promising discipline. However in its nascent stage, it lacks a stable interaction paradigm to support a range of users on pertinent tasks. This presents great opportunity for researchers to further this science by harnessing the powers of user-centered iterative design.

OBJECTIVE: Rigorous human-computer interaction (HCI) design methodologies have not traditionally been applied to the development of clinical trial participant tracking (CTPT) tools. Given the frequent us of iconic HCI models in CTPTs, and prior evidence of usability problems associated with the use of ambiguous icons in complex interfaces, such approaches may be problematic. Presentation Discovery (PD), a knowledge-anchored HCI design method, has been previously demonstrated to improve the design of iconic HCI models. In this study, we compare the usability of a CTPT HCI model designed using PD and an intuitively designed CTPT HCI model. METHODS: An iconic CPTP HCI model was created using PD. The PD-generated and an existing iconic CTPT HCI model were subjected to usability testing, with an emphasis on task accuracy and completion times. Study participants also completed a qualitative survey instrument to evaluate subjective satisfaction with the two models. RESULTS: CTPT end-users reliably and reproducibly agreed on the visual manifestation and semantics of prototype graphics generated using PD. The performance of the PD-generated iconic HCI model was equivalent to an existing HCI model for tasks at multiple levels of complexity, and in some cases superior. This difference was particularly notable when tasks required an understanding of the semantic meanings of multiple icons. CONCLUSION: The use of PD to design an iconic CTPT HCI model generated beneficial results and improved end-user subjective satisfaction, while reducing task completion time. Such results are desirable in information and time intensive domains, such as clinical trials management.

This paper presents a novel approach to learning semantic classes of clinical research eligibility criteria. It uses the UMLS Semantic Types to represent semantic features and the Hierarchical Clustering method to group similar eligibility criteria. By establishing a gold standard using two independent raters, we evaluated the coverage and accuracy of the induced semantic classes. On 2,718 random eligibility criteria sentences, the inter-rater classification agreement was 85.73%. In a 10-fold validation test, the average Precision, Recall and F-score of the classification results of a decision-tree classifier were 87.8%, 88.0%, and 87.7% respectively. Our induced classes well aligned with 16 out of 17 eligibility criteria classes defined by the BRIDGE model. We discuss the potential of this method and our future work.

OBJECTIVE:To propose a centralized method for generating global unique identifiers to link collections of research data and specimens. DESIGN/METHODS:The work is a collaboration between the Simons Foundation Autism Research Initiative and the National Database for Autism Research. The system is implemented as a web service: an investigator inputs identifying information about a participant into a client application and sends encrypted information to a server application, which returns a generated global unique identifier. The authors evaluated the system using a volume test of one million simulated individuals and a field test on 2000 families (over 8000 individual participants) in an autism study. MEASUREMENTS/METHODS:Inverse probability of hash codes; rate of false identity of two individuals; rate of false split of single individual; percentage of subjects for which identifying information could be collected; percentage of hash codes generated successfully. RESULTS:Large-volume simulation generated no false splits or false identity. Field testing in the Simons Foundation Autism Research Initiative Simplex Collection produced identifiers for 96% of children in the study and 77% of parents. On average, four out of five hash codes per subject were generated perfectly (only one perfect hash is required for subsequent matching). DISCUSSION/CONCLUSIONS:The system must achieve balance among the competing goals of distinguishing individuals, collecting accurate information for matching, and protecting confidentiality. Considerable effort is required to obtain approval from institutional review boards, obtain consent from participants, and to achieve compliance from sites during a multicenter study. CONCLUSION/CONCLUSIONS:Generic unique identifiers have the potential to link collections of research data, augment the amount and types of data available for individuals, support detection of overlap between collections, and facilitate replication of research findings.

Searches of bibliographic databases generate lists of articles but do little to reveal connections between authors, institutions, and grants. As a result, search results cannot be fully leveraged. To address this problem we developed Sciologer, a prototype search and visualization system. Sciologer presents the results of any PubMed query as an interactive network diagram of the above elements. We conducted a cognitive evaluation with six neuroscience and six obesity researchers. Researchers used the system effectively. They used geographic, color, and shape metaphors to describe community structure and made accurate inferences pertaining to a) collaboration among research groups; b) prominence of individual researchers; and c) differentiation of expertise. The tool confirmed certain beliefs, disconfirmed others, and extended their understanding of their own discipline. The majority indicated the system offered information of value beyond a traditional PubMed search and that they would use the tool if available.

Providing knowledge at the point of care offers the possibility for reducing error and improving patient outcomes. However, the vast majority of the physician's information needs are not met in a timely fashion. The research presented in this paper characterizes an expert librarian's search strategies as it pertains to the selection and use of various electronic information resources. The 10 searches conducted by the librarian to address the physician's information needs varied in terms of complexity and question type. The librarian employed a total of 10 resources and used as many as 7 in a single search. The longer term objective is to model the sequential process in sufficient detail as to be able to contribute to the development of intelligent automated search agents.