Faculty Bibliography

A principle concern of pharmacovigilance is the timely detection of adverse drug reactions that are novel by virtue of their clinical nature, severity and/or frequency. The cornerstone of this process is the scientific acumen of the pharmacovigilance domain expert. There is understandably an interest in developing database screening tools to assist human reviewers in identifying associations worthy of further investigation (i.e., signals) embedded within a database consisting largely of background 'noise' containing reports of no substantial public health significance. Data mining algorithms are, therefore, being developed, tested and/or used by health authorities, pharmaceutical companies and academic researchers. After a focused review of postapproval drug safety signal detection, the authors explain how the currently used algorithms work and address key questions related to their validation, comparative performance, deployment in naturalistic pharmacovigilance settings, limitations and potential for misuse. Suggestions for further research and development are offered.

The objective of this study was to apply 2 data-mining algorithms to a drug safety database to determine if these methods would have flagged potentially fatal/disabling adverse drug reactions that triggered black box warnings/drug withdrawals in advance of initial identification via "traditional" methods. Relevant drug-event combinations were identified from a journal publication. Data-mining algorithms using commonly cited disproportionality thresholds were then applied to the US Food and Drug Administration database. Seventy drug-event combinations were considered sufficiently specific for retrospective data mining. In a minority of instances, potential signals of disproportionate reporting were provided clearly in advance of initial identification via traditional pharmacovigilance methods. Data-mining algorithms have the potential to improve pharmacovigilance screening; however, for the majority of drug-event combinations, there was no substantial benefit of either over traditional methods. They should be considered as potential supplements to, and not substitutes for, traditional pharmacovigilance strategies. More research and experience will be needed to optimize deployment of data-mining algorithms in pharmacovigilance.

OBJECTIVE: To apply two data mining algorithms (DMAs) to Food and Drug Administration (FDA) Adverse Event Reporting System (AERS) reports that involved endotoxin-like reactions with intravenous gentamicin to determine whether a signal of disproportionate reporting of these events would have been generated concurrently with surveillance based on clinical observation. DESIGN: Multi-item gamma-Poisson shrinker (MGPS) and proportional reporting ratios (PRRs) were used. Data used for data mining consisted of an extract of the FDA AERS database. Previously published details of clusters of endotoxin-like reactions to intravenous gentamicin were used to select adverse events for data mining. RESULTS: The first signal of disproportionate reporting with any relevant event occurred in 1998, the year in which the outbreak was identified and evaluated by the Centers for Disease Control and Prevention and the FDA. In 1997, there were only 6 reports of rigors in the AERS; this jumped to 68 in 1998. In 1998, a signal was generated for endotoxic shock with PRRs but not with MGPS, based on one case. CONCLUSIONS: The two DMAs generated signals concurrently with the influx of reports. It would have been difficult for safety reviewers to ignore an increase in rigors by traditional methods of safety surveillance; therefore, DMAs might not have had a great deal to offer in this instance. If data mining were considered as a second-line defense to diligent clinical observations under similar circumstances, simple disproportionality methods such as PRRs might be more useful than DMAs such as MGPS when commonly cited thresholds are used.

Data mining is receiving considerable attention as a tool for pharmacovigilance and is generating many perspectives on its uses. This paper presents four concepts that have appeared in various professional venues and represent potential sources of misunderstanding and/or entail extended discussions: (i) data mining algorithms are unvalidated; (ii) data mining algorithms allow data miners to objectively screen spontaneous report data; (iii) mathematically more complex Bayesian algorithms are superior to frequentist algorithms; and (iv) data mining algorithms are not just for hypothesis generation. Key points for a balanced perspective are that: (i) validation exercises have been done but lack a gold standard for comparison and are complicated by numerous nuances and pitfalls in the deployment of data mining algorithms. Their performance is likely to be highly situation dependent; (ii) the subjective nature of data mining is often underappreciated; (iii) simpler data mining models can be supplemented with 'clinical shrinkage', preserving sensitivity; and (iv) applications of data mining beyond hypothesis generation are risky, given the limitations of the data. These extended applications tend to 'creep', not pounce, into the public domain, leading to potential overconfidence in their results. Most importantly, in the enthusiasm generated by the promise of data mining tools, users must keep in mind the limitations of the data and the importance of clinical judgment and context, regardless of statistical arithmetic. In conclusion, we agree that contemporary data mining algorithms are promising additions to the pharmacovigilance toolkit, but the level of verification required should be commensurate with the nature and extent of the claimed applications.