Faculty Bibliography

Having surveyed the etymology and previous definitions of the pharmacovigilance term 'signal', we propose a definition that embraces all the surveyed ideas, reflects real-world pharmacovigilance processes, and accommodates signals of both harmful and beneficial effects. The essential definitional features of a pharmacovigilance signal are (i) that it is based on one or more reports of an association between an intervention or interventions and an event or set of related events (e.g. a syndrome), including any type of evidence (clinical or experimental); (ii) that it represents an association that is new and important and has not been previously investigated and refuted; (iii) that it incites to action (verification and remedial action); (iv) that it does not encompass intervention-event associations that are not related to causality or risk with a specified degree of likelihood and scientific plausibility. Based on these features, we propose this definition of a signal of suspected causality: 'information that arises from one or multiple sources (including observations and experiments), which suggests a new potentially causal association, or a new aspect of a known association, between an intervention and an event or set of related events, either adverse or beneficial, which would command regulatory, societal or clinical attention, and is judged to be of sufficient likelihood to justify verificatory and, when necessary, remedial actions.' This defines an unverified signal; we have also defined terms - indeterminate, verified, and refuted signals - that qualify it in relation to verification. This definition and its accompanying flowchart should inform decision making in considering benefits and harms caused by pharmacological and nonpharmacological interventions

BACKGROUND: Pharmacovigilance data-mining algorithms (DMAs) are known to generate significant numbers of false-positive signals of disproportionate reporting (SDRs), using various standards to define the terms 'true positive' and 'false positive'. OBJECTIVE: To construct a highly inclusive reference event database of reported adverse events for a limited set of drugs, and to utilize that database to evaluate three DMAs for their overall yield of scientifically supported adverse drug effects, with an emphasis on ascertaining false-positive rates as defined by matching to the database, and to assess the overlap among SDRs detected by various DMAs. METHODS: A sample of 35 drugs approved by the US FDA between 2000 and 2004 was selected, including three drugs added to cover therapeutic categories not included in the original sample. We compiled a reference event database of adverse event information for these drugs from historical and current US prescribing information, from peer-reviewed literature covering 1999 through March 2006, from regulatory actions announced by the FDA and from adverse event listings in the British National Formulary. Every adverse event mentioned in these sources was entered into the database, even those with minimal evidence for causality. To provide some selectivity regarding causality, each entry was assigned a level of evidence based on the source of the information, using rules developed by the authors. Using the FDA adverse event reporting system data for 2002 through 2005, SDRs were identified for each drug using three DMAs: an urn-model based algorithm, the Gamma Poisson Shrinker (GPS) and proportional reporting ratio (PRR), using previously published signalling thresholds. The absolute number and fraction of SDRs matching the reference event database at each level of evidence was determined for each report source and the data-mining method. Overlap of the SDR lists among the various methods and report sources was tabulated as well. RESULTS: The GPS algorithm had the lowest overall yield of SDRs (763), with the highest fraction of events matching the reference event database (89 SDRs, 11.7%), excluding events described in the prescribing information at the time of drug approval. The urn model yielded more SDRs (1562), with a non-significantly lower fraction matching (175 SDRs, 11.2%). PRR detected still more SDRs (3616), but with a lower fraction matching (296 SDRs, 8.2%). In terms of overlap of SDRs among algorithms, PRR uniquely detected the highest number of SDRs (2231, with 144, or 6.5%, matching), followed by the urn model (212, with 26, or 12.3%, matching) and then GPS (0 SDRs uniquely detected). CONCLUSIONS: The three DMAs studied offer significantly different tradeoffs between the number of SDRs detected and the degree to which those SDRs are supported by external evidence. Those differences may reflect choices of detection thresholds as well as features of the algorithms themselves. For all three algorithms, there is a substantial fraction of SDRs for which no external supporting evidence can be found, even when a highly inclusive search for such evidence is conducted

Data mining algorithms are increasingly being used to support the process of signal detection and evaluation in pharmacovigilance. Published data mining exercises formulated within a screening paradigm typically calculate classical performance indicators such as sensitivity, specificity, predictive value and receiver operator characteristic curves. Extrapolating signal detection performance from these isolated data mining exercises to performance in real-world pharmacovigilance scenarios is complicated by numerous factors and some published exercises may promote an inappropriate and exclusive focus on only one aspect of performance. In this article, we discuss a variation on positive predictive value that we call the 'number needed to detect' that provides a simple and intuitive screening metric that might usefully supplement the usual presentations of data mining performance. We use a series of figures to demonstrate the nature and application of this metric, and selected adaptive variations. Even with simple and intuitive metrics, precisely quantifying the performance of contemporary data mining algorithms in pharmacovigilance is complicated by the complexity of the phenomena under surveillance and the manner in which the data are recorded in spontaneous reporting systems

The US Food and Drug Administration (FDA) recently published a warning regarding pancreatitis in association with the use of exenatide, an incretin mimetic used for the treatment of patients with diabetes mellitus. We note that this safety issue is not associated with a signal of disproportionate reporting (SDR) in the FDA Adverse Event Reporting System (AERS) database or the World Health Organization (Uppsala Monitoring Centre) Vigibase for any of four data-mining algorithms we tested (proportional reporting ratio, the multi-item gamma-Poisson shrinker, an urn model and the Bayesian Confidence Propagation Neural Network). Exenatide and acute pancreatitis may thus represent a 'false-negative' result for disproportionality-based data-mining methodology generally. We evaluate the possibility that this lack of an SDR is caused by the phenomenon known as 'masking' (or 'cloaking') and reject this hypothesis. While positive findings are understandably more exciting, we discuss why publishing negative findings, such as in this example, is important for placing the capabilities and limitations of drug safety data mining into proper perspective

OBJECTIVE: To compare reporting frequencies of hepatic adverse events between PEGylated and non-PEGylated formulations of active medicinal compounds in spontaneous reporting systems using a data mining algorithm (DMA). METHODS: Statistical DMAs are being promoted as a means of identifying drug-event combinations that are disproportionately reported in large spontaneous reporting systems databases, a critical data source for pharmacovigilance. After a review of case reports of hepatotoxicity with PEGylated drugs possibly associated with the polyethylene glycol moiety, we carried out a retrospective disproportionality analysis of WHO's multinational drug safety database for events related to hepatic dysfunction comparing PEGylated versus non-PEGylated formulations of four active moieties. A threshold of posterior interval (PI) 95% lower limit >0 was used to define a signal of disproportionate reporting with a drug and an event and 90% PIs of the information component were compared to identify statistical differences between the two compounds. RESULTS: On the basis of a total of 18 477 cases containing at least one of the drug pairs, we found disproportionate reporting for hepatic-related events with both PEGylated and non-PEGylated formulations. Overlapping of 90% PIs of the information components, however, suggested that there was no statistically significant difference between the frequency of hepatic injury reported with PEGylated versus non-PEGylated drug formulations. CONCLUSION: We did not find significant indicators of differential reporting of hepatic injury between PEGylated and non-PEGylated drug formulations in this exploratory analysis using one DMA. The analysis also suggests that comparative disproportionality methodology although not in itself determinative, could be one useful component of a risk management plan for monitoring the postmarketing experience of drug delivery systems that uses multiple methods and data streams

OBJECTIVE: Recent epidemiological studies identifying an association between some classes of gastric acid suppressants and Clostridium difficile colitis and community-acquired pneumonia prompted our analysis. Our objective was to retrospectively apply data mining algorithms (DMAs) to the Food and Drug Administration (FDA) drug safety database to see if they might have directed/redirected attention to the reported association of gastric acid suppressive drugs with C. difficile colitis and community-acquired pneumonia, prior to the published epidemiological findings that supported the association. DESIGN: Two statistical DMAs, proportional reporting ratios (PRRs) and multi-item gamma Poisson shrinker (MGPS), were applied to a spontaneous reporting system (SRS) database to identify signals of disproportionate reporting (SDRs). RESULTS: SDRs related to community-acquired pneumonia were observed for two proton pump inhibitors (lansoprazole and omeprazole), two H(2) antagonists (famotidine and roxatidine), and one antacid (magnesium silicate hydroxide). For C. difficile colitis, an SDR was generated for one proton pump inhibitor (lansoprazole). CONCLUSIONS: Although our analysis suggests that there may be an association between the SDRs using SRS data and the epidemiological findings, these results may not have alerted public health professionals in advance of published studies to an association between proton pump inhibitors/gastric acid suppressants and C. difficile colitis or community-acquired pneumonia. However, the analysis reveals the potential utility of DMAs to direct attention to more subtle indirect drug adverse effects in SRS databases that as yet are often identified from epidemiological investigations