Faculty Bibliography

OBJECTIVE: Data mining algorithms (DMAs) are being applied to spontaneous reporting system (SRS) databases in the hope of obtaining timely insights into post-licensure safety data. Some DMAs have been characterized as 'objective' screening tools. However, there are numerous available modifiable configuration parameters to choose from, including choice of vendor, that may affect results. Our objective is to compare the data mining results on pre-selected drug-event combinations (DECs) between two commonly used software programs using similar protocols. METHODS: Two DMAs, using three thresholds, were retrospectively applied to the USFDA safety database through Q2 2005 to a set of eight pre-selected DECs. RESULTS: Differences between the two vendors were found for the number of cases associated with a signal of disproportionate reporting (SDR), first year of SDRs, and the magnitude of the SDR scores for the selected DECs. These were deemed to be potentially significant for 45.8% (11/24) of the data points. CONCLUSION: The observed differences between vendors could partially be explained by their differing methods of data cleaning and transformation as well as by the specific features of individual algorithms. The choices of vendors and available data mining configurations maximize the exploratory capacity of data mining, but they also raise questions about the claimed objectivity of data mining results and can make data mining exercises susceptible to confirmation bias given the exploratory nature of data mining in pharmacovigilance. When reporting results, the vendor and all data mining configuration details should be specified

PURPOSE: To provide commentary and points of caution to consider before incorporating data mining as a routine component of any Pharmacovigilance program, and to stimulate further research aimed at better defining the predictive value of these new tools as well as their incremental value as an adjunct to traditional methods of post-marketing surveillance. METHODS/RESULTS: Commentary includes review of current data mining methodologies employed and their limitations, caveats to consider in the use of spontaneous reporting databases and caution against over-confidence in the results of data mining. CONCLUSIONS: Future research should focus on more clearly delineating the limitations of the various quantitative approaches as well as the incremental value that they bring to traditional methods of pharmacovigilance

Drug safety profiles are dynamic and established over time using multiple, complimentary datasets and tools. The principal concern of pharmacovigilance is the detection of adverse drug reactions that are novel by virtue of their clinical nature, severity and/or frequency as soon as possible with minimum patient exposure. A key step in the process is the detection of 'signals' that direct safety reviewers to associations that might be worthy of further investigation. Although the 'prepared mind' remains the cornerstone of signal detection safety reviewers seeking potential signals by scrutinising very large, sparse databases may find themselves 'drowning in data but thirsty for knowledge'. Understandably, health authorities, pharmaceutical companies and academic centres are developing, testing and/or deploying computer-assisted database screening tools (also known as data-mining algorithms [DMAs]) to assist human reviewers. The most commonly used DMAs involve disproportionality analysis that project high-dimensional data onto two-dimensional (2 x 2) contingency tables in the context of an independence model. The objective of this paper is to extend the discussion of the evaluation, potential utility and limitations of the commonly used DMAs by providing a 'holistic' perspective on their use as one component of a comprehensive suite of signal detection strategies incorporating clinical and statistical approaches to signal detection -- a marriage of technology and the 'prepared mind'. Data-mining exercises involving spontaneous reports submitted to the US FDA will be used for illustration. Potential pitfalls and obstacles to the acceptance and implementation of data mining will be considered and suggestions for future research will be offered

INTRODUCTION: A population-based analysis has suggested that the publication of the RALES (Randomized Aldactone Evaluation Study) in late 1999 was associated with both the wider use of spironolactone to treat heart failure and a corresponding increase in hyperkalaemia-associated morbidity and mortality in patients also being treated with ACE inhibitors. OBJECTIVES: To gain further insight into the reporting of spironolactone-associated hyperkalaemia in an independent dataset by analysing the spontaneous reporting experience in relation to the publication of RALES, and to determine whether the implementation of a commonly used data mining algorithm (DMA) might have directed the attention of safety reviewers to the spironolactone/hyperkalaemia association in advance of epidemiological findings. METHODS: We calculated the reporting rate of spironolactone-associated hyperkalaemia per 1,000 reports per year from 1970 through to the end of 2005 by identifying relevant cases in the US FDA Adverse Event Reporting System. We did this for reports of spironolactone-associated hyperkalaemia (where spironolactone was listed as a suspect drug) and according to whether the reports listed an ACE inhibitor as a co-suspect or concomitant medication. A further statistical analysis of the overall reporting of spironolactone (suspect drug)-associated hyperkalaemia was also performed. We also performed 3-dimensional (3-D; drug-drug-event) disproportionality analyses using a DMA known as the multi-item gamma-Poisson shrinker, which allows the calculation and display of a 3-D disproportionality metric known as the 'interaction signal score' (INTSS). This metric is a measure of the strength of a higher order reporting relationship of a triplet (i.e. drug-drug-event) association above and beyond what would be expected from the largest disproportionalities associated with the individual 2-way associations. RESULTS: Visual inspection of a graph of the reporting frequency of spironolactone (suspect drug)-associated hyperkalaemia per 1,000 reports was highly suggestive of a change point. The t-test on the arcsine-transformed data showed a significant difference in reporting of spironolactone-hyperkalaemia combination through 1999 compared with 2000 onwards (p < 0.001). When examining the reporting time trends according to the presence or absence of an ACE inhibitor, the change point seemed to be mostly attributable to an increase in the number of spironolactone (suspect drug)-associated hyperkalaemia reports with ACE inhibitors listed as a co-suspect drug. No obvious change points in INTSSs for spironolactone-ACE inhibitor-hyperkalaemia reports were observed. DISCUSSION: Although we could not pinpoint the relative contribution of many possible artifacts in the reporting process, as well as increased drug exposure, increased adverse event incidence and/or a change in patient monitoring practices, to our findings, we observed a notable change in reporting frequency of spironolactone-associated hyperkalaemia in temporal proximity to the publication of RALES. Evidence of this was provided by a trend analysis depicted in a simple graph that was supported by statistical analysis. The observed trend was in large part due to increased reporting of spironolactone-associated hyperkalaemia with reported co-medication with ACE inhibitors. CONCLUSION: These findings are consistent with those originally reported in an epidemiological analysis. In this retrospective exercise, a simple graph was more illuminating than more complex data mining analyses

Anecdotal reports of adverse drug reactions are generally regarded as being of poor evidential quality. This is especially relevant for postmarketing drug safety surveillance, which relies heavily on spontaneous anecdotal reports. The numerous limitations of spontaneous reports cannot be overemphasised, but there is another side to the story: these datasets also contain anecdotal reports that can be considered to describe definitive adverse reactions, without the need for further formal verification. We have previously defined four categories of such adverse reactions: (i) extracellular or intracellular tissue deposition of the drug or a metabolite; (ii) a specific anatomical location or pattern of injury; (iii) physiological dysfunction or direct tissue damage demonstrable by physicochemical testing; and (iv) infection, as a result of the administration of an infective agent as the therapeutic substance or because of demonstrable contamination. In this article, we discuss the implications of these definitive ('between-the-eyes') adverse effects for pharmacovigilance

BACKGROUND AND OBJECTIVE: Various data mining algorithms (DMAs) that perform disporportionality analysis on spontaneous reporting system (SRS) data are being heavily promoted to improve drug safety surveillance. The incremental value of DMAs is ultimately related to their ability to detect truly unexpected associations that would have escaped traditional surveillance and/or their ability to identify the same associations as traditional methods but with greater scientific efficiency. As to the former potential benefit, in the course of evaluating DMAs, we have observed what we call 'surprise reactions'. These adverse reactions may be discounted in manual review of adverse drug reaction (ADR) lists because they are less clinically dramatic, less characteristic of drug effects in general, less serious than the classical type B hypersensitivity reactions or may have subtle pharmacological explanations. Thus these reactions may only become recognised when post hoc explanations are sought based on more refined pharmacological knowledge of the formulation. The objective of this study was to explore notions of 'unexpectedness' as relates to signal detection and data mining by introducing the concept of 'surprise reactions' and to determine if the latter associations, often first reported in the literature, represent a type of ADR amenable to detection with the assistance of adjunctive statistical calculations on SRS data. METHODS: Using commonly cited thresholds, the multi-item gamma Poisson shrinker (MGPS) and proportional reporting ratios (PRRs) were applied to reports in the US FDA Adverse Event Reporting System (AERS) database of well documented 'surprise reactions' compiled by the authors. RESULTS: There were 34 relevant surprise reactions involving 29 separate drugs in 17 different drug classes. Using PRRs (PRR >2, chi(2) >4, N >2), 12 drug-event combinations were signalled before the first ADR citation appeared in MEDLINE, four occurred concurrently and 11 after. With empirical Bayes geometric mean (EBGM) analysis (EBGM >2, N >0), 12 signals occurred before, three concurrently and 11 after publication of the first literature citation. With EB(05) (EB(05)> or =2, N >0), six occurred before, two concurrently and 14 after MEDLINE citation. DISCUSSION: Pharmacovigilance is rather unique in terms of the number and variety of events under surveillance. Some events may be more appropriate targets for statistical approaches than others. The experience of many organisations is that most statistical disproportionalities represent known associations but our findings suggest there could be events that may be discounted on manual review of adverse event lists, which may be usefully highlighted via DMAs. CONCLUSIONS: Identification of surprise reactions may serve as an important niche for DMAs