Faculty Bibliography

This study evaluated multiple previously-identified Continuous Performance Test-Third Edition (CPT-3) scores as embedded validity indicators (EVIs) among 201 adults undergoing neuropsychological evaluation for Attention-Deficit/Hyperactivity Disorder (ADHD) divided into valid (n = 169) and invalid (n = 32) groups based on seven criterion measures. Although 6/10 CPT-3 scores accurately detected invalidity, only two reached minimally acceptable classification accuracy of ≥0.70. The remaining four had unacceptably low accuracy (AUCs = 0.62-0.69) with 0.19-0.41 sensitivity at ≥0.90 specificity. Composite scores did not provide better classification accuracy than individual CPT-3 scores. In sum, CPT-3 individual and composite scores generally are not accurate PVTs among adults undergoing clinical evaluation for ADHD.

Thorough assessment of performance validity has become an established standard of practice in neuropsychological assessment. While there has been a large focus on the development and cross-validation of embedded performance validity tests (PVTs) in recent years, new freestanding PVTs have also been developed, including the Word Choice Test (WCT) as part of the Advanced Clinical Solutions Effort System. And, while the WCT's general utility for identifying invalid performance has been demonstrated in the ensuing decade since its initial publication, optimal cut-scores and associated psychometric properties have varied widely across studies. This study sought to synthesize the existing diagnostic accuracy literature regarding the WCT via a systematic review and to conduct a meta-analysis to determine the performance validity cut-score that best maximizes sensitivity while maintaining acceptable specificity. A systematic search of the literature resulted in 14 studies for synthesis, with eight of those available for meta-analysis. Meta-analytic results revealed an optimal cut-score of ≤ 42 with 54% sensitivity and 93% specificity for identifying invalid neuropsychological test performance. Collectively, the WCT demonstrated adequate diagnostic accuracy as a PVT across a variety of populations. Recommendations for future studies are also provided.

The Victoria Symptom Validity Test (VSVT) is a performance validity test (PVT) with over two decades of empirical backing, although methodological limitations within the extant literature restrict its clinical and research generalizability. Chief among these constraints includes limited consensus on the most accurate index within the VSVT and the most appropriate cut-scores within each VSVT validity index. The current systematic review synthesizes existing VSVT validation studies and provides additional cross-validation in an independent sample using a known-groups design. We completed a systematic search of the literature, identifying 17 peer-reviewed studies for synthesis (7 simulation designs, 7 differential prevalence designs, and 3 known-groups designs). The independent cross-validation sample consisted of 200 mixed clinical neuropsychiatric patients referred for outpatient neuropsychological evaluation. Across all indices, Total item accuracy produced the strongest psychometric properties at an optimal cut-score of ≤ 40 (62% sensitivity/88% specificity). However, ROC curve analyses for all VSVT indices yielded statistically significant areas under the curve (AUCs; .73-81), suggestive of moderate classification accuracy. Cut-scores derived using the independent cross-validation sample converged with some previous findings supporting cut-scores of ≤ 22 for Easy item accuracy and ≤ 40 for Total item accuracy, although divergent findings were noted for Difficult item accuracy. Overall, VSVT validity indicators have adequate diagnostic accuracy across populations, with the current study providing additional support for its use as a psychometrically sound PVT in clinical settings. However, caution is recommended among patients with certain verified clinical conditions (e.g., dementia) and those with pronounced working memory deficits due to concerns for increased risk of false positives.

A forced-choice (FC) recognition trial was recently developed as an embedded validity indicator for the Rey Auditory Verbal Learning Test (RAVLT), although it has not been replicated outside of the initial validation study. This study cross-validated the RAVLT FC trial for detecting invalid neuropsychological test performance and assessed the degree to which material-specific verbal memory impairment severity impacts its classification accuracy as a performance validity test (PVT). This cross-sectional study included 172 neuropsychiatric patients who completed the RAVLT and 4 independent criterion PVTs, which were used to classify validity groups (134 valid/38 invalid). Overall results showed the RAVLT FC had excellent classification accuracy for detecting invalid performance at a ≤13 cut-score (66% sensitivity/87% specificity). When patients were subdivided by memory impairment status, FC retained excellent classification accuracy among the normal memory and mild memory impairment groups with 66%-82% sensitivity and ≥89% specificity. Accuracy decreased among those with severe memory impairment, but remained significant with a lower, alternative cut-score of ≤11 (37% sensitivity/88% specificity). Findings were consistent with FC trials developed for other memory measures and support the utility of this novel RAVLT FC index for reliably identifying invalid performance, even in the context of significant verbal memory impairment. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Although performance validity tests (PVTs) are an integral element of neuropsychological assessment, most PVTs have historically been restricted to the memory domain. The Dot Counting Test (DCT) is a nonmemory PVT shown to reliably identify invalid performance. Although several traditional and abbreviated scoring methods have been derived, no study to date has directly compared the available scoring approaches within a single sample. This cross-sectional study cross-validated 4 different DCT scoring approaches, including the traditional rounded E-score proposed within the manual, an unrounded E-score, and 2 abbreviated scoring procedures based on 4- and 6-card versions (DCT-4 and DCT-6, respectively) in a diverse mixed clinical neuropsychiatric sample (N = 132). Validity groups were established by 5 independent criterion PVTs (102 valid and 30 invalid). Receiver operating characteristic curve analyses yielded significant areas under the curve (AUCs = .84-.86) for the overall sample, with sensitivities of 50%-67% at ≥ 89% specificity. The DCT scores had outstanding classification accuracy (AUCs ≥ .92; sensitivities = 80%-83%) in the unimpaired group and excellent classification accuracy in the impaired group (AUCs = .79-.81; sensitivities = 43%-60%). Whereas negligible differences emerged between the 4 scoring methods for the cognitively intact group, the DCT-4 showed notably stronger psychometric properties among the overall sample in general and the mild cognitive impairment group in particular. Results corroborate previous findings suggesting that the DCT is a robust PVT, regardless of the employed scoring procedure, and replicate support for the abbreviated DCT-4 as the recommended validity indicator. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

This study examined the effect of increasing material-specific verbal and visual memory impairment severity on Rey 15-Item Test (RFIT) and RFIT/Recognition Trial performance. Data from 146 clinically referred patients (109 valid/37 invalid) who completed the RFIT, Brief Visuospatial Memory Test-Revised, and Rey Auditory Verbal Learning Test were analyzed. Rey Auditory Verbal Learning Test/BVMT memory impairment was operationalized as ≥40T (no memory impairment), 30T-39T (mild memory impairment), or ≤29T (severe memory impairment). Medium-to-large correlations emerged between the RFIT and memory measures. Significantly more patients with impaired visual memory, and to a lesser extent verbal memory, failed the RFIT and RFIT/Recognition. RFIT and RFIT/Recognition produced areas under the curve = .80-.90 for detecting invalidity and strong associated psychometric properties among patients without memory impairment, but both yielded low and largely unacceptable accuracy (areas under the curve = .57-.71) when verbal or visual memory impairment of any severity was present. In sum, RFIT performance was significantly affected by increasing material-specific memory deficits, such that it produced acceptable accuracy among unimpaired patients, but accuracy greatly diminished with memory impairment, which is antithetical to a sound performance validity test. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

INTRODUCTION:Embedded performance validity tests (PVTs) allow for continuous and economical validity assessment during neuropsychological evaluations; however, similar to their freestanding counterparts, a limitation of well-validated embedded PVTs is that the majority are memory-based. This study cross-validated several previously identified non-memory-based PVTs derived from language, processing speed, and executive functioning tests within a single mixed clinical neuropsychiatric sample with and without cognitive impairment. METHOD:This cross-sectional study included data from 124 clinical patients who underwent outpatient neuropsychological evaluation. Validity groups were determined by four independent criterion PVTs (failing ≤1 or ≥2), resulting in 98 valid (68% cognitively impaired) and 26 invalid performances. In total, 23 previously identified embedded PVTs derived from Verbal Fluency (VF), Trail Making Test (TMT), Stroop (SCWT), and Wisconsin Card Sorting Test (WCST) were examined. RESULTS:=.05-.22) with areas under the curve (AUCs) of.65-.81 and 19-54% sensitivity (≥89% specificity) at optimal cut-scores. When subdivided by impairment status, all PVTs except for WCST Failures to Maintain Set were significant (AUCs =.75-94) with 33-85% sensitivity (≥90% specificity) in the cognitively unimpaired group. Among the cognitively impaired group, most VF, TMT, and SCWT PVTs remained significant, albeit with decreased accuracy (AUCs =.65-.76) and sensitivities (19-54%) at optimal cut-scores, whereas all WCST PVTs were nonsignificant. Across groups, SCWT embedded PVTs evidenced the strongest psychometric properties. CONCLUSION:VF, TMT, and SCWT embedded PVTs generally demonstrated moderate accuracy for identifying invalid neuropsychological performance. However, performance on these non-memory-based PVTs from processing speed and executive functioning tests are not immune to the effects of cognitive impairment, such that alternate cut-scores (with reduced sensitivity if adequate specificity is maintained) are indicated in cases where the clinical history is consistent with cognitive impairment. In contrast, WCST indices generally had poor accuracy.

INTRODUCTION:Validity indicators embedded within standard neuropsychological tests have received increasing attention as more efficient measures for sampling performance validity throughout an evaluation. This cross-sectional study examined multiple performance validity tests (PVTs) embedded in the Wechsler Adult Intelligence Scale-IV (WAIS-IV) Working Memory (WMI) and Processing Speed (PSI) Indices for detecting invalid test performance. METHOD:This cross-sectional study examined data from a mixed clinical neuropsychiatric sample of 110 patients referred for outpatient evaluation. The sample was composed of 85 patients with valid neuropsychological performance and 25 with invalid performance based on multiple independent criterion PVTs. Among the patients with valid performance, 54% were cognitively impaired, whereas 46% were cognitively unimpaired. RESULTS: CONCLUSION:Overall, results indicated that embedded WAIS-IV WMI and PSI are useful embedded PVTs in conditions in which cognitive impairment is not expected; however, these embedded PVTs demonstrated questionable utility among patients with cognitive impairment due to poor sensitivity, if adequate specificity is maintained, suggesting limited efficacy among patients with cognitive impairment due to risk of false-positive classification.