Faculty Bibliography

PURPOSE: To measure accuracy and speed for detection of vascular progression in retinopathy of prematurity (ROP) from serial images. Two strategies are compared: static side-by-side presentation and dynamic flickering of superimposed image pairs. DESIGN: Prospective comparative study. METHODS: Fifteen de-identified, wide-angle retinal image pairs were taken from infants who eventually developed plus disease. Image pairs representing vascular disease progression were taken >/=1 week apart, and control images without progression were taken on the same day. Dynamic flickering pairs were created by digital image registration. Ten experts independently reviewed each image pair on a secure website using both strategies, and were asked to identify progression or state that images were identical. Accuracy and speed were measured, using examination date and ophthalmoscopic findings as a reference standard. RESULTS: Using static images, experts were accurate in a mean (%) +/- standard deviation (SD) of 11.4 of 15 (76%) +/- 1.7 image pairs. Using dynamic flickering images, experts were accurate in a mean (%) +/- SD of 11.3 of 15 (75%) +/- 1.7 image pairs. There was no significant difference in accuracy between these strategies (P = .420). Diagnostic speed was faster using dynamic flickering (24.7 +/- 8.3 seconds) vs static side-by-side images (40.3 +/- 18.3 seconds) (P = .002). Experts reported higher confidence when interpreting dynamic flickering images (P = .001). CONCLUSIONS: Retinal imaging provides objective documentation of vascular appearance, with potentially improved ability to recognize ROP progression compared to standard ophthalmoscopy. Speed of identifying vascular progression was faster by review of dynamic flickering image pairs than by static side-by-side images, although there was no difference in accuracy.

PURPOSE: To review findings from the authors' published studies involving telemedicine and image analysis for retinopathy of prematurity (ROP) diagnosis. METHODS: Twenty-two ROP experts interpreted a set of 34 wide-angle retinal images for presence of plus disease. For each image, a reference standard diagnosis was defined from expert consensus. A computer-based system was used to measure individual and linear combinations of image parameters for arteries and veins: integrated curvature (IC), diameter, and tortuosity index (TI). Sensitivity, specificity, and receiver operating characteristic areas under the curve (AUC) for plus disease diagnosis were determined for each expert. Sensitivity and specificity curves were calculated for the computer-based system by varying the diagnostic cutoffs for arterial IC and venous diameter. Individual vessels from the original 34 images were identified with particular diagnostic cutoffs, and combined into composite wide-angle images using graphics editing software. RESULTS: For plus disease diagnosis, expert sensitivity ranged from 0.308-1.000, specificity from 0.571-1.000, and AUC from 0.784 to 1.000. Among computer system parameters, one linear combination had AUC 0.967, which was greater than that of 18 of 22 (81.8%) experts. Composite computer-generated images were produced using the arterial IC and venous diameter values associated with 75% under-diagnosis of plus disease (ie, 25% sensitivity cutoff), 50% under-diagnosis of plus disease (ie, 50% sensitivity cutoff), and 25% under-diagnosis of plus disease (ie, 75% sensitivity cutoff). CONCLUSIONS: Computer-based image analysis has the potential to diagnose severe ROP with comparable or better accuracy than experts, and could provide added value to telemedicine systems. Future quantitative definitions of plus disease might improve diagnostic objectivity.

PURPOSE: To demonstrate a methodology for generating composite wide-angle images of plus disease in retinopathy of prematurity (ROP), using quantitative analysis of expert opinions. METHODS: Thirty-four wide-angle retinal images were independently interpreted by 22 ROP experts as "plus" or "not plus." All images were processed by the computer-based Retinal Image multiScale Analysis (RISA) system to calculate two parameters: arterial integrated curvature (AIC) and venous diameter (VD). Using a reference standard defined by expert consensus, sensitivity and specificity curves were calculated by varying the diagnostic cutoffs for AIC and VD. From these curves, individual vessels from multiple images were identified with particular diagnostic cutoffs, and were combined into composite wide-angle images using graphics-editing software. RESULTS: The values associated with 75% underdiagnosis of true plus disease (i.e., 25% sensitivity cutoff) were AIC 0.061 and VD 4.272, the values associated with 50% underdiagnosis of true plus disease (i.e., a 50% sensitivity cutoff) were AIC 0.049 and VD 4.088, and the values associated with 25% underdiagnosis of true plus disease (i.e., 75% sensitivity cutoff) were AIC 0.042 and VD 3.795. Composite wide-angle images were generated by identifying and combining individual vessels with these characteristics. CONCLUSIONS: Computer-based image analysis permits quantification of retinal vascular features, and a spectrum of abnormalities is seen in ROP. Selection of appropriate vessels from multiple images can produce composite plus disease images corresponding to expert opinions. This method may be useful for educational purposes, and for development of future disease definitions based on objective, quantitative principles.

PURPOSE: To measure accuracy of plus disease diagnosis by recognized experts in retinopathy of prematurity (ROP), and to conduct a pilot study examining performance of a computer-based image analysis system, Retinal Image multiScale Analysis (RISA). METHODS: Twenty-two ROP experts independently interpreted a set of 34 wide-angle retinal images for presence of plus disease. A reference standard diagnosis based on expert consensus was defined for each image. Images were analyzed by the computer-based system using individual and linear combinations of system parameters for arterioles and venules: integrated curvature (IC), diameter, and tortuosity index (TI). Sensitivity, specificity, and receiver operating characteristic areas under the curve (AUC) for plus disease diagnosis compared with the reference standard were determined for each expert, as well as for the computer-based system. RESULTS: Expert sensitivity ranged from 0.308 to 1.000, specificity ranged from 0.571 to 1.000, and AUC ranged from 0.784 to 1.000. Among individual computer system parameters, venular IC had highest AUC (0.853). Among all computer system parameters, the linear combination of arteriolar IC, arteriolar TI, venular IC, venular diameter, and venular TI had highest AUC (0.967), which was greater than that of 18 (81.8%) of 22 experts. CONCLUSIONS: Accuracy of ROP experts for plus disease diagnosis is imperfect. A computer-based image analysis system has potential to diagnose plus disease with high accuracy. Further research involving RISA system parameter cut-off values from this study are required to fully validate performance of this computer-based system compared with that of human experts.

OBJECTIVE: To measure accuracy and reliability of the computer-based Retinal Image Multiscale Analysis (RISA) system compared with those of recognized retinopathy of prematurity (ROP) experts, for plus disease diagnosis. DESIGN: Evaluation of diagnostic test or technology. PARTICIPANTS: Eleven recognized ROP experts and the RISA image analysis system interpreted a set of 20 wide-angle retinal photographs for presence of plus disease. METHODS: All experts used a secure Web site to review independently 20 images for presence of plus disease. Images were also analyzed by measuring individual computer-based system parameters (integrated curvature [IC], diameter, and tortuosity index) for arterioles and venules and by computing linear combinations and logical combinations of those parameters. Performance was compared with a reference standard, defined as the majority vote of experts. MAIN OUTCOME MEASURES: Diagnostic accuracy was measured by calculating sensitivity, specificity, and receiver operating characteristic area under the curve (AUC) for plus disease diagnosis by each expert, and by each computer-based system parameter, compared with the reference standard. Diagnostic agreement was measured by calculating the mean kappa value of each expert compared with all other experts and the mean kappa value of each computer-based system parameter compared with all experts. RESULTS: Among the 11 experts, sensitivity ranged from 0.167 to 1.000, specificity ranged from 0.714 to 1.000, AUC ranged from 0.798 to 1.000, and mean kappa compared with all other experts ranged from 0.288 to 0.689. Among individual computer system parameters, arteriolar IC had the highest diagnostic accuracy, with sensitivity of 1.000; specificity, 0.846; and AUC, 0.962. Arteriolar IC had the highest diagnostic agreement with experts, with a mean kappa value of 0.578. CONCLUSIONS: A computer-based image analysis system has the potential to perform comparably to recognized ROP experts for plus disease diagnosis.