Faculty Bibliography

OBJECTIVE:To determine the feasibility, validity, inter-rater, and intertest reliability of 4 previously published video-based rating scales, for technical skills assessment on a benchmark laparoscopic procedure. SUMMARY BACKGROUND DATA/BACKGROUND:Assessment of technical skills is crucial to the demonstration and maintenance of competent healthcare practitioners. Traditional assessment methods are prone to subjectivity through a lack of proven validity and reliability. METHODS:Nineteen surgeons (6 novice and 13 experienced) performed a median of 2 laparoscopic cholecystectomies each (range 1-5) on 53 patients within 2 Academic Surgical Departments. All patients had a diagnosis of biliary colic. Surgical technical skills were rated posthoc in a blinded manner by 2 experienced observers on 4 video-based rating scales. The different scales used had been developed to assess generic or procedure-specific technical skills in a global manner, or on a procedure-specific checklist. RESULTS:Six of 53 procedures were excluded on the basis of intraoperative difficulty. Of the remaining 47 procedures, 14 were performed by 6 novice surgeons and 33 by the 13 experienced surgeons. There were statistically significant differences between performance of the 2 groups on the generic global rating scale (median 24 vs. 27, P = 0.031), though not on procedural or checklist-based scales. All scales demonstrated inter-rater reliability (alpha = 0.58-0.76), though only the global rating scales exhibited intertest reliability (alpha = 0.72). CONCLUSIONS:Video-based technical skills evaluation in the operating room is feasible, valid and reliable. Global rating scales hold promise for summative assessment, though further work is necessary to elucidate the value of procedural rating scales.

OBJECTIVE:To assess the use of a synchronized video-based motion tracking device for objective, instant, and automated assessment of laparoscopic skill in the operating room. SUMMARY BACKGROUND DATA/BACKGROUND:The assessment of technical skills is fundamental to recognition of proficient surgical practice. It is necessary to demonstrate the validity, reliability, and feasibility of any tool to be applied for objective measurement of performance. METHODS:Nineteen subjects, divided into 13 experienced (performed >100 laparoscopic cholecystectomies) and 6 inexperienced (performed <10 LCs) surgeons completed LCs on 53 patients who all had a diagnosis of biliary colic. Each procedure was recorded with the ROVIMAS video-based motion tracking device to provide an objective measure of the surgeon's dexterity. Each video was also rated by 2 experienced observers on a previously validated operative assessment scale. RESULTS:There were significant differences for motion tracking parameters between the 2 groups of surgeons for the Calot triangle dissection part of procedure for time taken (P = 0.002), total path length (P = 0.026), and number of movements (P = 0.005). Both motion tracking and video-based assessment displayed intertest reliability, and there were good correlations between the 2 modes of assessment (r = 0.4 to 0.7, P < 0.01). CONCLUSIONS:An instant, objective, valid, and reliable mode of assessment of laparoscopic performance in the operating room has been defined. This may serve to reduce the time taken for technical skills assessment, and subsequently lead to accurate and efficient audit and credentialing of surgeons for independent practice.