Self-assessment in laparoscopic surgical skills training: Is it reliable?

Abstract

Background: The concept of self-assessment has been widely acclaimed for its role in the professional development cycle and self-regulation. In the field of medical education, self-assessment has been most used to evaluate the cognitive knowledge of students. The complexity of training and evaluation in laparoscopic surgery has previously acted as a barrier in determining the benefits self-assessment has to offer in comparison with other fields of medical education. Methods: Thirty-five surgical residents who attended the 2-day Laparoscopic Surgical Skills Grade 1 Level 1 curriculum were invited to participate from The Netherlands, India and Romania. The competency assessment tool (CAT) for laparoscopic cholecystectomy was used for self- and expert-assessment and the resulting distributions assessed. Results: A comparison between the expert- and self-assessed aggregates of scores from the CAT agreed with previous studies. Uniquely to this study, the aggregates of individual sub-categories—‘use of instruments’; ‘tissue handling’; and errors ‘within the component tasks’ and the ‘end product’ from both self- and expert-assessments—were investigated. There was strong positive correlation (rs > 0.5; p < 0.001) between the expert- and self-assessment in all categories with only the ‘tissue handling’ having a weaker correlation (rs = 0.3; p = 0.04). The distribution of the mean of the differences between self-assessment and expert-assessment suggested no significant difference between the scores of experts and the residents in all categories except the ‘end product’ evaluation where the difference was significant (W = 119, p = 0.03). Conclusion: Self-assessment using the CAT form gives results that are consistently not different from expert-assessment when assessing one’s proficiency in surgical skills. Areas where there was less agreement could be explained by variations in the level of training and understanding of the assessment criteria.