Comparison of remote and in-person respiratory function grading of brachycephalic dogs

Abstract

Objective: To compare the reliability of respiratory function grading (RFG) scores assigned in-person and remotely via video and electronic stethoscope recordings, evaluated by novice and expert graders. Study design: Prospective study. Sample population: Fifty-seven brachycephalic dogs. Methods: Dogs were evaluated in person by expert graders and RFG scores were assigned. Audio and video recordings were made during the in-person evaluations. Four expert and four novice graders evaluated the recordings and assigned an RFG score to each dog. Agreement between in-person and remote RFG scores was assessed using Cohen's kappa statistic. Interobserver reliability was assessed using Fleiss’ kappa statistic. Results: The median RFG score from the in-person assessment was 1 (range, 0–3). Distribution of RFG scores included 12 grade 0 scores, 19 grade 1 scores, 25 grade 2 scores, and 1 grade 3 score. The raw percentage agreements between remote and in-person scores were 68.4%, 59.6%, 64.9%, and 61.4% for the four experts, and 52.6%, 64.9%, 50.9%, and 42.1% for the four novices. Reliability between remote and in-person RFG scores was poor to moderate both for the experts (Cohen's kappa:.48,.37,.46,.41) and novices (Cohen's kappa:.28,.47,.28,.21). Interobserver reliability was moderate among the experts (Fleiss’ kappa:.59) and poor among the novices (Fleiss’ kappa:.39). Conclusion: Remote RFG scores had poor to moderate interassessment and interobserver reliability. Novice evaluators performed worse than experts for remote or in-person RFG evaluations. Clinical significance: Remote RFG, as measured in this study, is not reliable for assigning RFG scores. Modifications could be made to remote evaluation to improve reliability. Based upon the performance of novice evaluators, training of evaluators is justified.