Feasibility and Preliminary Data for a Training Protocol and Perceptual Rating Scale of Linguistic Conversation Measures in Aphasia

Abstract

Purpose: Speech-language pathologists and people with aphasia recognize the importance of social conversation (Wallace et al., 2017; Worrall et al., 2011). However, linguistic analysis of conversation is time intensive and difficult to clinically implement. Perceptual rating scales are a potential method that could increase the clinical feasibility of evaluating linguistic aspects of conversation. In this study, a perceptual training and rating protocol were created, and validity, reliability, and rater confidence were explored. Method: A protocol was developed to train four linguistic measures of conversation and practice perceptual ratings of those measures. The four measures included percent Correct Information Units, percent utterances with subject + verb structure (+SV Utterances), Communicative Success, and Global Coherence. Thirteen research assistants (RAs) completed the training and then five test ratings in which they rated 3-min conversation samples on the four linguistic measures. RAs also rated their confidence and provided feedback on the training. Results: Trained RAs rated the four linguistic measures within 20% of the stablished utterance-by-utterance lab-coded values on average, but with variable performance on individual conversations. Communicative Success and + V Utterances demonstrated good interrater reliability based on intraclass correlation coefficients. A significant weak negative correlation was found between RA rating confidence and the difference between coded values and perceptual ratings for +SV Utterances and Communicative Success. Discussion: These findings suggest that further development and refinement are required before this training procedure can be implemented in a clinical setting. Valuable insight was gained that will help refine the training for future research. Supplemental Material: https://doi.org/10.23641/asha.29594606