Reliability and validity testing of team emergency assessment measure in a distributed team context

Abstract

Medical multi-professional teams are increasingly collaborating via telemedicine. In distributed team settings, members are geographically separated and collaborate through technology. Developing improved training strategies for distributed teams and finding appropriate instruments to assess team performance is necessary. The Team Emergency Assessment Measure (TEAM), an instrument validated in traditional collocated acute-care settings, was tested for validity and reliability in this study when used for distributed teams. Three raters assessed video recordings of simulated team training scenarios (n = 18) among teams with varying levels of proficiency working with a remotely located physician via telemedicine. Inter-rater reliability, determined by intraclass correlation, was 0.74–0.92 on the TEAM instrument’s three domains of leadership, teamwork, and task management. Internal consistency (Cronbach’s alpha) ranged between 0.89–0.97 for the various domains. Predictive validity was established by comparing scores with proficiency levels. Finally, concurrent validity was established by high correlations, >0.92, between scores in the three TEAM domains and the teams’ overall performance. Our results indicate that TEAM can be used in distributed acute-care team settings and consequently applied in future-directed learning and research on distributed healthcare teams.