Design, validation, and reliability of an observation instrument for technical and tactical actions of the offense phase in soccer

Abstract

The use of observational methodology in the sports context provides coaches and other sports professionals with flexible tools that adapt to their needs. In collective sports, the use of these instruments is common for the technical and tactical analysis of the game. Based on the importance of data quality in these instruments, the purpose was to design, validate, and test the reliability of a mixed observational instrument of field formats and category systems to analyze technical and tactical actions in the offense phase in soccer. The instrument collects information regarding the actions with the ball, moment of the play (start, development, and end), and contextual situation for the offensive team and for the goalkeeper. The instrument design, validation, and reliability calculation were done in four stages: (a) review of the literature, (b) design the first draft of the instrument, (c) experts' qualitative and quantitative review of the instrument, and (d) observer training test (reliability calculation). The content validity was established by 12 experts (Ph.D. in sports science or soccer coach with at least of 10 years of coaching experience). The Delphi methodology was used. Experts did a quantitative (scale 0-10) and qualitative evaluation. Experts were asked about: (a) comprehension of the criteria, categorical cores, degree of openness, and their definitions, (b) pertinence of categorical cores and degree of openness, and (c) whether to include other categorical cores or degree of openness in the observation instrument. The lowest Aiken's V index was 0.91 for the categorical core "numerical situation with opponent goalkeeper." The inter- and intra-observer reliability presented good levels of agreement. The lowest Kappa index was 0.96 for the inter-reliability in the categorical core "defensive pressing lines" and was 0.98 for the intra-reliability in the categorical core "ball height (start of ball possession)," "distance of the defensive player," "ball height (end of ball possession)," "numerical situation," and "defensive pressing lines." The coefficients of the generalizability analysis showed a high level of accuracy, validity and reliability of the instrument. The results show that the instrument allows to obtain objective, valid and reliable information about the offensive phase in soccer.