Performance analysis in esports: part 1 - the validity and reliability of match statistics and notational analysis in League of Legends

Abstract

Performance analysis in sports objectively captures aspects of athlete performances to inform coaching. Comparatively, esports is an emerging expertise domain with limited performance analysis research, however, no research has yet investigated the quality of available data. Therefore, this research aimed to: 1) assess the validity of publicly accessible Match History statistics from professional League of Legends matches; 2) assess the agreement of notational analysis between three experienced players; and 3) assess the agreement between a novice and Match Histories. 30 professional matches were randomly selected from the North American and European 2019 seasons. The Match Histories for each match were copied from the publicly accessible repository, while each of the authors independently viewed videos on the public Video On Demand repository, and encoded action variables corresponding to the Match Histories. Data were compared 1) between the most experienced player and Match Histories; 2) between the three experienced authors; and 3) between the novice author and Match Histories. Krippendorff’s Alpha was calculated with acceptable agreement set at α ≥ 0.8. The most experienced player was in good agreement with Match Histories (α=0.868-1.000), while the novice rater was in good agreement except for First Tower Time (α=0.764) and First Inhibitor Team (0.740). The three experienced players were in good agreement (α=0.861- 1.000). Match Histories and experienced player annotations can be used interchangeably for all observed measures to facilitate performance analysis in professional League of Legends. A novice can be used to capture some basic statistics, while other measures require domain expertise.