Automated assessment of the visual design of android apps developed with app inventor

Abstract

One way to teach computing in K-12 is through the development of mobile applications with App Inventor. Although already broadly used worldwide, there is still a need for support for the assessment of the applications created by the students. Existing rubrics focusing mostly on programming concepts do not cover more comprehensively the performance-based assessment of user interface design concepts, important for the usability and aesthetics of the applications. Thus, in order to support the assessment of the visual design of apps based on its compliance with design theory and guidelines, we developed the CodeMaster UI Design App Inventor rubric in the context of computing education. In order to facilitate its application in practice, we automated the assessment of applications created with App Inventor through a static code analysis by an online tool. We evaluated the reliability and validity of the rubric based on the automated assessment of 1,775 projects from the App Inventor Gallery. The results indicate that the rubric can be considered reliable (Cronbach's alpha = 0.84). In terms of construct validity, there is also evidence of convergent validity. The results presented in this article can be used to support the assessment of computing education in practice as well as to point out further research opportunities.