AI Technology and Personalized Learning Design—Uncovering Unconscious Incompetence

Abstract

We posit that while much is made about new technologies, without application of sound learning science principles, the benefits of the new technology may not be realized. This may lead to mis-inferences when conducting evaluations of new technologies where the assumption is that the technology didn’t work when the issue actually related to intervention design or implementation. Using data from a large scale role out of a cutting edge AI-powered personalized learning technology, we explore these issues empirically and also uncover evidence of Maslow’s unconscious incompetence where learners perceive they have more expertise than the demonstrate. Finally, we create a model to compare the personalized learning approaches to a standard asynchronous learning approach applying sound learning science principles and find significant mean performance increase but also large increases in variance with respect to time to completion.