“what i see is what you get” explorations of live artwork generation, artificial intelligence, and human interaction in a pedagogical environment

Abstract

In this paper we review the overall process for the design, development, and deployment of “What I See Is What You Get”, an experiential installation that creates live interactive visuals, by analyzing human facial expressions and behaviors, accompanied by text generated using Machine Learning algorithms trained on the art collection of The J. Paul Getty Museum in Los Angeles. The project is developed by students and faculty in an academic environment and exhibited at the Getty Museum. We also study the pedagogical process implemented to address the curriculum’s learning outcomes in an “applied” environment while designing a contemporary new media art piece. Special attention is paid to the level and quality of the interaction between users and the piece, demonstrating how advances in technology and computing such as Deep Learning and Natural Language Processing can contribute to deeper connections and new layers of interactivity.