Google Home, Listen: Building Helper Intelligences for Non-Verbal Sound

Abstract

Voice assistants have become a mainstay of our increasingly smart domestic spaces. Within this design-through-research inquiry, how new forms of intelligence might creatively respond to everyday interactions. In this, we are particularly interested in non-verbal communication and expression as a design material for interaction design. In this demonstration, we present a prototype illustrating this opportunity: a tinyML-enabled helper that augments the Google Home and enables it to recognize and respond to everyday phenomenon such as a sneeze, a door opening, or rapping on a table. The helper offers a creative toolkit for designers and end-users to quickly explore, understand, and imagine the affordances of everyday sounds.