Sand-in-the-loop: Investigating embodied co-creation for shared understandings of generative AI

Abstract

Generative Artificial Intelligence (AI) applications in creative practices have grown tremendously over the past few years. However, these generative AI applications lack clarity on how they work and operate for their users. Revealing how generative AI tools work enables designers to understand these tools' limitations and capabilities. We developed a tangible interface with sand as a medium for human-AI co-creation, Sand Playground. It extends the work in human-AI drawing practices beyond two-dimensional digital surfaces. Sand playground has three co-drawing modes, artistic mimicry, zen garden and doodling. We conducted a user study with ten designers. One of our findings is that the interface enabled users to predict the AI agent's actions. Our research introduces novel insights into the role of tangible interactions and physical interfaces in generative AI literacy and explainability in design tools.