Rapid composition for networked devices: Happybrackets

Abstract

This article introduces an open-source Java-based programming environment for creative coding ofagglomerative systems using Internet-of-Things (IoT) technologies. Our software originally focused on digital signalprocessing of audio—including synthesis, sampling, granular sample playback, and a suite of basic effects—butcomposers now use it to interface with sensors and peripherals through general-purpose input/output and externalnetworked systems. This article examines and addresses the strategies required to integrate novel embedded musicalinterfaces and creative coding paradigms through an IoT infrastructure. These include: The use of advanced toolingfeatures of a professional integrated development environment as a composition or performance interface rather thanjust as a compiler; techniques to create media works using features such as autodetection of sensors; seamless andserverless communication among devices on the network; and uploading, updating, and running of new compositionsto the device without interruption.Furthermore, we examined the difficulties many novice programmers experience when learning to write code,and we developed strategies to address these difficulties without restricting the potential available in the codingenvironment. We also examined and developed methods to monitor and debug devices over the network, allowingartists and programmers to set and retrieve current variable values to or from these devices during the performanceand composition stages. Finally, we describe three types of art work that demonstrate how the software, calledHappyBrackets, is being used in live-coding and dance performances, in interactive sound installations, and as anadvanced composition and performance tool for multimedia works.