Ten-Hand Piano: A Networked Music Installation

Álvaro Barbosa
Research Center for Science and Technology of the Arts (CITAR)
Portuguese Catholic University – School of the Arts
Rua Diogo Botelho 1327, 4169-005 Porto, Portugal
+351 22 616 62 91
abarbosa@porto.ucp.pt


ABSTRACT
This paper presents the latest developments of the Public Sound
Objects (PSOs) system, an experimental framework to implement
and test new concepts for Networked Music. The project of a
Public interactive installation using the PSOs system was
commissioned in 2007 by Casa da Musica, the main concert hall
space in Porto. It resulted in a distributed musical structure with
up to ten interactive performance terminals distributed along the
Casa da Musica’s hallways, collectively controlling a shared
acoustic piano. The installation allows the visitors to collaborate
remotely with each other, within the building, using a software
interface custom developed to facilitate collaborative music
practices and with no requirements in terms previous knowledge
of musical performance.
Keywords
Network Music Instruments; Real-Time Collaborative
Performance; Electronic Music Instruments; Behavioral Driven
Interfaces; Algorithmic Composition; Public Music; Sound
Objects;
1. INTRODUCTION
The Public Sound Objects (PSOs) project consists of the
development of a networked musical system, which is an
experimental framework to implement and test new concepts for
on-line music communication. It not only serves a musical
purpose, but it also facilitates a straight-forward analysis of
collective creation and the implications of remote communication
in this process.
The project was initiated in 2000 [1] [2] at the Music Technology
Group (MTG) from the Pompeu Fabra University in Barcelona,
and most developments since 2006 have been undertaken by the
Research Center for Science and Technology of the Arts (CITAR)
at the Portuguese Catholic University in Porto.
The PSOs system approaches the idea of collaborative musical
performances over a computer network as a Shared Sonic
Environment aiming to go beyond the concept of simply using
computer networks as a channel to connect performing spaces. It
can run entirely over WWW, and its underlying communication
protocol (Hypertext Transfer Protocol - HTTP), in order to
perform over a regular Internet Connection and achieve the sense
of a Public Acoustic Space where anonymous users can meet and
be found performing in collective Sonic Art pieces.
The system itself is an interface-decoupled Musical Instrument, in
which a remote user interface and a sound processing engine
reside with different hosts, given that it is possible to
accommodate an extreme scenario where a user can access the
synthesizer from any place in the world using a web browser.
Specific software features were implemented in order to reduce
the disruptive effects of network latency [3], such as dynamic
adaptation of the musical tempo and dynamics to communication
latency measured in real-time.
In particular, the recent developments presented in this paper,
result from a commission in 2007 of an Interactive Sonic Art
Installation form Casa da Musica, the main concert hall space in
Porto. The resulting Setup is a distributed musical structure with
up to ten interactive performance terminals distributed along the
Casa da Musica’s hallways, collectively controlling a shared
acoustic piano.
It Includes:
• The adaptation of the Original synthesizer (a Pure-Data
[4] sound Engine) to a Yamaha Disklavier Piano [5]
• Redesign of the interactive sound paradigm in order to
constructively articulate multiple instances of
experimental users to an ongoing musical piece in real
time.
• Introduction o an Ethersound [6] acoustic broadcast
system for the clients musical feed-back
• Design of a physical infrastructure, coherent with the
Casa da Musica architecture, to support the client and
server terminals.
2. BACKGROUND TOPICS
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee.
NIME08, June 4-8, 2008, Genova, Italy
Copyright remains with the author(s).

2.1 Sound Objects
Community-driven creation, results in a holistic process, i.e., its
properties cannot be determined or explained by the sum of its
components alone [7]. A community of users involved in a

creation process, through a Shared Sonic Environment, definitely
constitutes a Whole in Holistic sense.
According to Jan Smuts (1870-1950), the father of Holism
Theory, the concept of a Whole implies its individual parts to be
flexible and adjustable. It must be possible for the part to be
different in the whole from what it is outside the whole. In
different wholes a part must be different in each case from what it
is in its separate state.
Furthermore, the whole must itself be an active factor or influence
among individual parts, otherwise it is impossible to understand
how the unity of a new pattern arises from its elements. Whole
and parts mutually and reciprocally influence and modify each
other.
Similarly, when questioning object’s behaviors in Physics it is
often by looking for simple rules that it is possible to find the
answers. Once found, these rules can often be scaled to describe
and simulate the behavior of large systems in the Real World.
This notion applies to the Acoustic Domains through the
definition of Sound Objects as a relevant element of the music
creation process by Pierre Schaeffer in the 1960’s. According to
Schaeffer, a Sound Object is defined as:
“Any sound phenomenon or event perceived as a coherent whole
(…) regardless of its source or meaning” (Schaeffer, P., 1966).
Sound Object (I’object sonore), refers to an acoustical object for
human perception and not a mathematical or electroacoustical
object for synthesis. One can consider a sound object the smallest
self-contained particle of a Soundscape [8]. Defining a universe of
sound events by subsets of Sound Objects is a promising approach
for content-processing and transmission of audio [9], and from a
psychoacoustic and perceptual point of view it provides a very
powerful paradigm to sculpt the symbolic value conveyed by a
Soundscape.
In an artistic context the scope for the user’s personal
interpretation is wider. Therefore such Sound Objects can have a
much deeper symbolic value and represent more complex
metaphors. Often there is no symbolic value in a sound, but once
there is a variation in one of its fundamental parameters it might
then convey a symbolic value.
All these ideas about Sound Objects and the Holistic nature of
community music are the basis for the main concept behind the
Public Sound Objects System. In fact, in PSOs raw material
provided for each user, to create his contribution to a shared
musical piece, is a simple Sound Object. These Sound Objects,
individually controlled, become part of a complex collective
system in which several users can improvise simultaneously and
concurrently.
In the system a server-side real-time sound synthesis engine (a
Disklavier Piano in the case of the Casa da Musica installation)
provides an interface to transform various parameters of a Sound
Object, which enables users to add symbolic meaning to their
performance.
2.2 About Networked Music
In his Keynote Speech from ICMC 2003 Roger Dannenberg
mentioned “Networked Music” as one of the promising research
topics and at least four papers [2], [10] and [11] were centered on
this topic, even though they were scattered over different panels,
instead of one distinct session.
Since then the term Networked Music has become increasingly
consensual in defining the area, and according to Jason Freeman’s
definition [12]: it is about music practice situations where
traditional aural and visual connections between participants are
augmented, mediated or replaced by electronically-controlled
connections.
In order to have a broad view over the scientific dissemination of
Networked Music research I present some of the most significant
Landmarks in the field over the last decade:
2.2.1 Summits and Workshops
The ANET Summit (August 20-24, 2004)
The summit was organized by Stanford University’s Center for
Computer Research in Music and Acoustics (CCRMA) and held
at the Banff Center in Canada, was the first Workshop event
addressing the topic of High quality Audio over Computer
Networks. The guest lecturers were Chris Chafe, Jeremy
Cooperstock, Theresa Leonard, Bob Moses and Wieslaw
Woszczyk. A New edition of the ANET Summit is planed for
April 2008
The Networked Music Workshop at ICMC (September 4, 2005).
This Workshop was held in Barcelona and resulted from
experience in previous ICMCs, which called for the need to
realize such an event. Guest Lecturers were: Álvaro Barbosa
(Pompeu Fabra University, MTG), Scot Gresham-Lancaster
(Cogswell College Sunnyvale, CA), Jason Freeman (Georgia
Institute of Technology), Ross Bencina (Pompeu Fabra
University, MTG).
2.2.2 PhD Dissertations
These are some relevant dissertations published on the topic:
2002 Golo Föllmer “Making Music on the Net, social and
aesthetic structures in participative music” [13]; 2002 Nathan
Schuett “The Effects of Latency on Ensemble Performance” [14];
2003 Jörg Stelkens “Network Synthesizer” [15]; 2003 Gil
Weinberg “Interconnected Musical Networks: Bringing
Expression and Thoughtfulness to Collaborative Music” [16];
2006 Álvaro Barbosa “Displaced Soundscapes” [17]
2.2.3 Journal Articles
There is a number of Survey and partial overview articles on the
topic of Networked Music [18], [19], [20] [21] and [22] however
a special issue of the journal Organised Sound from 2005 [23],
edited by Leigh Landy, specifically focused on the topic of
Networked Music and includes many of the relevant references in
this area.
3. THE PSOs INSTALLATION
Casa da Musica is the main concert venue in the city of Porto,
and it has a strong activity in what concerns contemporary and
experimental forms of Music. The commission for the Public
Sound Objects Installation had the underlying idea of bringing
music to the hallways of the house of music, so that the visitors
could actually interact with it.

Fig.1 Casa da Musica Building1
The final implementation consists of a Disklavier Piano controlled
via MIDI by s server that simultaneously can be used as a
terminal, located at the main foyer of Casa da Musica. This server
accepts incoming control data generated by 10 client computers
located in diverse points of the hallways of a scenic route of the
building. Incoming data is transmitted over the building’s IP
Network using Open Sound Control [24].

Fig. 2 The PSOs Server connected to the Disklavier Piano and
two of the clients which remotely control the same Piano
The sound generated at the Servers site conveys the overall
performance of every user and is streamed back to each client
using an ETHERSOUND [6] system, which produces latencies
under 100 ms on the building’s LAN.

Fig. 3 A PSOs Client with the ETHERSOUND Hub, Speakers
and Keyboard concealed on the structure.
All the computer hardware for the server and clients has been
cloaked by a metal structure created in coherence with the
building’s unique architecture (a project by Rem Koolhaas), so
that the users only access a one key mouse and a screen, or in
case of the server a touch screen.

1 Image Source “House of Music Opening Day” Wikipedia Commons
under the license GFDL (GNU Free Documentation License)
3.1 The User Interface
The graphical user interface is based on a bi-dimensional
graphical metaphor of an ever-going bouncing ball enclosed in a
square shape box. Each time the ball hits one of the walls a piano
key is triggered by the server according to a pitch defined by the
value of a stylized fader that frames the Box (each fader
determines the pitch of a sound triggered in its adjacent wall).

Fig. 4 PSOs Client interface showing the representation of 5 users
Each of the clients actuating at a given moment are visually
represented in real-time by grey balls while the user himself
controls a distinctive orange ball. The user can also add a trail to
his ball producing an arpeggio sound (or a chord if the trail
extension is zero), given that the scale of notes each client can
produce was anticipated to create a harmonic soundscape when
different sound overlap in time.
The PSOs system integrates several features to overcome
Network Latency issues already published in [3]. Nonetheless, in
this version a new Latency tolerance feature was implemented to
improve the perceptive correlation between an impact and a
triggered sound, using a simple sound panorama adjustment at the
sound server and consequently adding consistent sound panning
with the object’s behavior at the graphical user interface.
Δt 2
TimeΔt 1
Δt 3
R
L
t 1 t 2 t 3

Fig. 5 Representation of Impacts VS Triggered Sound with sound
panorama adjustment in the presence of latency (Dt)

The basic idea consists of only transmitting a sound object trough
the Right Channel of the streamed Soundscape stereo mix, when a
ball hits the right wall, transmitting only through the Left Channel
when a ball hits the left wall and transmitting in booth channels
(L+R) if the ball hits the top or bottom wall.
Sound Panorama Adjustment adds an extra cue to perception in
temporal order of triggered Sound Objects and respective
correlation to ball impacts.
4. CONCLUSIONS AND FUTURE WORK
The PSOs Installation at Casa da Musica allows a piano to be
controlled by 10 instances simultaneously (Ten Hands!) in a
coherent and constructive manner, which would hardly be
possible to do in a traditional way.
Even though the interface is radically different than the normal
control paradigm of a piano it is based on the same fundamental
musical facets (Rhythm, Pitch, Timbre and Dynamic) and
therefore it is an engaging experience, since the users recognize a
familiar result achieved trough a totally different way.
The interface is simple enough to achieve a musical soundscape
with zero learning time and without any previous musical practice
experience, which made the system very accessible and popular
for the average 500 daily visitors of the Casa da Musica.
Controlling a popular acoustical instrument brings the users closer
to the musical experience and in this sense we would like to
further develop this system adding a pool of instruments to the
piano, such as wind, string and percussion instruments controlled
by Robotics.
5. ACKNOWLEDGMENTS
The author would like to thank the people that collaborated in this
project: Jorge Cardoso (UCP), Jorge Abade (UCP) and Paulo
Maria Rodrigues (Casa da Musica).
6. REFERENCES
[1] Barbosa, A. and Kaltenbrunner, M. Public Sound Objects: A shared
musical space on the web. 2002. IEEE Computer Society Press.
Proceedings of International Conference on Web Delivering of
Music (WEDELMUSIC 2002) - Darmstadt, Germany
[2] Barbosa, A., Kaltenbrunner, M. and Geiger, G. Interface Decoupled
Applications for Geographically Displaced Collaboration in Music.
2003. Proceedings of the International Computer Music Conference
(ICMC2003)
[3] Barbosa, A., Cardoso, J. and Geiger, G. Network Latency Adaptive
Tempo in the Public Sound Objects System. 2005. Proceedings the
International Conference on New Interfaces for Musical Expression
(NIME 2005); Vancouver, Canada.
[4] Puckette, M. Pure Data. 269-272. 1996a. International Computer
Music Association. Proceedings of the International Computer
Music Conference, San Francisco (ICMC96)
[5] Yamaha Disklavier Piano:
http://www.yamaha.co.jp/english/product/piano/product/europe/dl/dl
.html (Cunsulted 2008/01/30)
[6] ETHERSOUND:
http://www.ethersound.com/ (Cunsulted 2008/01/30)
[7] Smuts, J. Holism and Evolution. 1926. Macmillan, London UK
[8] Schaeffer, P., Traité des Objets Musicaux., Le Seuil, Paris, 1966
[9] Amatriain, X. and Herrera, P. Transmitting Audio Content as Sound
Objects. 15-6-2002. Proceedings of the AES22 International
Conference on Virtual, Synthetic and Entertainment Audio
[10] Stelkens, J. peerSynth: A P2P Multi-User Software with new
techniques for integrating latency in real time collaboration. 2003.
Proceedings of the International Computer Music Conference
(ICMC2003)
[11] Obu, Y., Kato, T. and Yonekura, T. M.A.S.: A Protocol for a
Musical Session in a Sound Field Where Synchronization between
Musical Notes is no garanteed. 2003. International Computer Music
Association. Proceedings of the International Computer Music
Conference (ICMC2003), Singapore
[12] Freeman, J. The Networked Music Workshop at ICMC 2005,
Barcelona (September 4, 2005)
[13] Föllmer, G. 2002 Making Music on the Net, social and aesthetic
structures in participative music. Ph.D. Thesis, Martin Luther
Universität Halle-Wittenberg – Germany
[14] Schuett, N. 2002 The Effects of Latency on Ensemble Performance.
Ph.D. Thesis, Stanford University, California – USA
[15] Stelkens, J. 2003 Network Synthesizer. Ph.D. Thesis, Ludwig
Maximilians Universität, München – Germany
[16] Weinberg, G. 2003 Interconnected Musical Networks – Bringing
Expression and Thoughtfulness to Collaborative Music Making.
Ph.D. Thesis, Massachusetts Institute of Technology, Massachusetts
– USA
[17] Barbosa, A. 2006 Displaced Soundscapes: Computer Supported
Cooperative Work for Music Applications. Ph.D. Thesis, Pompeu
Fabra University, Barcelona – Spain
[18] Sergi Jordà, S. 1999 Faust Music On Line (FMOL): An approach to
Real-time Collective Composition on the Internet, Leonardo Music
Journal, Volume 9, pp.5-12
[19] Tanzi, D. 2001 Observations about Music and Decentralized
Environments, Leonardo Music Journal, Volume 34, Issue 5, pp.431-
436
[20] Barbosa, A. 2003 Displaced Soundscapes: A Survey of Network
Systems for Music and Sonic Art Creation, Leonardo Music Journal,
Volume 13, Issue 1, pp.53-59
[21] Weinberg, G. 2005 Interconnected Musical Networks: Toward a
Theoretical Framework, Computer Music Journal, Vol. 29, Issue 2,
pp.23-29
[22] Traub, P. 2005 Sounding the Net: Recent Sonic Works for the
Internet and Computer Networks, Contemporary Music Review, Vol.
24, No. 6, December 2005, pp. 459 – 481
[23] Landy, L. 2005 Organised Sound 10 (Issue 3), Cambridge University
Press, U.K. (OS: ISSN: 1355-7718)
[24] Wright, M. and Freed, A. 1997 Open Sound Control: A New
Protocol for Communicating with Sound Synthesizers, proceedings
of the International Computer Music Conference
[25] Nella, M. J. Constraint Satisfaction and Debugging for
Interactive User Interfaces. Ph.D. Thesis, University of
Washington, Seattle, WA, 1994.