DYNAMIC CUES FOR NETWORK MUSIC INTER-ACTIONS
Alain B. Renaud
Bournemouth University, England arenaud@bournemouth.ac.uk
ABSTRACT This paper provides an overview of a cueing system, the Master Cue Generator (MCG) used to trigger performers (humans or computers) over an IP-based network. The performers are scattered in several locations and receive cues to help them interact musically over the net-work. The paper proposes a classification of cues that dynamically evolve and reshape as the performance takes place. This begets the explo-ration of various issues such as how to represent and port a hierarchy of control over a net-worked music performance (NMP) and also takes into account parameters inherent to a net-work such as latency and distance. This ap-proach is based on several years of practice-led research in the field of NMP, a discipline that is gaining grounds within the music technology community both as a practice and through the development of tools and strategies for interact-ing over disparate locations. 1. INTRODUCTION Performing in real time over high-speed net-works is a now well-accepted paradigm and has become an integral part of Telematic Art, con-sidered by authors such as Roy Ascott, “as an artistic medium in itself” [1]. There have been several attempts to achieve interactive telematic performances. However the first telematic con-cert, using high-speed research networks with no audio compression and thus allowing CD like audio quality took place between two spaces at Stanford University in 2000, with an ensemble split and performing about one kilo-meter apart [3]. This initial test was led by the Sound Waves over the Internet from Real-time Echoes (SoundWIRE) project founded in 1998 at Stanford University [3]. SoundWIRE,
through various experiments and studies such as the “clapping experiment” [4], which measured a threshold in milliseconds for ensemble accu-racy, set the grounds for further development in the field of networked music performance (NMP). The discipline of NMP branched out in many directions, and due to its nature, which involves being distributed, led to the involve-ment of several new participants. The work of SoundWIRE, however, demonstrated that it was possible to interact musically over a long dis-tance despite the inherent latency of the net-work. The excitement of being able to play apart led to the challenge of choosing whether music performed over a network could be sim-ply improvised or formally structured through the help of network-centric cueing mechanisms. 2. IMPROVISATION The network provides a platform for sharing synchronization information and cues as it al-lows several performers to share a common infrastructure for exchanging common musical structures. Performing over the network intro-duces the principle of dislocation of performers as they are not in the same space but are playing in real time together whilst being located in several spaces. Free improvisation has been a practice often employed in NMP due to its em-phasis on musician-to-musician interaction and flexibility of materials; thus providing a good basis for developing musical strategies for in-teracting over a network regardless of its la-tency. Free improvisation as outlined by Derek Bailey, “pre-dates any other music – mankind’s first musical performance could not have been anything other than a free improvisation” [2]. It is therefore not surprising that new media envi-ronments, such as NMPs, resort to basic sorts of

musical forms, which do not involve a formal structure. NMP is such a recent practice that most performances will start from an empty shell, where the infrastructure will first be put into place followed by numerous tests to make sure that the communication works and final-ised by a short rehearsal. The fact that the IP –based network is the medium that interconnects them will play a crucial role in the development of those social interactions through space and time. In this context, and based on several years of research in the field through large scale NMPs such as the Disparate Bodies series [7] as well as the experimentations of the Net. Vs. Net Collective [8], the development of coherent network centric cueing strategies was needed. It is due to the fact that, very quickly the im-provized performance requires some sort of formalization so that performers can be cued over the network, leading to the inclusion of a basic structure within the improvisation. In this context and as a result of the practice in the field, a formal classification of networked cues and how they can be used to interact musically over the network made sense. 3. CUES 3.1 Rationale In order to provide an easy way to represent various cue information over the network, an integrated cueing system called the Master Cue Generator (MCG) was developed. The MCG aims to provide a rough standard to distribute cues over the network. The MCG has been used and tested in several NMPs such as the Dispa-rate Bodies series [7] and with the Net. Vs. Net Collective [8]. The system is continuously be-ing developed further with the goal to achieve a common cueing structured language for net-worked music improvisation. The MCG was built with Max/Msp [6] and is able to send cues to a multitude of locations as standard OpenSoundControl (OSC) [9] mes-sages, meaning that any OSC compliant appli-cation is able to receive the cues and converse back to the MCG should a direct feedback be necessary. The MCG was initially designed to
function based on a client/server architecture. However, it was later discovered that modifica-tions of the network configuration should be possible based on the changing attribution of roles, defined as who plays the role of the MCG in the network. This complex and challenging aspect is currently being developed. Currently, the MCG broadcasts important musi-cal information by providing a basic structure to the nodes playing over the network, such as which section of the piece the nodes are in, as well as warning messages that the piece is about to switch to another section. The types of cues and their specific nature can be customized de-pending on the artistic approach given to the piece.
Figure 1. The MCG engine 3.2 Types of Cues There are three types of cues that have been so far identified as part of the classification: tem-poral behavioural and notational. All the cues below have been developed based on the prac-tice in the field and the classification is con-stantly being updated as the practice progresses.
3.2.1 Temporal Cues Temporal cues are sent out as information from the server to the nodes and are related to timing. Examples include the length of a cue, a warning that the cue is about to finish, or how much time a given node is in control of the improvisation until the given node delegates its control to an-other node and thus conceptually modifying the topology of the network. They are the most im-portant types of cues, in order to keep the en-semble together and, thereby, can be synchro-nized with various audio triggering cues, which are indicators that the structured improvisation is about to change from one section to another.
3.2.2 Behavioral Cues Behavioral cues are cues that are sent with a certain scenario attached to them. This can, for example, include the triggering of a waveform, or the suggestion that a given node needs to

play certain nodes only above the note C4. Be-havioural cues can also trigger physical ele-ments in a remote space such as the ringing of a distant bell or the triggering of an analogue syn-thesizer. Behavioural cues are more complex types of cues. They allow the broadcast of mes-sages that will have an influence on the actual audio content of the piece being performed over the network. Behavioural cues are often part of the process of a structured network improvisa-tion. An example is the triggering of pre-recorded waveforms that reside on remote com-puters. The waveforms are being played back remotely but triggered by the MCG. Behavioural cues also have the potential to influence the actual frequency content of each remote node by broadcasting messages that will interpolate or cut-off. The MCG, in this case, provides the intelligence behind the system by ensuring that each node has a different frequency bandwidth. The frequency dependent interconnections that are being created in this case, also allow for a morphing of frequency range distributions across the nodes. For example, a node can de-cide to borrow a frequency range from another node, in which case the frequency distribution is swapped between nodes. Amplitude control is also an important type of behavioral cue. As part of the pre-defined structured improvisation, a distribution of amplitudes across the nodes can be implemented in advance. It allows the distribution of intensities across the network and is a very democratically aware way of ensuring that each node can be properly identified during a performance. For example, in the case of a performance between three nodes, the MCG will make sure that in Section 1 of a piece, Node 1 is the loudest, while Node 2 is the quietest and Node 3 is at mid-level. As a result, reasonably complex interdependencies can be achieved by swapping loudness informa-tion between nodes as well as interpolating them. They are, of course, many other interac-tion cues that can be created and their types and resulting actions wholly depend on desired ob-jectives in the design of the performance.
3.2.3 Notational Cues Notational cues are able to display content that can be identified by the performers as being helpful in the good running of the performance. This can include the visualisation of the wave-forms from each site, the display of the cue number, a countdown or dynamic shapes that can be activated by various factors in the per-formance. Transmitting concrete or abstract notation based information is a real challenge over a network due to the latency and of the different distances between the MCG and the nodes. Even though the MCG is capable of re-triggering events so that a cue information ar-rives simultaneously at all nodes, which is a punctual or periodical type of information, the triggering mechanism does not work well for continuous information and, thereby, a drift is likely to occur over a period of time. Therefore, notational cues have traditionally been sent over the network in a punctual fashion, where the graphical representation is analogous to a slide show. If some events are of continuous nature, they tend to be transferred to remote nodes be-fore the performance and triggered remotely. In order to efficiently represent the cues and the synchronicity information, a set of visualisation tools has been developed to simply, but effi-ciently, display score information on various sites.
3.2.4 Active/Passive Cues The three types of cues identified above can have two distinct modes of operations: Passive: the cues are only sent as a suggestion to the nodes. Each node can decide whether or not to follow the guidelines suggested by the MCG. One particular example includes a sug-gestion that one node should decrease its gen-eral amplitude while another node should stay steady or a flashing warning indicating that the improvisation is about to switch to another sec-tion. Passive cues are generally rendered graphically as part of the score of the structured improvisation so that performers can take the suggestions (or obligations) into account while performing in remote sites.

Active: the cues are actively trigger-ing/processing a concrete element on a distant node. This includes, amongst others, the open-ing of a filter or the interpolation of its center frequency, the reduction or augmentation of the amplitude of a distant node or the activation of a remote oscillator. Another aspect of active cueing that is currently being explored is not only the triggering of events from the MCG, but also the triggering of events from node to node. This possibility adds to the complexity of dis-tributed cues and permits the building of com-plex patterns and interdependencies that use the network to create them. A cue can be both passive and active simulta-neously. One example would be the triggering of a sample along with the visual indication that a sample is about to be triggered. This intro-duces both an automated musical event (the sample) and an indication to a human performer that an event is about to occur, hence, suggest-ing a reaction of some sort.
Figure 2. Cue types and corresponding examples
4. CHANGING TOPOLOGIES The MCG should not be location centric by always being located in the same physical space, but should be able to take over a specific node at a given time. This leads to a far greater flexibility of the network topology as the MCG can virtually travel between nodes and position itself at any point on the network. The option to change topologies means that, an NMP can start as a basic star network topology with one node being at the center of the net-work. In this case, the chosen node is not only at the center of the network but also takes the role of a leader in the performance. At any time the controlling node can transfer its powers to another node on the network. The move can happen when switching from a cue to another in the piece or it can be randomly attributed based on the distribution of roles and voting by other participants or the audience. Many permutations are possible, which lead ultimately to a change in the network topology. For example, in the case of a network with four nodes, called A, B, C and D respectively, if node C takes the lead, all the commands from the MCG will be issued by node C until the next change in topology. This series of permutations, as a network im-provisation takes place, is analogous to the mo-dus operandi of a musical improvisation that would happen on a real stage in terms of dele-gating control to a performer over others. This approach adds various levels of interplays be-tween dislocated performers and leads to the creation of music that uses the network archi-tecture as the core and to a certain extent as the score. The concept of changing topologies outlined above allows the creation of complex interde-pendencies over the network (local or wide-area) and can easily implement some of the earlier network topology principles for musical collaboration such as the ones illustrated by Weinberg [11]. The MCG to node relationships allows the implementation of a, “Process Cen-tered Musical Network” [12]. The flexibility brought by the MCG in terms of network topol-

ogy for the improvisation of musical content brings an additional layer of structure that is not necessarily musical, per se, but allows an allo-cation of performative roles despite the distance and the absence of physical contacts between the performers. This concept also ties up well with Weinberg’s principles especially when the notion of Goal Oriented Interaction is men-tioned [12]. The latter introduces two separate principles of interactions: Collaboration and Competition. The MCG along with structured improvisation allows the ensemble to morph between the no-tion of collaboration and the notion of competi-tion, in musical terms, which creates diametri-cally opposed musical forms that are created by the changes in network topology.
Figure 3. Changing relationship between nodes and MCG 5. THE ISSUE OF LATENCY Latency is a pretty common term in the field of computer music and is defined as, “the delay between the stimulus and the response” [13]. In a more musical fashion and when parallelised with the speed of sound in air, latency can be defined as, “the speed of sound through com-puter algorithms” [10]. In the context of NMP, latency is often considered as a musical feature in its own right and, “can be used as a specific compositional tool” [13]. It needs to be high-lighted that regardless of the quality and band-width of the networks used for NMPs the dis-tance between two nodes will introduce a cer-tain amount of latency. Even data traveling over fiber optic networks will be subject to a certain latency, not only because it cannot travel faster than the speed of light but also because that data will go through several switches and hubs along
the way, introducing conversions and thus slow-ing down its delivery to destination. The MCG includes two approaches to latency, which are defined as synchronous interactions and asyn-chronous interactions. 5.1 Synchronous interactions In this case the relationship between the MCG and the nodes is calculated in terms of time lag. For example, if the relationship from the MCG to node A is 100 milliseconds and the relation-ship from the MCG to node B is 75 millisec-onds, an additional 25 milliseconds will be added to the relationship between the MCG and node B so that cues arrive at exactly the same time at node A and C. Since the networks used in this case are very stable, the timing is very likely to stay firm through the piece. 5.2 Asynchronous interactions In this case, the MCG ignores the latency values between the MCG and the nodes and deals with the network as it is, leading to the generation of rhythmical patterns created by the network it-self. 6. CONCLUSION AND FUTURE WORK As illustrated through this paper, the MCG is the outcome of several years of practice in the field of NMP to answer the growing needs for distributed cueing structures. This ever-evolving exercise is an attempt to formalize some sort of convention in the practice of NMP and will be developed further as the practice progresses over time. In the short to medium term, the goals regarding the development of the MCG and associated cueing strategies are: - To develop a proper cross-platform ap-plication so that the system can be em-braced by a wider community - To offer a web platform on which a set of standard messages fitting in the cues classification outlined in this paper can be implemented and formalized by the NMP community - To advertise in a more formal manner, mostly through online channels, applica-
148
relationships allows the implementation of a, “Process Centered Musical Network” (Weinberg 2005). The flexibility brought by the MCG in terms of network topology for the improvisation of musical content brings an additional layer of structure that is not necessarily musical, per se, but allows an allocation of performative roles despite the distance and the absence of physical contacts between the performers. This concept also ties up well with Weinberg’s principles especially when the notion of Goal Oriented Interaction is mentioned (Weinberg 2005). The latter introduces, according to Weinberg, two separate principles of interactions: Collaboration and Competition. The MCG along with structured improvisation allows the ensemble to morph between the notion of collaboration and the notion of competition, in musical terms, which creates diametrically opposed musical forms that are created by a change in network topology.
Figure 19: Reconfiguration of the MCG and the changing relationship to nodes

tions and event in which the MCG is be-ing used and can be further developed. - To make the MCG freely available on-line to the NMP community. 7. REFERENCES [1] Ascott, R.: Telematic Embrace: Visionary Theories of Art, Technology, and Consciousness. 1 edn. University of California Press, 2004. [2] Bailey, D., 1993. Improvisation: its nature and practice in music. New York : Da Capo Press, 1993. [3] Chafe, C., Wilson, S., Leistikow, R., Chisholm, D. and Savone, G., 2000. “A Simplified Approach to High Quality Music and Sound Over IP.” Proc. COSTG6 Conference on Digital Audio Effects (DAFx-00), Verona, 2000. [4] Chafe, C., Gurevich, M., Leslie, G. and Tyan, S., 2004. “Effect of time delay on ensemble accuracy”. Proc. 2004 AES 117th Conf., San Francisco, 2004. [5] Chafe, C., Wilson, S. and Walling, D., 2002. “Physical Model Synthesis with Application to Internet Acoustics.” Proc. 2002 Intl. Conference on Acoustics, Speech and Signal Processing, Orlando, 2002 [6] Cycling ’74. Available: http://www.cycling74.com [04/01/2008, 2008]. [7] Disparate Bodies. Available: http://www.sarc.qub.ac.uk/pages/db/ [04/20/2010, 2010]. [8] Net Vs. Net. Available: http://www.netvsnet.com [04/20/2010, 2010]. [9] OpenSoundControl. Available: http://opensoundcontrol.org/ [04/17/2010, 2010]. [10] Tanaka, A., 2000. “Speed of Sound. In: J. BROUWER, ed, Machine Times”. NAI Publishers, V2_Organisation.
[11] Weinberg, G., 2002. “The Aesthetics, History, and Future Challenges of Interconnected Music Networks”, Proc. 2002 International Computer Music Conference, Göteborg, 2002. [12] Weinberg, G., 2005. “Interconnected Musical Networks: Toward a Theoretical Framework.” Computer Music Journal - Volume 29, Issue 2, pp. 23-29. [13] Will, U.K., 2004. “Computer music modeling and retrieval”, International Symposium, CMMR 2003, Montpellier, France, May 26-27, 2003: revised papers. Berlin ; Springer, c2004.