Liquid Interface - A Malleable, Transient, Direct-Touch
Interface

Jeffrey Tzu Kwan Valino Koh^

Mili John Tharakan*

*Keio-NUS CUTE Center, National
University of Singapore
Kasun Karunanayaka*

Manoj Krishnan*

IDM Institute, 21 Heng Mui Keng
Terrace, #02-02-09, Singapore
119613
(+65) 65167514

jeffrey@mixedrealitylab.org
Jose Sepulveda*

Adrian David Cheok^*

^NGS, National University of
Singapore







ABSTRACT
We present a new interface based on ferromagnetic fluids in
which the user can have direct interaction (input/output) through a
tangible and malleable interface. Liquid Interfaces uses the
physical qualities of ferromagnetic fluids in combination with
capacitive, multi-touch technology to produce a 3D, multi-touch
interface where actuation, representation and self-configuration
occur through malleable, ferromagnetic fluid. This, combined
with the ability to produce sound, enables users to create musical
sculptures that can be morphed in real time by interacting directly
with the fluid
Categories and Subject Descriptors
H.5.2 [Information Interfaces and Presentation]: User
Interfaces— Input devices and strategies
General Terms
Measurement, Documentation, Design, Experimentation.
Keywords
Haptics, liquid, multi-touch, tangible user interfaces, self-
configurable, interaction.
1. INTRODUCTION
For the last two decades, designers, scientists, and engineers have
researched new interaction methods and technologies to take
virtual reality beyond the digital display [6]. These efforts have
produced solutions such as augmented reality [1] and tangible
user interfaces (TUI's) [5]. Ubiquitous robotics and
nanotechnology have also provided inspiration for research into
the next logical step in TUI's [4], where tangible, self-configuring
display interfaces can occur in the third dimension, beyond the
confines of a two-dimensional surface.
Liquid Interface allows users to experience not only a tangible and
multi-touch interface hybrid, but also offers a truly physical yet
animated feedback system. Three-dimensional animations that
occur on the surface of the system are not virtual or augmented.
They are real-world and actual morphing objects.
2. RELATED WORK
Research for this project inherits ideas from many other
implementations. Hiroshi Ishii's seminal work (Tangible Bits) [2]
provides a solid basis for tangible interaction on which the liquid
interfaces project expands and builds upon. Ishii's research
describes the physical and cyberspace components of modern life,
and attempts to reduce the gap between these two components by
developing new tangible user interfaces (TUI). Tangible bits use
common objects and transform them into interactive interfaces to
cyberspace by projecting information on to them. The goal of
liquid interfaces is not only to use a common object as a platform
for projection, but to have the object interact with the user.
The ferromagnetic art installations by Sachiko Kodama [3]
provide an aesthetic viewpoint of how the power of fluid and
transient shapes can capture the imagination of viewers. In this
project ferromagnetic fluids are used to create organic shapes that
change structure dynamically. Ferromagnetic fluid is actuated by
adjusting the power of the electromagnets. The magnetic field
produced by the electromagnets controls the movement of the
ferrofluid, producing a visual output for music, light, or human
communication. Liquid interfaces builds on this work, providing
the user with a means to directly interact with the ferromagnetic
fluid in the formation of embodiments, while inheriting the

Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee.
ACE '10, 17-NOV-2010, Taipei, Taiwan
Copyright © 2010 ACM 978-1-60558-863-6/10/11…$10.00

magical and kinetic qualities that Kodama's pioneering works
offer.
3. SYSTEM DESCRIPTION
The electromagnet control system is responsible for dynamically
actuating the ferrofluid in the liquid display. This actuation is
dependent on the pattern selected by the controller circuit and the
multi-touch input of the user.

Figure 1: The Liquid Interface system.
The electromagnetic field collusion is calculated by the controller
firmware and PC software before displaying the changes of the
patterns caused by the user input in the ferrofluid display. Multi-
touch input relies on optical touch detection using infrared (IR).
We use a multi-touch overlay which can detect 6 true touch points
with up to 32 touch points simultaneously. This makes it possible
to use the system for collaborative interaction where a number of
people can interact with the system in tandem.
The ferrofluid display is housed in an acrylic container with
2mm's of thickness and contains 2mm of ferrofluid. We used
EHF1-Type ferrofluid for the display. The ferrofluid makes
different patterns with variable dimensions due to the fluctuating
magnetic field created by the electromagnet grid placed within the
ferrofluid display.
3.1 Hardware Design
The systems electronics are divided into three main subsystems:
The ATmega circuit, the magnet driver circuit and the power
regulation circuit (current limiter circuit).
Two header boards of ATmega 2560 microcontrollers are used
and support 16 independent Pulse Width Modulation (PWM)
channels that can be programmed individually. The Magnet driver
circuit consists of 20 H-bridge drivers which can operate PWM
frequencies up to 10 KHz and deliver a current of 5 Amperes,
constantly. The Power regulation circuit consists of 5 Amp
adjustable regulators.
3.2 Software Design
Liquid TUI Framework consists of a system background service
with a Java API (Application Programming Interface) extension.
The Liquid Interface system could be configured as a tangible
input device, tangible display, or perform both operations to the
external system simultaneously. Any kind of external system
could communicate with the Liquid TUI through the Liquid TUI
framework API.
Furthermore, the framework provides communication to the
external application connected to the system. In this application
we attached it to our own instrumental music generation program
developed in the MAX/MSP environment.
The circuit firmware is written in C. It was programmed into
ATMEGA 2560 micro-controllers within the circuit. One micro-
controller acts as the master micro-controller for the system. The
master micro-controller passes the information of the ferrofluid
patterns received from the PC to the slave micro-controller.
3.3 Transient Response of the System
The transient state of the system is measured by recording a video
using a high definition camera and measuring the delay between
the time of the input and time of the maximum level of actuation.
The results of our first experiment are used to take 2.3 A of
current and measured the time until it reach 14.20 mm height.

Figure 2: Transient response of the system.
The results of the experiment are illustrated in Figure 2. The total
time it takes to reach the steady state is 1.7 seconds. The touch
input, software framework, and micro controller takes 0.4 second
to detect and process the signal and it take another 1.2 seconds to
actuate the ferrofluid. This delay may be due to the time it takes to
magnetize the core of the electromagnet.
3.4 Steady State Response of the System
Our second experiment, the steady state test, was executed using
the results obtained in the first experiment. We used results from
the first experiment to control the ferrofluid spikes in the system.
The purpose of the experiment was to check the linearity response
of the system.
We selected 10 different spikes heights from the first experiment
and tried to recreate them using the system and measured the
responsive spikes height. A digital Vernier caliper and a digital
camera were used to take the readings. The achieved results were
plotted in a graph as illustrated in Figure 8. The resulting graph
was linear and suggests that the system is linearly controllable.
Since magnetic systems are highly nonlinear, this represents a
significant step in controlling ferrofluids for our purposes.

Figure 3: Static linearity of the system.
4. USER INTERFACE DESIGN AND
INTERACTION
When designing the first liquid interface device, functional and
aesthetic considerations were weighed equally. Since ferrofluid is
used, the first liquid interface device is designed to look like a
pool of water in the form of a scrying pool.

Figure 4: Initial state of interaction.
Figure 4 shows the Liquid Interface device in its initial state,
before actuation by the user. Emanating from the 0,0 coordinate of
the pool of ferrofluid are ripples that glide across the surface, as if
drops of water are hitting the center of the liquid. These ripples
are looped and timed to radiate in a rhythmic pattern, which is
representational of a "timeline" scanning the surface.
Figure 5 shows the user's initial interaction by touching the
surface of the liquid to produce "ferrofluid spikes" such as the
ones produced when ferromagnetic fluid is held near a magnetic
field.

Figure 5: The user’s initial actuation.
Figure 6 shows what happens when the rhythmical, center-
radiating ripple/timeline hits ferrofluid spikes. When a ripple hits
the ferrofluid spikes, it implodes and a sound is produced. Once
the radiating ripple passes the area where the spikes once existed,
the ferrofluid spikes form again automatically, waiting to be hit by
another ripple, thus producing a loop, which is synonymous to the
creation of electronic music.

Figure 6: The user’s initial actuation.
Figure 7 shows our real and fully functioning system, with key
elements of our interaction technique implemented. The sequence
displayed narrates actuation to output.

Figure 7: Sequence of events from actuation to output.

5. FUTURE WORK
We are working on developing a magnetic sensing input
mechanism for the project using magnetic field distortion of the
nearby fields. Hall effect will be used as the sensing devices and
they will be embedded into wearable gloves or worn on the finger
tips. This will added another dimension to our existing 2D multi-
touch input system so that users will be able to more viscerally
sculpt, manipulate and form ferromagnetic fluids as the system
will be able to sense in three dimensions. The proposed magnetic
distortion sensing mechanism is illustrated in figure 8.

Figure 8: Simulation of proposed electromagnetic field
disruption sensing input.
Along with the above-mentioned technical goals, there is an effort
to create a concise library of gestures that are more intuitive for
use, with the intention to explore a first application using Liquid
Interface as a tangible music and sculpture creation device. Once
an initial gestural library is created and a general-use prototype is
achieved, the project will focus on studies with musicians and
artists in order to fine-tune the usability of the system for real-
time performance and creative content development.
6. CONCLUSION
In this paper we have presented an implementation of
ferromagnetic controllability for use in the context of artistic
expression, content creation, collaborative performance and
physical human-computer interaction.
As demonstrated with our system, it is fully possible to have a
tangible user interface based on a morphable and quasi-intangible
material. We have also shown that by linearly regulating the
electromagnetic response of our system, we are able to control the
electromagnetic field in order to reproduce a steady and transient
state for ferromagnetic fluid.
It is our hopes that researchers of tangible and haptic interaction
consider the exponential qualities of morphable materials. The
Liquid Interface project is an attempt to bring to light the gravity
of such a potential avenue for experimentation and exploration.
We believe that there are too few forms of tangible computing
that offer the means to directly manipulate and animate three-
dimensional shapes in such a way as we have demonstrated.
Opposed to tangible objects, the uncommon and subtle material
used in the Liquid Interface project shows that physical
computing does not need to be solid to be sensitive. It is
specifically this quality found in fluids that offers us an intimate
and temporal relationship with the objects around us, and makes
apparent the intangibility of human imagination and creativity
mirrored by the shapes in flux formed by the Liquid Interface
system.
7. ACKNOWLEDGMENTS
Development of this project would not have been possible without
the invaluable assistance from the following people: Hassan
Shafeeq, Johan Ling Wei, Eishem Bilal Naik, Liu Yijiang and
Jeremy Heng Zhi Wei.
This research is carried out under CUTE Project No. WBS R-
7050000-100-279 partially funded by a grant from the National
Research Foundation (NRF) administered by the Media
Development Authority (MDA) of Singapore.
8. REFERENCES
[1] Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S.
and MacIntyre, B. 2001. Recent Advances in Augmented
Reality. IEEE Computer Graphics and Applications. 21, 6 ,
34 - 47.
[2] Ishii, H. and Ullmer, B. 1997, Tangible Bits: Towards
Seamless Interfaces between People, Bits and Atoms. In
Proceedings of CHI '97 on Human Factors in Computing
Systems. ACM, Atlanta, (March 1997), pp. 234 - 241.
[3] Kodama, S. 2008. Dynamic ferrofluid sculpture: organic
shape-changing art forms. Commun. ACM 51, 6 (Jun. 2008),
79-81.
[4] Poupyrev, I., Nashida, T. and Okabe, M. Feb. 2007.
Actuation and Tangible User Interfaces: the Vancanson duck,
robots and shape displays. In Proceedings of the 1st
International conference on Tangible and Embedded
Interactions (Banton Rouge, Louisiana, Feb. 15 - 17) . ACM,
(Feb. 2007) pp. 205 - 212.
[5] Ullmer, B. and Ishii, H. 1997. The metaDESK: models and
prototypes for tangible user interface. In Proceedings of the
10th annual ACM symposium on User Interface Software
and Technology (Banff, Alberta, Canada). ACM, New York,
, pp. 223-232.
[6] Weiser, M. 1999. The Computer for the 21st Century,
SIGMOBILE Mob. Comput. Commun. Rev., 3(3):3-11, 1999.