Capturing Water and Sound Waves to

Interact with Virtual Nature

Demo ISMAR 2003

Marissa Díaz, Eduardo Hernández, Leonardo Escalona, Isaac Rudomin, Daniel Rivera
ITESM – CEM Department of Computer Science
00443059@academ01.cem.itesm.mx,00716539@academ01.ccm.itesm.mx,
00472062@academ01.cem.itesm.mx, rudomin@itesm.mx, transfaces@hotmail.com

Abstract interfaces as shown by Yonezawa [3] and Marti[4] has

To improve the interaction of people with virtually already been proven as a succesful interface.
generated environments we need first to break the
barriers that prevent the user from getting an experience 2.Description of the demo
as close to reality as possible. The main problem is to
give the user the sensation that his/her presence affects The demo has the following items involved as described
the virtual world and then to let the user perceive that on image 1:
the actions he/she takes on the real world can change the
virtual one in a smooth natural way in order to achieve 1) A computer terminal (PC)
to render the virtual world.
virtual biofeedback. 2) Viewing media, it can be
shown on the PC´s monitor
Interacting with virtual nature can transport us out from or it can be displayed on a
wall by a small projector.
reality. We developed an interactive application with two 3) The interaction hardware
interfaces on which the user can pretend to be the wind that can be confined in a
and interact with a virtual pond. Our application makes 6inch X 4inch box.
the user believe and feel that he/she is modifying a 3D 4) A hanging water wave
sensor.
virtual pond by the interaction with a small water 5) The pedestal to hang the
receptacle in the real world using a innovative wave- water sensor it can be
sensing device. The user, by speaking to a microphone, adjusted from 5inch to 1/2ft.
6) Water container on which
can also interact with a virtual tree by making it move the user can move the water
according to his/her wishes. The physics for the tree are to interact with the Virtual
calculated to present the user’s action as a wind force Pond.
7) Voice-pressure-sensing
making this an entertaining experience. device.

Keywords: human-computer interaction, virtual reality

hardware, immersive systems, ambient media, tangible
user interface.
1.Motivation
Natural interfaces development is one of the most
important goals in virtual reality applications. As stated
by Ishii[1], water and air can be considered ambient media
to be used as a reinforcing element for interaction [2],.
Using tangible media, as he suggests, can give an
interface the ability to reproduce an environment as seen
by the brain of the user and give him/her the idea of
being immersed into the virtual environment. To feel and
see the changes in the virtual world ensures that
biofeedback is achieved. Using natural elements to build
2.1.1.How to interact with the Pond.
The virtual world we propose is a simple world on witch
the user, on a first person perspective, can walk around a
virtual city. The city has a small museum and a park.
In the virtual park there is a pond. This pond can be
altered by moving a virtual ball into it using the keyboard
but also is altered by modifying the behavior of the water
on a real receptacle. The users by moving the water in the
receptacle have a real time response in the virtual pond.
The main contribution of this part of the demo is to give
biofeedback to the user by using real water giving the
feeling of being near/at the virtual pond.

Proceedings of the Second IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR ’03)
0-7695-2006-5/03 $17.00 © 2003 IEEE
 The user can also interact with the virtual weather and
choose if it is snowing or not. When it is snowing the
pond begins to freeze.
figure 2: The virtual pond.
2.1.2. How to interact with the tree.
In the other part of the application there is a lonely tree.
To make it happy we need to fill its world with some
action, and to do so the user can interact with this tree by
playing the role of the wind.
In this virtual world the normal physical rules are applied
but also the rules that the user commands by his/her
voice are followed.
The main contribution of this other part of the
demonstration is to illustrate an easy, natural way to
interact with a virtual element at a higher conceptual
level. One can make the user believe that he/she can do
extraordinary things and that he/she really has power
over the virtual world.
figure 3: Virtual tree moving
2.1.3.How does it work
A wave sensor attached above the real water receptacle
governs the physics of the virtual water in the pond. This
wave sensor is a differential sensing device that responds
uniquely to the alterations on the water surface and that
returns a frequency that is proportional to the wave’s
frequency in the water.
To feel the water and see the instantaneous reaction on
the projected virtual world gives the user a better
sensation of immersion. The movement of the tree is
Proceedings of the Second IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR ’03)
0-7695-2006-5/03 $17.00 © 2003 IEEE
calculated by adding to the wind force the noise created
by a frequency generated by a circuit that relates the
duration of the speech its force to the output frequency.
3. Description of the videos
Video 1 illustrates a user interacting with the virtual
pond. To the right the user is altering the water with her
hand to make the waves that will be rendered in the
virtual world. In the middle one can see the hardware
that controls the data fl owing from the sensing device to
the PC serial port. To the left is a laptop PC rendering
the virtual world on witch one can see the virtual pond’s
water moving.
Video 2 gives a close up view of the items described on
video 1 showing the quality of the render and the
interaction with the water more clearly.
Video 3 shows the microphone, the transformation device
and the simulated tree render. The user blows like the
wind and the tree’s reaction can be seen.
4.Future work
It is important to continue developing applications like
the ones described in this document, which let the user
interact with virtual worlds and receive appropriate
biofeedback from them. To do so, it is essential to create
modules for people to interact with these worlds by
means of the internet and by using economic and friendly
interfaces.
5.Bibliography
[1] Ishii, Hiroshi. “Tangible Interfaces”, Conference Abstracts
and Applications SIGGRAPH 99. pp 127,1999.
[2] Ishii, Hiroshi, Brygg Ullmer. “Tangible Bits: Towards
Seamless Interfaces between People, Bits and Atoms.”,
Proceedings of CHI 97. ACM. March, 1997.
[3] Yonezawa,T.,Mase, K. “Interaction of Musical Instruments
Using Fluid”. (Also in Japanese) VR Society of Japan 2000.
[4] Marti, Stephan, Deva Seetharam, Hiroshi Ishii.
”WeatherTank: Tangible Interface using Weather Metaphors”.
http://web.media.mit.edu/~nitin/papers/sand.html.
Contact Author:
Marissa Díaz Pier
Phone: +52 58080094, Fax: +52 58080094
E-Mail: al00443059@academ01.cem.itesm.mx
Mailing Address:
Virtual Environments and Robotics Laboratory
ITESM – CEM, Department of Computer Science
Carretera Lago de Guadalupe Km 3.9 Atizapán Estado de
México, México