VISUAL MUSIC: DEVELOPMENT OF AN ART

By Annaliese Micallef Grimaud

2015

1
ABSTRACT

This paper explores the evolution of visual music throughout the years, from
theories on music and colour in the classical antiquity, to today’s contemporary
visual techniques. The various definitions of visual music are discussed, and a
brief summary of how the term came to be used, is given. An overview of this
recurring idea of visual music throughout the different aspects of abstract art is
also considered. A historical timeline of the growth of visual music is presented,
highlighting some of the most renowned pioneers in visual music, their theories
and inventions. The development of the technology involved in the advancement
of visual music and their contributions to music videos, cinematic films and live
audiovisual performances of nowadays, are also researched.

The historical timeline of visual music is divided in several parts – theories of

colour and music, a discussion of the hypotheses on the relationship of colour and
music by Greek theorists; colour organs, an overview of the first colour music
theories which were put into practice; from colour-organs to hand-painted films,
a description of the development of colour-instruments that led to the first hand-
painted films; non-representational films, a detailed account of the evolution of
abstract films; colour in film, recounting the technological development which
introduced coloured films; different techniques of abstract film, an overview of
the processes used by pioneers of non-representational films; synthetic sound
and motion graphics, the birth of the handwritten sound track and the first special
effects ever made. A general overview of today’s visual music is also given, and a
speculation of what the future might hold for visual music is also made.

2
1. INTRODUCTION

This paper explores the development of visual music throughout the years, how
this idea came to be and evolved through various forms of abstract art, from
paintings to music, film and today’s interactive live performances. Before one can
go any further, a clear definition of visual music needs to be established. This
may however prove problematic since the term visual music has been used to
define various graphical structures, from abstract paintings to sculptures and film.
The definition of visual music evolves with time, as the art itself develops with the
birth of new technologies and ideas.

2. DEFINING VISUAL MUSIC

Roger Fry (1866-1934), an English artist and art critic was presumably the first
person to use the term, to describe paintings by Picasso, Matisse, Cézanne and
others in a catalogue of a Post-Impressionist Exhibition held in London in 1912.
Fry explained how the painters in question:

They do not seek to imitate form, but to create form; not to imitate life,
but to find an equivalent for life…In fact, they aim not at illusion but at
reality. The logical extreme of such a method would undoubtedly be the
attempt to give up all resemblance to natural form, and to create a purely
abstract language of form – a visual music. (1920: 157)

Russian abstract painter and art theorist Wassily Kandinsky (1866-1944) was the
leading artist who focused on painting with the intention of stimulating more than
one of the audience’s senses. It is thought that Kandinsky had synaesthesia, a
medical condition whereby stimulation of any one sense triggers an involuntary
response in another sense, through which he experienced seeing colours when
listening to music. He believed that music could be visualized in the form of
abstract art and experimented with this theory in the early 1900s, associating
colours with specific music elements and using musical structures in visual
imagery. Kandinsky was influenced by contemporary composers like Schoenberg
and applied methods they used in their music to his painting (Tyler et al., 2003:
223-226). Fry also described Kandinsky’s improvisatory paintings at an exhibition
in London in 1913 as:

The improvisations become more definite, more logical and more closely
knit in structure, more surprisingly beautiful in their colour oppositions,
more exact in their equilibrium…They are pure visual music. (1927: 17)

Hence in the context of abstract art, visual music can be defined as:

A visual composition that is not done in a linear, time-based manner, but

rather something more static…However…the movement of the painted
elements can and have achieved a kind of visual music, serving as an
artist’s visual interpretation of specific music. (Ox and Keefer, 2008)

Although the term visual music was initially used as a designation for avant-garde
paintings, it also became a denotation for non-representational film. William
Moritz (1941-2004) a film historian specializing in visual music and the works of
pioneer Oskar Fischinger (1900-1967), described visual music as ‘a music for the

3
eye comparable to the effects of sound for the ear’ (Moritz, 1986). Artists like
Oskar Fischinger and Len Lye (1901-1980) experimented with abstract
filmmaking and animation artwork, with the aim of using a non-static medium to
provide movement to visual structures. Lye used a technique called ‘direct
animation’ where images were scratched directly on celluloid and presented one
after the other in a specific order to create the illusion of motion.
With regards to non-representational film, visual music is defined as:

A direct translation of [an] image to sound or music, as images

photographed, drawn or scratched onto a film’s soundtrack [which] are
directly converted to sound when the film is projected. (Ox and Keefer,
2008)

Although music was an important component in Fischinger’s work, he stated that

his ‘films are no illustrations of music’ (1947). Fischinger focused on coordinating
his films with musical compositions, by using the time signature and rhythm of a
musical work as “an architectural ground plan” for his films, but he ‘never tried to
translate sound into visual expressions’ (Fischinger, 1947). Regardless,
Fischinger’s work falls in the visual music category, and this aspect of visual
music can be defined as:

A time based narrative visual structure that is similar to the structure of a

kind or style of music. It is a new composition created visually but as if it
were an aural piece. (Ox and Keefer, 2008)

Although the term visual music first appeared in the 1900s, people before that
time were already interested in the relationship between music (and sound) and
visual imagery. In the catalogue of the Sound & Vision Exhibition held in Frankfurt
in 1993, William Moritz wrote:

The “divine” correlation between the tone scale of music and the rainbow
spectrum of colors intrigued Aristotle and Pythagoras, Leonardo and the
17th-century Jesuit from Fulda, Athanasius Kircher, who shocked and
delighted with his "magic lantern" shows. (1993: 132-145)

Theories about the correlation of colour and music were the first primitive ideas
that hinted at what visual music is known as nowadays. Hence, another aspect of
visual music is colour music. Colour music was produced by colour organs. These
were instruments that displayed specific shades of light depending on what notes
were played (Moritz, 1997).

As the history of this art goes back centuries, one definition cannot be given to
the holistic term visual music. Hence, this paper will give a historical timeline of
the developments of this art, to help understand better how the different times
moulded visual music.

4
3. HISTORICAL TIMELINE

3.1 Theories of colour and music

It seems that the relationship between music and visual imagery has been of
interest for centuries, as theories bringing elements of this relationship together
have existed for a very long time. In 1997 Moritz wrote, ‘Ancient Greek
philosophers, like Aristotle and Pythagoras, speculated that there must be a
correlation between the musical scale and the rainbow spectrum of hues’ (1997).
In his Harmony of Spheres or Musurgia universalis concept, Pythagoras created a
mathematical scale of tones that represented the cosmic order, and also assigned
a particular colour to each one (Hall, 1928: 82).

Sir Isaac Newton, Johann Wolfgang von Goethe1 and others also studied the
relationship between music and colour (Ox and Keefer, 2008), but as Professor of
Art History and Media Theory Dieter Daniels (1957-) writes, ‘a new chapter began
for this long history of color/sound correspondences’ (Daniels et al., 2011: 11)
with the works of the French Jesuit Father Louis Bertrand Castel (1688-1757).

3.2 Colour organs

Father Castel, a popular mathematician and physicist of his time, followed on

Newton’s observations of ‘a correspondence between the proportionate width of
the seven prismatic rays and the string lengths required to produce the musical
scale D,E,F,G,A,B,C’ (Peacock, 1988: 398) and wrote a proposal for a device that
could demonstrate these theories. After being criticized by contemporaries of that
time that a hypothesis had to be proved with a demonstration, he constructed an
Ocular Harpsichord around 1734 (Peacock, 1988: 399). His harpsichord
connected ‘each note on the keyboard with a corresponding flash of colored light’
(Moritz, 1996: 224). This was the first of many colour organs. The model
consisted of a normal harpsichord with coloured-glass panes inserted in its frame.
A curtain hid the glass panes, and as a key was selected, the corresponding
coloured-glass pane would be shown. Castel later built a better machine around
the 1750s, which used myriad candles and reflecting mirrors instead of glass-
panes (Moritz, 1997). This second ocular harpsichord received negative feedback
from critics, as the change in colours was not responding accurately to the played
chords. Although there does not seem to be any evidence that the harpsichord
was true to Castel’s hypothesis, his studies and experiments were the first
stepping-stone in the evolution of other theories that led to the audio-visual
technology of nowadays (Peacock, 1988: 401).

Other “colour organists” referred to this art as colour music. Alexander Wallace
Rimington (1854-1918) built the ‘best-known color instrument of the 19 th
century…in 1893’ (Kim, 2008: 27). Rimington’s model used electricity to display
light, but emitted no sound. Instead, he proposed that an instrument such as a
piano or organ should accompany the colour compositions.

1
For a detailed account of Goethe’s Theory of Colour see
http://enfilade18thc.com/2013/03/08/exhibition-mozart-and-goethe-the-quest-of-tone-colours/
[Accessed 8 January 2015]

5
3.3 From colour-organs to hand-painted films

Many followed in Castel and Rimington’s steps by inventing other mechanical

devices with the aim of projecting visual music. Mary Hallock Greenewalt (1871-
1951), a pianist, constructed a similar instrument that she called Sarabet, after
her mother Sara Tabet. An innovation of Hallock Greenewalt’s was that instead of
a keyboard, the Sarabet had a sliding rheostat that controlled the intensity of the
coloured lights displayed (Peacock, 1988: 404). Hallock Greenewalt believed that
she had created a new type of art, which she called Nourathar – a name
consisting of two Arabic words, meaning “essence of light” (Hallock Greenewalt,
1946: 3). The musician Thomas Wilfred (1889-1968) also invented a type of
colour-organ; one of the most famous colour-instruments of the twentieth
century, called the Clavilux. Unlike Hallock Greenewalt and others prior to him,
Wilfred only focused on the coloured light aspect of former theories, and named
this new art form ‘Lumia’ (Peacock, 1988: 405). His works consisted of a
composition of coloured lights and slowly changing forms. Instead of changing
forms, Hallock Greenewalt hand-painted film rolls with geometric patterns, to be
used during Nourathar concerts.

Apart from film rolls used for concerts, Hallock Greenewalt experimented with
drawing pattern on celluloid. Michael Betancourt (1971-), an art and film
historian, noticed that:

The patterns on these films appear to have been produced with a spray
and templates cut with organic and geometric shapes, and when back-lit
on a light table, they reveal brilliant, saturated colors and rhythmic
organization across the roll as a whole…Hallock-Greenewalt's films are
continuous lengths of color rather than divided into specific "frames."
Unlike the colored film used for tinting light in her original Sarabet patent,
the films are patterned. (Betancourt, 2006a)

Although Hallock Greenewalt’s films are not motion pictures, certain

characteristics of her films are similar to some of the earliest hand-painted films
of futurists Bruno Corra (1892-1976) and Arnaldo Ginna (1890-1982), along with
works of pioneers Oskar Fischinger and Len Lye, which are considered crucial to
today’s visual music history.

3.4 Non-representational films

The development of abstract film was due to artists trying to give movement to
their artwork. Kandinsky and other abstract painters like Paul Klee (1879-1940)
and Arthur Dove (1880-1946) tried to integrate musical structures in their
abstract paintings, but the results lacked one important aspect of music; being a
time-based medium (Brougher et al., 2005: 19). Hence, painters turned to the
medium of film.

Painters Walter Ruttmann (1887-1941), Viking Eggeling (1880-1925) and Hans

Richter (1888-1976) all saw film as the perfect medium for their art (Daniels et
al., 2011: 74). Ruttmann’s first short film Lichtspiel opus 1 (1921) was one of the
first non-representational films of Visual Music’s history (Betancourt, 2006b).
Lichtspiel Opus 1 consisted of a sequence of various single-frame hand-drawn
images, accompanied by music. Eggeling and Richter experimented with “scroll

6
paintings”, which similar to Ruttmann’s film consisted of a continuous sequence of
painted images on paper (Dusan, 2014).

Continuing on Ruttmann’s, Eggeling’s and Richter’s experiments, were Alexander

László (1895-1970) and machine engineer Oskar Fischinger (1900-1967). Làszló
was a pianist and composer, interested in combining light and music together.
Làszló is best known for his Sonchromatoscope, premiered in 1925. This device
‘enabled the accompaniment of piano music by projections. It consisted of a
switchboard connected to four large and four small color projectors and was fitted
with keys and levers like a sort of harmonium’ (Barth et al., 2012: 223). The
Sonchromatoscope was one of the first devices that resemble today’s VJ mixing
consoles (Scheel, 2006). László also wrote a theoretical text called Colour-Light-
Music which describes how his technique combines two art genres - music and
painting - to form a new art (Bruce Elder, 2008: 67). Fischinger worked with
László on his Colour-Light-Music project, as he created abstract 35mm film
material needed to accompany Làszló’s presentations (Daniels et al., 2011: 81).

Before collaborating with Làszló, Fischinger had already begun experimenting

with animation. Inspired by Ruttmann’s Lichtspiel Opus 1, Fischinger started
developing an ‘abstracted musical animation language in 1920s…[which] was
composed of a series of geometric shapes and synchronized music’ (Kim, 2008:
31). He produced a series of Studies, which consisted of seventeen black-and-
white films, all tightly synchronized to music (CVM and Moritz, 2006-13). Films
like these are the predecessors of today’s music videos.

Apart from creating images in synchronization with music, Fischinger also

invented a ‘Wax Slicing machine to produce unique abstract imagery’ (Keefer,
2009: 1). William Moritz describes how:

[Fischinger] set about engineering a precision slicing machine to connect

with a film camera in such way that each time a slice would be removed,
the camera would film a single frame of the remaining block. By inserting
a cone of different-coloured wax in the centre of a square block, one could
make a circle appear or disappear, depending on whether the point of the
cone faced toward or away from the camera. So soft supple organic
shapes could mingle with hard geometric forms. (2001)

Fischinger saw his experiments as a new art which he named Raumlichtmusik

(space light music). He believed that eventually all the arts would combine
together into this new art. On Raumlichtmusik, Fischinger wrote:

Of this Art everything is new and yet ancient in its law and forms. Plastic-
Dance- Painting- Music become one. The Master of the new Art forms
poetical work in four dimensions…Cinema was its beginning…
Raumlichtmusik will be its completion2. (Keefer, 2009: 2)

In the 1940s, Fischinger also invented the Lumigraph – his variant of a colour-
instrument, with a new interface. Like Wilfred’s Lumia instrument, the device only
displayed light, but during performances a musical instrument often accompanied
it. In the article ‘Writing Light’, Fischinger’s wife Elfriede Fischinger (1910-1999)
describes the Lumigraph:

The wooden panels had become a box-like frame about 1 foot wide and 1
foot deep. This "frame" contained an opening that encased the latex sheet
2
This quote is from Keefer’s article “Raumlichtmusik” – Early 20 th Century Abstract Cinema Immersive
Environments. Keefer cited the original quote from an unpublished typescript by Fischinger found in
the Fischinger Collection, Center for Visual Music, Los Angeles.

7
 mounted on a wooden canvas-support 3 feet high by 4 feet wide. The
colored gels had been fastened to glass strips that rotated on a wheel
inside the wooden frame-case, and a thin slit just inside the front edge of
the case only allowed the light from the (cool) neon tubes inside the case
to emerge at this one point to make a thin layer of light in front of the
rubber screen. This light slit was just far enough in front of the screen so
that only those portions of the screen that were pushed forward would fall
into the path of the light and become visible to the spectator sitting in
front of the instrument - and unless something did protrude into the thin
light layer, nothing would be visible at all! (The case was even painted
black). Each color of gel had been mounted on a different glass strip, and
these colored glasses could be rotated by pulling a canvas strip on the
back side of the case - Oskar had made a little scale to show you just
where to stop to get red, blue, yellow, etc. (1980)

Fischinger gave multiple performances with the Lumigraph, and believed that it
would become a household device, but his dream of selling it on a commercial
level did not happen. Regardless, Michael J. Lyons noted, ‘this seems to be a very
early example of a touch-screen, or at least tactile input device, that are currently
changing the way people use computers’ (Lyons, 2012).

All of Fischinger’s inventions, experiments and accomplishments make him a very

important figure in the history of visual music, and hence he is deemed as the
Father of this art. Fischinger created more than fifty films and 800 paintings, and
his work influenced artists and filmmakers from the early 1900s and still
continues to do so nowadays.

3.5 Colour in film

The use of colour in film was still at an early start in the 1930s. In 1932, Bela
Gaspar (1898-1973), a Hungarian chemist, created a system for shooting and
printing film with colour, and named it Gasparcolor. In the article ‘Gasparcolor:
Perfect Hues for Animation’, Moritz describes the process:

Gasparcolor involved a subtractive-process print film that contained three

separate emulsions: magenta and yellow layers sandwiched on one side,
and cyan-blue on the other. This meant that Gasparcolor prints could be
made from any three-color separation negative: either a single-strip black-
and-white film with three successive red-green-blue exposures for each
final image, or three separate strips of black-and-white film, each one
containing a different red-, green-, or blue-sensitive information…
Gaspar's particular chemistry allowed the three-layered film to project
brilliant, saturated colors in a complete spectrum of all hues. (1995)

Fischinger’s expertise was crucial in the development of the first multimedia

program. He built camera mechanisms that coordinated the camera shutter with
a wheel containing the three strips of coloured film for the three-strip
Gasparcolor-film process. After completing the system, Fischinger used it to
create his first of many colour films, called Kreise (Circles), in 1933. More
experimental projects followed, where Fischinger used different techniques to
synchronize images of geometrical patterns, now in colour, to different genres of
music, such as jazz in Allegretto (1936) and classical music in An Optical Poem
(1938). All the shapes and pattern used in An Optical Poem were paper cut outs,
suspended on a transparent wire and shot frame by frame. One could say that
these were a few of the first primitive music videos ever made.

8
3.6 Different techniques of abstract film

During Fischinger’s time, other artists were also experimenting with various
abstract filmmaking techniques. Mary Ellen Bute (1906-1983), an American
experimental filmmaker, also created non-representational films, accompanied by
classical, romantic or modern music. Unlike Fischinger, she did not synchronize all
the imagery with the music, as ‘she was less interested in the direct visualization
of music than in the creation of an equivalent counterpart’ (Daniels et al., 2011:
77). Instead, Bute used mathematical equations to create a relationship between
the musical score and visual elements of her composition and combined them
together. Interestingly, the two would be developed separately and only layered
together at the end. Like Fischinger, Bute also used the trichromatic process to
create bright-coloured geometric patterns for the visual element of her work. For
one of her most popular audiovisual works, Tarantella (1940), Bute ‘made over
7,000 hand drawings for four and a half minutes of film’ (Daniels et al., 2011: 95)
and had to use separate celluloid strips for each individual colour.

Another important figure of the 1930s was artist and filmmaker Len Lye. Lye
focused on animation and films and finding ways of how to create movement in
static images and sculptures. Unlike Fischinger, Lye did not use a direct camera
for his animation works. Instead, he used a different technique, which he called
the “direct film” or “direct animation” approach. Lye discovered this technique, as
he could not afford to hire a film camera, and had to find another approach to
create abstract films. Lye directly painted, stained or scratched patterns and
images onto celluloid and hence created camera-less films. Similar to Fischinger,
Lye also liked to synchronize his films to music, preferring Latin and African music
over classical. He also re-coloured black and white films using new printing
techniques by Technicolor (Gasparcolor’s competitor), and combined his original
abstract images with representational documentary footage, by drawing the
images directly on the documentary’s filmstrip (Brougher et al., 2005: 111).

In the book Visual Music, Kerry Brougher describes how:

[Lye’s films] are also among the first films to underscore the nature of the
medium itself: watching them the viewer is fully aware of the film stock
running vertically through the projector, the jittery registration, the
flickering imperfect effects of persistent of vision, and the material quality
of the celluloid itself. In this sense, they are precursors…to structural films
of the 1960s and 70s with their insistence on deconstructing the medium.
(Brougher et al., 2005: 112)

Along with Fischinger, Lye’s influence on avant-garde artists is noticeable and

clearly shown in both visual music artists and filmmakers of early cinema.

3.7 Synthetic Sound and Motion Graphics

Fischinger’s ideas and innovations were endless. Similar to Kandinsky’s believes

Fischinger was convinced that there was a relationship between sound and visual
imagery. In the 1930s, Fischinger noticed that the optical soundtrack on a
filmstrip was composed of abstract patterns. Following on this conviction,
Fischinger started experimenting with drawing forms and patterns on the
soundtracks of celluloid rolls. This breakthrough led to the first films having direct

9
sound instead of recorded sound. These works, which he named Sounding
Ornaments (1932) did not produce actual music, but mostly noise. Author Marcel
Schwierin commented that this work ‘anticipate[d] later developments in the
areas of electronic and New Music’ (2009).

Norman McLaren (1914-1987), an internationally acclaimed filmmaker, also

experimented with synthetic sound. McLaren employed two methods in producing
his films. He either used “direct animation” as Lye did before him, or used real
actors and objects and recorded their actions through a conventional camera.
McLaren also created the synthetic sound for some of his films. He used to
scratch marks on the sound track like Fischinger, make incisions in the filmstrip,
paint images directly on the sound track or lay photographs of hand-drawn
images on the sound track (MENC, 1968: 114). His works Dots and Loops both
produced in the 1940s, and two of McLaren’s most popular works had
handwritten sound tracks. McLaren also produced one work without visuals and
called it Rumba (1939-41): a synthetic sound composition made with drawing
images directly on the sound track.

During a radio interview with Forsythe Hardy, McLaren shared his views on
synthetic sound:

I like to think of synthetic sound, it would be better to call it animated

sound, as a new medium, with a new set of inherent qualities and
limitations. As a rhythmic instrument it is definitely superior to most of the
traditional instruments in the subtlety, the speed, and the complexity of
the rhythms which it can make, for each of these rhythms can be carefully
plotted in advance. (Hardy, 1951)

Around the time McLaren and Fischinger were experimenting with synthetic
sound, the Whitney brothers, John Whitney Sr. (1917-1995) and James Whitney
(1921-1982), were also experimenting with new technologies to create abstract
films and sound. Amongst many of their creations, Whitney Sr. constructed an
eight-millimetre optical printer and a sound-producing pendulum system while his
brother James Whitney developed a stencil-system for their optical experiments.
Following on Bute’s methods, the Whitney brothers focused on creating
audiovisual music, by ‘not only applying an overarching structure consisting of
basic motifs (developed and varied in accordance with classical counterpoint and
serial principles), but also translating these into image and sound using
analogous production methods’ (Daniels et al., 2011: 78). The brothers’
inventions not only helped in the development of synthetic sound, but also
started the earliest work on motion graphics and special effects.

The pendulum system consisted of several pendulums of different lengths fitted

with variable weights, and hung ‘by a wire to an aperture in the camera which
recorded their movements, thereby creating sound from motion. Using this
system, the Whitneys were able to create a four-octave range of electronically
produced tones’ (Brougher et al., 2005: 125). Using James Whitney’s optical
stencil system, the brothers would then trace images onto animation paper and
display them with light shot through the stencils, and synchronize the visual
imagery to the produced sound.
One of James Whitney’s most innovative machines for creating visual imagery
before the computer era was Yantra. In Sanskrit3, Yantra means “machine” or
“implement”. Multiple cards were punched with a pin until thousands of minute
3
Sanskrit is one of the most ancient languages. It is one of the official languages of India, and the
liturgical language of Buddishm, Hinduism and Jainism. See
http://www.omniglot.com/writing/sanskrit.htm [Accessed 17 January 2015]

10
dots were created. These cards were then used as stencils to draw shapes on
other cards. By using an optical printer that the brothers had invented, the
patterned cards were combined to create a visual film. Influenced by the Austrian
composer Arnold Schoenberg (1874-1951) and his twelve-tone serial music scale,
the brothers acknowledged that the only way to make sure their films were fully
non-representational they had to:

Reduce the image down to its most fundamental state - essentially a point
of light, which could then be ordered like a tone row. Indeed, these are no
longer images at all, but particles that can be multiplied, grouped, and
manipulated into ghost-like traces of cosmic energy flows. (Brougher et
al., 2005: 125-132)

The importance of the Whitney brothers’ work is that their experiments produced
a new type of visual music; both the sound and visual imagery created were more
electronic and non-representational than anything produced so far by anyone.
Furthermore, the sound and visual imagery produced were independent of each
other. But most of all, the brothers are known for their work in the computer era
– as John Whitney Sr. started using computers to create animation and was the
first person to generate visual effects with computers.

As the computer era started, John Whitney Sr. developed an interest in computer
technology, and created his own computer software that synchronized computer-
generated graphics with music. This breakthrough brought around a new meaning
for visual music, and also affected the history of cinema. With this new
technology, John Whitney Sr. not only created graphics for music, but also for
commercial cinema, the most popular being the title sequence for Alfred
Hitchcock’s (1899-1980) Vertigo (1958).

In 1994, in his article ‘To Paint on Water: The Audiovisual Duet of

Complementarity’ John Whitney Sr. described the power of computer technology
as “complementarity”.

A major art form based on a common foundation of harmony is developing

that intertwines color with tone in a complementary bond. I call this
associative relationship a ‘complementarity.’ For the first time, one can
design and execute visual and musical patterns in an inter-reactive form of
temporal union. The ancient belief that the primary attributes of color and
tones posses a magical affinity now finds its technical basis of truth.
(1994: 46)

The Whitney brothers’ work influenced several artists that came afterwards, in
the sound, visual and special effects departments. The ground-breaking
techniques used by the Los Angeles brothers were the foundation for today’s
technological methods used in cinema and music. The Whitney brothers’ aim to
create cosmic cinema particularly influenced painter and filmmaker Jordan Belson
(1926-2011). Moritz describes him as being ‘the last of the great masters of the
California Visual Music artists’ (1998). Like the Whitney brothers, Belson created
abstract films, and was very careful not to have any representational images in
his work. In an interview with MacDonald, Belson explains how:

I tried to be pretty ruthless about eliminating any images where the

means by which the imagery was obtained was obvious. I didn’t want the
viewer to be more aware of the process than of the event taking place on
the screen…I like a convincing illusion. I don’t want there to be any ideas
connected to my images, and if there are any there, if anybody sees any,
those are entirely in the eyes of the beholder…Actually, the films are not

11
 meant to be explained, analysed, or understood. They are more
experiential, more like listening to music. (1998: 77-78)

Although Belson had the same ideas as the Whitney brothers, he approached his
works using different techniques. The Whitney brothers made use of the
contemporary technology at hand and worked with minute particles, while Belson
focused on creating geometric transformations that seemed to have no artificial
origin. He persisted on creating mystical organic forms, of a natural state
(Brougher et al., 2005: 148).

Belson, like Fischinger and the Whitney brothers, also experimented with
synthetic sound and coloured light. Wilfred’s ‘Lumia’ inspired Belson to produce
the popular Vortex concerts (1957-59). Belson collaborated with electronic music
composer Henry Jacobs (1924-) and together they created audio-visual
performances. The concerts featured projected images and psychedelic lights by
Belson, electronic music by Jacobs, Stockhausen (1928-2007) and other music
composers, and also films by James Whitney amongst others. These Vortex
concerts marked the beginning of psychedelic light shows and rock concerts of
the 1960s (Kim, 2008: 40).

From there onwards, all the aspects of visual music developed hand in hand with
technology. As the world became more technologically based, going from
handmade works to computerized ones, turning from analogue to digital, and
evolving at a rapid rate, so did the arts.

4. VISUAL MUSIC TODAY

With the birth of MTV (1981), music videos and commercial abstract video clips
became the norm. As huge advances in technology continued to be made, from
the 1980s till today, the combination of visual and auditory elements became so
common that it is now taken for granted. Sandra Naumann writes about how
‘today, digital technology seems to have dissolved the distinction between sounds
and images into bits and bytes and to have assimilated all the strategies of
musicalization used in the visual arts of the past’ (Daniels et al., 2011: 71).

Nowadays, the term visual music is used to describe a new audiovisual discipline
used in ‘raves, the clubs, the parties and concerts, but also galleries and
museums over the last several years, combining filmic images and music in a
public space’ (Lund, 2004). This field of audiovisual art is also known as VJing –
Visual Jockeying. A VJ is a visual artist who produces sequences of visual imagery
and mixes them on the spot to the music of a DJ (Disc Jockey) or a musician.
Colour-organs and light-instruments of the nineteenth and twentieth centuries
clearly still have an influence on the visual artists of today. A VJ uses coloured
lights projected on a screen, and a series of images for his performances. With
today’s technology, instead of having a colour-organ like Castel’s Ocular
Harpsichord, or Wilfred’s ‘Lumia’, the VJ has a mixer with sliders for the altering
of the lights and a laptop for the graphics used. It can be said that Hallock
Greenewalt’s sliding rheostat on her Sarabet instrument could have been the first
prototype of today’s visual mixers. Similar to the work of Kandinsky, Klee,
Fischinger, the Whitney brothers, Belson and other avant-garde painters and
filmmakers, the visuals used for performances are mostly non-representational,
can be symbolic but rarely tell a story (Lund, 2004).

12
Recent developments of technology have provided artists with interactive dance
systems that allow them to spontaneously create graphics and music as a
response to hand and body movement. With the combination of computer
programs such as Cycling 74’s Max, MSP and Jitter, and motion sensor devices
like Nintendo’s Wii and Microsoft’s Kinect, visual music is becoming less
programmed and more unpremeditated. Most importantly, today’s technology is
less complicated and user-friendlier than it was years ago. These new systems
take the combination of music, improvisation and visual imagery to a whole new
level. This concept is called comprovisation. Comprovisation is defined as the
amalgamation of ‘hierarchical process aspects of composition with the embodied
spontaneity that characterizes improvisation’ (Mailman, 2013: 346).

5. CONCLUSION: VISUAL MUSIC TOMORROW

Looking at the historical timeline of visual music from the very beginning to the
present, one can see the astounding development of this art. As discussed in this
paper, visual music comprises more than one aspect of art and hence cannot
have a specific definition. The definition of visual music evolves along with the
technology and discipline itself.

Visual music has already come a long way. One may now ask, where to next?
Music and visual imagery have already become entwined together. I believe that
artists will continue to pursue the idea of having a multi-sensorial experience
through audiovisual works. I think that with the development of technology that
is yet to come, artists will be able to come closer to what Kandinsky had dreamt
of in the beginning of the abstract period: the merging of arts into one, which
stimulates more than one of the senses simultaneously.

The arts are encroaching one upon another, and from a proper use of this
encroachment will rise the art that is truly monumental.

- Wassily Kandinsky (1914: 20)

REFERENCES

Barth, F., Giampieri-Deutsch, P., Klein, H. (2012) Sensory Perception: Mind

and Matter. Austria: Springer.

13
Betancourt, M. (2006a) ‘Mary Hallock-Greenewalt’s ‘Abstract Films’’. Millennium
Film Journal, Fall 2006, No. 45/46. [Online] Available from:
http://mfjonline.org/journalPages/MFJ45/Betancourtpage.html [Accessed 12
January 2015].

Betancourt, M. (2006b) Visual Music and the Birth of Abstract Art. Barbara
Wilson Lecture. University of Nebraska, Omaha. 23 February 2006. [Online]
Available from: http://www.michaelbetancourt.com/pdf/Wilson_Lecture-
Abstraction.pdf [Accessed 9 January 2015].

Brougher, K., Strick, J., Wiseman, A. and Zilczer, J.(2005) Visual Music:
Synaesthesia in Art and Music Since 1900. Los Angeles: Thames & Hudson.

Bruce Elder, R. (2008) Harmony and Dissent: Film and Avant-garde Art
Movements in the Early Twentieth Century. Waterloo, Canada: Wilfrid Laurier
University Press.

CVM Editors & Moritz, W. (2006-2013) ‘Oskar Fischinger Biography’. Text

adapted by CVM from biographical material provided by Moritz, W. and The
Fischinger Archive. [Online] Available from:
http://www.centerforvisualmusic.org/Fischinger/OFBio.htm [Accessed 9 January
2015].

Daniels, D., Naumann, S., Thoben J. eds. (2011) See This Sound
Audiovisuology 2. Essays. Köln: Verlag der Buchhandlung Walther König.

Dusan (2014) Viking Eggeling, Monoskop.org. [Online] Available from:

http://monoskop.org/Viking_Eggeling#Films [Accessed 7 January 2015]

Fischinger, O. Raumlichtmusik. Unpublished typescript. California: Collection

Fischinger Archive.

Fischinger, O. (1947) ‘Corrections to Richters [sic] Article about the History of

the Avantgarde’. Unpublished typescript. Los Angeles: Collection Centre for Visual
Music. [Online] Available from:
http://www.centerforvisualmusic.org/Fischinger/OFCorresp.htm [Accessed 15
January 2015].

Fischinger, E. (1980) ‘Writing Light’. First printed in Fischinger: A Retrospective

of Paintings and Films Exhibition Catalogue. Colorado: Denver. Reprinted in The
Relay. (3.2):4-7 (May 1984). Reprinted in First Light, Haller, A. ed. (1998) New
York: Anthology Film Archives, p.30-34. [Online] Available from:
http://www.centerforvisualmusic.org/WritingLight.htm [Accessed 10 January
2015].

Fry, R. (1920) Vision and Design. London: Chatto & Windus.

Fry, R. (1927) Cézanne: A Study of His Development. New York: Macmillan.

Hall, M. (1928) The Secret Teachings of All Ages. San Francisco: H.S. Crocker, p.
82. [Online] Available from: http://www.sacred-texts.com/eso/sta/sta19.htm
[Accessed 30 December 2014]

Hallock Greenewalt, M. (1946) Nourathar: The Fine Art of Light Color Playing.
Philadelphia: Westbrook.

Hardy, F. (1951) Interview with Norman McLaren. Dots and Loops. BBC. 22
November 1951.

14
Kandinsky, W. (1914) The Art of Spiritual Harmony. trans. Sadler, M. London:
Constable & Co.

Keefer, C. (2009) ‘Raumlichtmusik – Early 20th Century Abstract Cinema

Immersive Environments’. Leonardo Electronic Almanac, October 2009, 16, (6-7).
[Online] Available from:
http://www.leonardo.info/LEA/CreativeData/CD_Keefer.pdf [Accessed 11 January
2015].

Kim, M. (2008) New Techniques for Managing Audio-to-Video Translation. PhD

dissertation. Evanston: Northwestern University. pp. 19-41.

LiteraryDevices Editors. (2013) Synesthesia LiteraryDevices.net [Online]

Available from: http://literarydevices.net/synesthesia/ [Accessed 20 December
2014].

Lund, H. (2004) Visual Music. Revised in November 2004. Originally from Art
History and Gender Colloquium. 2002. Berlin Universität der Künste, Germany.
[Online] Available from:
http://www.fluctuating-images.de/files/images/pdf/Visual_Music_engl.pdf
[Accessed 9 January 2015].

Lyons, M. (2012) Zemi, Oskar Fischinger’s Lumigraph and The Time Travelers.
16 February, 2012. [Online] Available from:
http://zemi.kasrl.org/2012/02/16/oskar-fischingers-lumigraph-and-the-time-
travelers/ [Accessed 10 January 2015].

MacDonald, S. (1998). A Critical Cinema 3: Interviews with Independent

Filmmakers. Berkeley and Los Angeles: University of California Press.

Mailman, J. (2013) ‘Imrovising Synesthesia: Comprovisation of Generative

Graphics and Music’. Leonardo Electronic Alamanac, 13 November 2013, 19 (3):
346-79. [Online] Available from:
https://www.academia.edu/5104492/Improvising_Synesthesia_Comprovisation_o
f_Generative_Graphics_and_Music [Accessed 8 January 2015].

MENC: The National Association for Music Education (1968) ‘The

Handwritten Sound Track’. Music Educators Journal, November 1968, 55 (3):
114.

Moritz, W. (1986) ‘Towards an Aesthetic of Visual Music’. Asifa Canada Bulletin,

December 1986, 14: 3. Montreal: ASIFA Canada. [Online] Available from:
http://www.centerforvisualmusic.org/TAVM.htm [Accessed 27 December 2014].

Moritz, W. (1993) ‘Visuelle Musik: Höhlenmalereien für MTV?’ (Visual Music:

Cave Paintings to MTV?). Sound & Vision Exhibition Catalogue, 1993, 132-145.
Frankfurt: Deutsches Filmmuseum. [Online] Available from:
http://www.centerforvisualmusic.org/library/WMCavePtgs.htm [Accessed 18
December 2014].

Moritz, W. (1995) Originally: Le Gasparcolor: Une Procedure Chromatique.

Lecture at Musee du Louvre, Paris. October 6, 1995. Furniss, M. ed. ‘Gasparcolor:
Perfect Hues for Animation’ Animation Journal (5): 52-57, 1996. [Online]
Available from: http://www.oskarfischinger.org/GasparColor.htm [Accessed 6
January 2015].

15
Moritz, W. (1996) ‘Visual Music and Film-as-an-Art Before 1950’. On the Edge of
America:California Modernist Art, 1900-1950. Karlstrom, P. ed. Berkeley:
University of California Press, p. 224.

Moritz, W. (1997) ‘The Dream of Color Music, and the Machines that made it
Possible’. Animation World Magazine, April 1997, (2.1). [Online] Available from:
http://www.awn.com/mag/issue2.1/articles/moritz2.1.html [Accessed 9 January
2015]

Moritz, W. (1998) Jordan Belson, Last of the Great Masters. Tenth Annual
Society of Animation Studies Conference. Chapman University, Orange,
California. August 1998. [Online] Available from:
http://www.centerforvisualmusic.org/BelsonAJ.htm [Accessed 14 January 2015].

Moritz, W. (2001) ‘Oskar Fischinger: Artist of the Century’, reprinted in Animac

magazine, 2001. Lleida, Spain.

Ox, J. and Keefer, C. (2008) ‘On Curating Recent Digital Abstract Visual Music’.
Abstract Visual Music Catalogue, 2008, New York: The New York Digital Salon.
[Online] Available from: http://www.centerforvisualmusic.org/Ox_Keefer_VM.htm
[Accessed 20 December 2014].

Peacock, K. (1988) ‘Instruments to Perform Color-Music: Two Centuries of

Technological Experimentation’. Leonardo, 21 (4): 397-406.

Scheel, S. (2006) ‘Music Visualization – The Interplay of Color and Sound’, Ars
Electronica Simplicity Festival Catalog, trans. Taylor-Giada, J. [Online] Available
from: http://90.146.8.18/en/archives/festival_archive/festival_catalogs/
festival_artikel.asp?iProjectID=13708 [Accessed 7 January 2015].

Schwierin, M. (2009) ‘Ornament Sound’. See This Sound. Academy of Visual

Arts, Leipzig. [Online] Available from:
http://www.see-this-sound.at/works/296/asset/49 [Accessed 13 January 2015].

Spalding, F. (1980) Rogery Fry, Art and Life. Berkeley and Los Angeles:
University of California Press.

Tyler, A., Tyler C., Tyler, I. (2003) ‘Neurohistory and the Arts. Was Kandinsky a
synesthete?’ Journal of The History of the Neurosciences, 12 (2): 223-226.

Whitney, John Sr. (1994) ‘To Paint on Water: The Audiovisual Duet of
Complementarity’. Computer Music Journal, Autumn 1994, 18 (3): 46.

ANNOTATED BIBLIOGRAPHY

Betancourt, M. (2006b) Visual Music and the Birth of Abstract Art. Barbara
Wilson Lecture. University of Nebraska, Omaha. 23 February 2006.

This text provides a detailed historical timeline of visual music and how it and the
need for synaesthesia led to the birth of abstract art. Betancourt explains how the

16
history of visual music was concurrent to that of cinema. A description of the
developments in visual music is given in parallel to the evolution of non-
representational film. The works and key inventions of filmmakers that led to the
growth of the abstract art are mentioned, in relation to synaesthesia. Betancourt
compares and contrasts the current development of the art with that of the
eighteenth and nineteenth centuries. Although this article spans multiple aspects
of visual music, the main concern of the paper is the development of abstract art,
and how synaesthesia has affected and is still affecting it. This text was very
helpful for my research, as apart from giving me a historical description of
events, it also discussed synaesthesia and its impact on the arts.

Brougher, K., Strick, J., Wiseman, A. and Zilczer, J.(2005) Visual Music:
Synaesthesia in Art and Music Since 1900. Los Angeles: Thames & Hudson.

This book was crucial for my research on visual music. It holds the history of
visual music ever since the 1900s. All the information in this book is about an
exhibition held in The Museum of Contemporary Art in Los Angeles, named ‘visual
music’ as it showcases works of several artists that fall in the category of visual
music. Apart from illustrations and photos of the exhibition, the editors compiled
five essays that together serve as a very descriptive history of how visual music
came around.

Of particular interest was the essay of Brougher, along with the foreword and
essay written by Strick. In the foreword, Strick gives an overview of how visual
music started, and who are the primary artists concerned with this art. In his
essay, also called ‘Visual Music’, Strick discusses how there are multiple aspects
to this genre, but there is a recurring common idea in all of them: that of music
being translated into visual imagery. Strick also makes connections between art,
music and film through synaesthesia.

Brougher’s essay is called ‘Visual-Music Culture’. It particularly focuses on motion

painting, and how Kandinsky and other avant-garde painters influenced
filmmakers such as Fischinger. It gives detailed descriptions of non-
representational film methods, colour music and post-impressionist paintings.
Every important figure associated with abstract film is mentioned, and a thorough
account of his/her theories, discoveries and constructions is given. Various
examples of works are also mentioned in detail, supported with coloured
illustrations and photos.

Each essay moves in chronological order; most of time the author picked up from
where the previous one had left. At the end of the book, one finds a list of all the
works displayed at the exhibition. A very helpful chronology of accomplishments
and works of different artists from 1895 to 2004 is also present, along with
concise biographies of important visual music artists.

Daniels, D. and Naumann, S. with Thoben J. eds. (2011) See This Sound
Audiovisuology 2. Essays. Köln: Verlag der Buchhandlung Walther König.

This book comprises a collection of essays on the development of audiovisual

media and art from the eighteenth century to nowadays. The texts discuss the
correlation between the auditory and visual senses in art, and their relationship to
science and technology. The ten essays are divided in three categories: Music and
Art; Sound and Image; Hearing and Seeing. Detailed descriptions of some of the
significant works mentioned in the essays are distributed throughout the book.
The book was key in my research as it delivers a detailed historical account of the

17
development of audiovisual art along with its relevant theories. Specific attention
is given to the twentieth century, the evolution of the avant-garde movement,
and the progress of technology.

Of particular interest was Naumann’s essay ‘The Expanded Image: On the

Musicalization of the Visual Arts in the Twentieth Century’. Naumann gives a
detailed account of the history of musical elements in art. She first looks at post-
impressionist art, highlighting painters like Kandinsky and Klee, and discusses the
shift from static visual paintings to film. A recount of the development of film and
commercial cinema is also given, focusing on pioneers of filmmaking such as
Richter, Fischinger and Conrad. A brief look at spatialization and improvisation in
real-time productions is also given. Finally, Naumann describes the process from
analogue to digital media and nowadays’ audiovisual technology brings the essay
to an end.

Hallock Greenewalt, M. (1946) Nourathar: The Fine Art of Light Color Playing.
Philadelphia: Westbrook.

Hallock Greenewalt’s book is entirely devoted to her art Nourathar, and her
colour-instrument the Sarabet. This book proved to be very useful, as it gives
great detail on Hallock Greenewalt’s take on colour-music. The fact that it was
written by the inventor herself already made it a key text in my research for this
paper. The book is divided into six parts, but part one was the most useful for my
research.

In part one of the book, Hallock Greenewalt writes about how she came to invent
this new fine art, which she called Nourathar. Hallock Greenewalt starts by
describing the five fine arts that existed before the creation of Nourathar,
comparing and contrasting them, and takes the reader through the creative
process that led her to Nourathar. The other parts focus on all the aspects of
Nourathar. Hallock Greenewalt writes about the instrument itself, giving full
details of the apparatus. The notation of the colour-instrument and how it came
to be is also explained, along with how to compose for the instrument, tune it and
most importantly the techniques used to play the instrument.

Keefer, C. (2009) ‘Raumlichtmusik – Early 20th Century Abstract Cinema

Immersive Environments’. Leonardo Electronic Almanac, October 2009, 16, (6-7).

Keefer’s article focuses on a specific attribute of twentieth century’s abstract

cinema, which is ‘Raumlichtmusik’. ‘Raumlichtmusik’ was a very significant
concept in the historical development of visual music; hence this article was a
prime source of information for my research. Keefer discusses two pioneers of
raumlichtmusik, Fischinger and Belson. Keefer makes connections from
Fischinger’s and Belson’s experiments to abstract cinema of the twentieth
century. In the first half of the text, a detailed account of Fischinger’s discoveries
and inventions is given. The work relationship between Fischinger and László is
also mentioned in depth, and Keefer also writes about the projects they worked
on together. The second half of the text focuses on Belson’s projects, particularly
the Vortex concerts, which brought about the first psychedelic lights concerts.
Throughout the text, Keefer describes how the work of Fischinger and Belson
helped to form the art of abstract cinema and hints that their developments are
helping to slowly merge science and art together, quoting multiple art critics and
even the pioneers themselves.

18
Kim, M. (2008) New Techniques for Managing Audio-to-Video Translation. PhD
dissertation. Evanston: Northwestern University. pp. 19-41.

Kim’s dissertation was very useful while researching for this paper. The
dissertation explores the idea of combining visual and auditory senses together,
giving a thorough examination of visual music through the times. Kim starts from
the very beginning, describing the theories of Greek theorists such as Pythagoras
and Aristotle. The dissertation offers extensive information about the relationship
between colour and music and how the theories of classical antiquity evolved and
brought us to today’s visual music.

This paper served as a key text in my research, as it offers a detailed history of

colour music. Kim writes about all the specifications of Castel’s Ocular
Harpsichord and his tone to colour relationship. This paper also provides a good
history of abstract film, with reference to the pioneers of non-representational
film: Ruttmann; Eggeling; Fischinger; Lye and McLaren. Kim also recounts the
work of Belson and the Whitney brothers at the beginning of the computer era.

The rest of the paper discusses the developments technology brought about,
highlighting instruments and computer programs that enable audio-to-video
translation.

Mailman, J. (2013) ‘Imrovising Synesthesia: Comprovisation of Generative

Graphics and Music’. Leonardo Electronic Alamanac, 13 November 2013, 19 (3):
346-79.

This recent article gives an overview of the new categories of visual music that
one finds nowadays. Mailman gives a description of new interactive systems that
let the artist create visual music spontaneously, instantly and using his own body
to do so. This text was very useful for my paper as it gives a brief summary of
the technology used to produce a visual music project nowadays. A thorough
account of a live system is given, accompanied with photos and diagrams. The
article deliberates about how these new interactive systems allow for more
improvisation and creativity in visual music. Mailman discusses this new concept
of ‘comprovisation’, which is a combination of the words composition and
improvisation, and talks about how this new idea will enhance the synaesthesia
experience of visual music. The rest of the article focuses on the computer
programs, technology and apparatus used for this new interactive system.

Moritz, W. (1997) ‘The Dream of Color Music, and the Machines that made it
Possible’. Animation World Magazine, April 1997, (2.1).

This article gives an overview of colour music, and the developments of the
machinery used, from classical antiquity times to the eighteenth century and
finally to the twentieth century. Although the article is relatively short, it provides
an in depth account of apparatuses used by several artists. This text helped in my
research as various artists were mentioned, unlike other sources that only
described the pioneers of visual music. The article moves in chronological order,
and also compares and contrasts the different techniques used to construct the
colour-organs.

Ox, J. and Keefer, C. (2008) ‘On Curating Recent Digital Abstract Visual Music’.
Abstract Visual Music Catalogue, 2008, New York: The New York Digital Salon.

19
The text was primarily used for a catalogue of an abstract visual music exhibition,
and later made available online. This text was very useful, as Ox and Keefer
expertly recognise the various aspects of visual music, and give a description on
each one. A brief historical background of visual music is also provided, from the
initial hypotheses of Greek theorists, to the development of visual music through
the nineteenth and twentieth centuries. Ox and Keefer also highlight the most
important artists of visual music, in chronological order, from the 1910s to the
1960s. The remainder of the text discusses the works exhibited and how they
represent specific aspects of visual music’s history, the process of curating the
exhibition, and the difficulties Ox and Keefer encountered while doing so.

Peacock, K. (1988) ‘Instruments to Perform Color-Music: Two Centuries of

Technological Experimentation’. Leonardo, 21 (4): 397-406.

Peacock’s article proved to be a very good source of information regarding colour-

music and the instruments used for this type of art. Peacock gives a detailed
description of how the theories on the relationship of colour and music evolved,
and how they led to the creation of colour-instruments. This articles spans over
the eighteenth and nineteenth centuries, but also gives an account of the colour-
instruments developed in the twentieth century. Peacock discusses how the
relationship of colour and music has had several definitions throughout the times.
An explanation of how the various definitions came to be is also given.

This text was of great value to my research as theories significant to the

development of colour music were thoroughly explained, and connections were
made amongst the theorists that helped to understand exactly who was key to
the development of this art. In particular, attention is given to Castel’s theories
and how these led to the construction of the Ocular Harpsichord, the first colour-
organ ever produced. A brief description of nineteenth and twentieth century
colour-instruments and how they vary from Castel’s invention is also given.
Important figures and their colour-instruments and methods such as Hallock
Greenewalt, Rimington and particularly Wilfred are highlighted.

20