There have been many attempts to blur the boundaries of sound and visual arts. Instruments for the eyes have been conceptualized since the obscure ocular harpsicord in 1725.
In the early 20th century many artists claim to be affected by synesthesia and tried to incorporate their condition into their art making.
Color Organ invented by composer Alexander Scriabin in 1915 for his Prometheus: Poem of Fire.
“Color is the keyboard, the eyes are the harmonies, the soul is the piano with many strings. The artist is the hand that plays, touching one key or another, to cause vibrations in the soul.”
He worked with other composers, making librettos and stage designs.
Film and video
Experimental filmmakers embrace the idea of visual music. Music as a pathway to abstraction.
Oskar Fischinger – An Optical Poem (1938)
Norman McLaren – Dots (1940)
“Made by painting directly onto clear frames of film, as pictured in the frame enlargements below. Interestingly, the music was created in the same way, painting directly into the area on the film strip usually reserved for the soundtrack.”
Norman McLaren – Synchromy 1971
“To produce the film’s musical soundtrack, McLaren photographed rectangular cards with lines on them. He arranged these shapes in sequences on the analog optical sound track to produce notes and chords.”
John Whitney Sr – Arabesque (1975)
Early computer animation, he applied musical principles such as the harmonic progression to the generation of images.
Using real-time sound analysis (fft) control parameters of a generative composition
Atari video music, 1976. The first (analog) electronic music visualizer.
Jeff Minter, Virtual Light Machine 1990, Atari Jaguar
Live visuals or VJing
Visual artist creating visuals live in response to a live musical performance.
Exploding Plastic Inevitable was a series of multimedia events organized by Andy Warhol in 1966 and 1967, featuring musical performances by The Velvet Underground, dancers, and projections. Related to the notion of expanded cinema
The interest in enhancing musical experiences with visuals re-emerged in the 60s and 70s with psychedelia and the “expanded cinema”
Liquid light shows – overhead projector + live manipulation of fluids. Late 60s – today.
Hexstatic+Coldcut 1997 in collaboration with Greenpeace.
This is a 1997 music video but gives an idea of the visuals/sound relationship.
Ryoji Ikeda, musician and installation artist. a/v performance
Midi controller + real time engine
Tech heavy VJing with real time capture, surround screens etc – they’ve got interesting web based stuff as well
Speedrunning as live AV performance
Currently dead genre… or not?
Rez – Tetsuya Mizuguchi – 2001
Inspired by a techno street parade
Thumper – Drool – 2016
beyond the tunnel/rail gameplay
Soundself – VR based guided meditation
Interactive/dynamic Music Videos
Installation / Digital Art
Country Ball is the attempt to recreate a home video from the late 80’s of my family’s mother’s day cookout. My process involved looking for 35 of my mothers drawings that illustrated outdoor recreational utilities, then I trace my mothers drawings by hand onto the computer, import them into a 3D program: build them to create a computer generated landscape. I perform in front of the camera and green screen 100 times; later inserting those videos into the virtual space to create a Hieronymus Bosch “Garden of Earthly Delights” inspired landscape. This gesture is an attempt to use drawing, performance and technology as a device to translate and document a personal mythology.
Re-Animated by Jakob Steensen, 2D video/VR work. “RE-ANIMATED is based on the last Kaua’i ʻōʻō bird that died in 1987, marking the extinction of its species. In 2009, its mating call – first recorded in 1975 and later digitized in an ornithology lab in New York – was uploaded to YouTube. Since then, the song of the Kaua’i ʻōʻō calling in vain to a mate who was no longer there has been played more than half a million times. RE-ANIMATED is a response to this mating call. It is a magnificent reanimation of the bird and its song, inhabiting a distorted digital reconstruction of the Kaua’i ʻōʻō’s original habitat on the Hawaiian island of Kaua’i.”
Built around a song, album, or composition
Band/promo games used to be more common than you may expect
Love Birds, Night Birds, Devil-Birds by cassie mcquater
Not interactive but shot within an engine in real time.
“Lately, I’ve been working with 3d animation using Maya and the unreal game engine. I think creating animation in a game engine gives you a special malleability in your work, stemming from being able to work in an environment that is already rendered, alive, textured, lit up. It gives what you’re doing at the moment more opportunity for mistakes. It illuminates directions that wouldn’t ordinarily have been taken if you were rendering out every frame through an ordinary 3d renderer.”
Dynamic sound as procedural music generation
By Queasy Games (Jessica Mak) who also made sound shapes
AV creation tools, sound toys that are almost instruments. See Kate Compton’s concept of casual creator
Bloom by Brian Eno, Peter Chilvers – Initial release 2008
Panoramical – Fernando Ramallo