To remove the barriers between sight and sound, between the seen world and the heard world! To bring about a unity and a harmonious relationship between these two opposite spheres. What an absorbing task! The Greeks and Diderot, Wagner and Scriabin – who has not dreamt of this ideal? Is there anyone who has made no attempt to realise this dream? - Sergei Eisenstein 
The word synesthesia stems from the ancient Greek words σύν (syn), "together," and αἴσθησις (aisthēsis), ‘sensation’. Traditionally, (in the field of neuropsychology) synesthesia refers to an anomaly in the human brain (estimates vary between 1 in 20 and 1 in 20000) that results into the transference of an experience by one sense into another. This means that people with synesthesia can, involuntary, for instance ‘see’ sounds, ‘taste’ shapes, and so on. Although synesthesia has been the subject of intense scientific investigation during many centuries, modern researchers have only recently rediscovered the concept. Today, both artists and scientists research synesthesia and the rules that “should be applied to transform sound into image, spatial movement into timbre or harmony into color." Because synesthesia is a highly subjective matter that only relies upon what is referred to as an anomaly, scientists and artists have not been able to trigger the effect purposefully. Consequently, in the arts the concept of synesthesia has evolved into a metaphor that relates to the act of crossing boundaries between different disciplines or fusing different media into one work. An example of this is the integrated, ‘complete artwork’, which originates from 1849, when the German opera composer Richard Wagner first described his Gesamtkunstwerk. For Wagner, the individual arts of painting, dance, music and poetry had progressed as far as they could. He described that the future of the arts lied within the Gesamtkunstwerk, a synthesis of the poetic, visual, musical and dramatic arts. This total work of art would speak to all the senses at once and result in total immersion.
A Color Box (Len Lye, 1935) After 40 years of more or less successful attempts to record sound and moving image synchronised, it was Len Lye who finally set out to prove once and for all that music and video together could be part of the language of art. Lye is known to have stated: "All of a sudden it hit me. If there was such a thing as composing music, there could be such a thing as composing motion." Driven by a lack of money, Lye started off inventing new, controversial ways to create film without a camera. In 1935, he made the experimental movie A Colour Box. In this movie he color-painted and scratched the celluloid, connecting the visuals to the tunes of a Caribbean jazz piece. In 1937 A Colour Box was acquired by the General Post Office, who reissued it with the addition of a 'cheaper parcel post' message. The video continued its existence as a commercial. Today, Len Lye is known as one of the pioneers of the music video genre. Another telling example of cross modal practice finds its origin in the 1960s, when the artist Dick Higgins, a founder of the Fluxus movement, became well known for promoting interdisciplinary art practice. Following the example of Marcel Duchamp, Higgins calls for unusual combinations of genres. He popularises the term intermedia, which stands for media that investigate the border of multiple art practices. According to Higgins, in the realm of intermedia, any available object or experience can and should be incorporated into the artwork by mapping one structure onto another (in the digital sometimes referred to as transcoding). As a result, these intermedia artworks act in unpredictable ways, producing a new, more complex whole than its original parts. Variations V (John Cage and Higgins, 1965) In 1965 John Cage answered to Higgins’ call for intermedia by composing the performance Variations V (with the help of David Tudor, Billy Klüver and others). As most of Cage’s works, the piece revolved around a central concept of chance. The play was generated on the spot, from thirty-five randomly equipped ‘remarks’ that outlined its structure, components, and methodology. The performance’ score was created within two different sound systems. The first sound system was dependent on directional photocells (light sensors) that were connected to an ‘orchestra’ of tape-recorders and record players. As the dancers moved around, they interrupted the light falling on the photocells, which switched the sound on and off. The second system used a series of antennas. When a dancer came within four feet of an antenna, this would initiate sound stemming from short-wave radios. In the background of the performance, screens would show a movie by Stan VanDerBeek and manipulated film footage from Nam June Paik. Cage’s performance Variations V united technology, dance, video and sound, mapping their structures on top of each other, while leaving them dependent on chance. In doing so he created a noisy, multilayered synesthetic whole, that incorporated a cacophony sounds and visuals that stretched the limits of what was understood by the audience.
Advanced Beauty, Sample clip (flight404, 2007). (one the most beautiful audiovisual transcodings I know, yet missing some kind of conceptual relation)