The art of digital synaesthesia

Artist and researcher Mitchell Whitelaw wrote an interesting and in-depth article on the links between audio-visual fusion art and synaesthesia for the Senses and Society journal. Whitelaw has just put the piece online, has illustrated it with embedded videos of some of the stunning pieces he references, but also discusses the neuroscience of synaesthesia with considerable care and insight.

In the age of ubiquitous digital media, synesthesia is everywhere. In human, neurological form, it is rare: for perhaps three in a hundred people, a stimulus in one sensory modality automatically induces a sensation in another. Auditory-to-visual synesthesia, or ‚Äúcolored hearing‚Äù is much rarer still. Yet now this phenomenon is realised, apparently, inside every digital music player, on VJ screens in every club, in robot lightshows. On these screens sound is transformed into visual pattern and form instantly and automatically; an exotic perceptual phenomenon becomes a technically mediated commonplace…

Synesthesia is widely used as an analogy around this work. The analogy provides a mapping that aligns subjective sensation with audiovisual signals; it maps perceptual or even neurological structures onto technical structures. The analogy also plays another role, foregrounding sensation in the reception of the artworks; proposing to operate, for the subject, at the level of direct sensation…

This paper‚Äôs main aim is to test this analogy, and the related historical drive that Strick suggests; to consider if, and how, such practice can be thought of as synesthetic, and examine structural parallels between synesthesia as a perceptual and neurological phenomenon, and the automatic or transcoded linking of audio and visual media…

The article is quite dense in places but well worth the effort as it carefully picks out whether these digital artworks tell us anything about synaesthesia or are just dropping neurological buzz words to sound cutting-edge.

BTW, the image is a still from a fantastic piece by artist Robert Hodgin which is embedded in the article, but which you can also view here.

Link to ‘Synesthesia and Cross-Modality in Contemporary Audiovisuals’ (thanks Alex!).