Reinvented Through Translation
{aesthetic}
Data Across
{sensory}
Mediums
Concept
A physical and digital design generator, performance piece, and future art exhibit built around cross-disciplinary questions such as: what is the connection between music, weaving, computers, and whales? What can we do to preserve, enhance, or subdue effects from the original piece in a different audio-visual/sensorial format? How do we approach transforming aesthetic inputs to outputs?
Exploration
Born out of observations made during my time living in Italy, I became fascinated by the deep connections between two of humankind’s most ancient technologies — music and weaving. As I began to research the history of these fields, the parallels between music notation and weaving notation, pianos and looms, and looms and computers became all the more apparent.
Direction
Building on my research of computing’s origins in weaving and music, as well as my exploration into the structure of whale songs, I am tinkering around the concept of transforming data across different audio/visual mediums. The first version is taking shape as a design generator that uses music as the input to drive a multitude of possible 2D designs and 3D models. Where this differs from other audio visualizations is in the flexibility a designer has to customize what aspects of music drive different visual outputs, meaning one song has the possibility to output many potential designs. And this customization allows for a deeper consideration of how to preserve meaning when the experience shifts from aural to ocular.
While I’m starting with a design generator based on music, I intend to experiment with soundscapes and other formats such as live performances and art exhibits. I hope that the transformations will enrich our experience of audiovisual/sensorial media and inspire new modes of thinking around how we approach connecting seemingly dissimilar topics.
I have already connected with musicians, technologists, and marine scientists to explore different avenues for development and collaboration. If this sounds interesting to you, please reach out!
For more detailed background information and moodboard inspiration, please continue reading below.
Historical Context
I am interested in the transformation of (aesthetic) data across different audio/visual mediums. I am inspired by deeply cross-disciplinary questions such as: what is the connection between music, weaving, computers, and whales? As it turns out…
The guy (Bouchon) who invented the first known semi-automated loom was the son of an organ maker. He applied the process of making piano rolls (think of music boxes and pegs) to a weaving loom in the early-ish 1700s. Within a hundred years, a different guy (Jacquard) perfected the process into a fully automated loom (think of punch cards). A few decades later, a different guy (Babbage) leveraged the punch card concept to represent mathematical data. And after a gal (Lovelace) suggested any type of data could be represented, it only took a few decades to develop those computers.
This origin story, which showcases the influence of technologies on other industries, inspired an interest in the transformation of data across mediums. How might we represent auditory data visually in a 2D vs 3D context? How do we consider the preservation of intent or feeling when we use different modes of perception (e.g. ears vs eyes)? Framed differently, how do we manipulate input/output to achieve different aesthetic effects or inspire intellectual stimulation?
And now, for the whales.
When the “Songs of the Humpback Whale” album was released in 1970, it inspired a wave of ocean conservation and fueled interest in marine mammals. Deep dedication and time (the songs were sometimes hours long) led a husband-wife duo to realize the whale vocalizations held a structure similar to music, and over the course of the past few decades, scientists and enthusiasts have developed music notation and other systems to catalog and visualize the patterns of their songs. Leveraging the research and wealth of data freely available (such as Google’s Pattern Radio), I’m curious to see if the data can be visualized in a different way to illuminate new insights. Perhaps combining data from different formats, like song recordings in tandem with field notes observing whale behavior, will reveal novel patterns.
Process
My research has inspired a range of art and product avenues to explore. For the initial round of trials, I am using this moodboard as a visual reference for a digital design generator prototyped in Python. I am limiting the palette to black/white/grayscale and plan to introduce more colors in subsequent phases.
Images cited here
Vision
While I have begun with 2D digital visualizations of simple songs, I plan to expand to a wider range of soundscapes and 3D models, flat weaving patterns, and sculptural pieces. My dream is to develop design generators for live performance as well as kinetic sculptures that leverage recursion to enable computer-generated loops transitioning from audio to visual and back again.
I hope the resulting media will enable us to see the underlying patterns intrinsic to audiovisual data as they are translated and transformed across different mediums. I hope they will inspire cross-disciplinary modes of thinking and encourage collaboration between seemingly orthogonal fields.