Collective Pulse: Listening to the Natural World
Jon and Diego setting up bird spy microphones (Audiomoths and raspberry-Pi with microphones attached) at the Yale Farm in New Haven, Connecticut. This allowed us to listen to and identify birds during the annual spring migration season. Left image courtesy of Diego Ellis and right image courtesy of Vicky Blume.
Humans have always been fascinated and inspired by nature and how species interact with each other, particularly large aggregations of animals. Take a flock of birds flying in unison, or a school of fish. Their collective movement appears synchronous and harmonic. Until about a hundred years ago it was not uncommon to believe that animal collectives were communicating through telepathy. Today, the study of how animal collectives sense the environment and interact with each other is a burgeoning research field.
My (Diego’s) fascination with understanding how animals navigate environments across their journeys has turned into a profession through my PhD research in Ecology & Evolutionary Biology at Yale, affiliated with the Center for Biodiversity and Global Change. As a music producer and DJ, I also incorporated sounds of animals and ecosystems from my research travels into my songs to create place-based electronic music, such as a song featuring birds from the amazon and a song about migratory wildebeest crossing the Mara river.
Through a robust collaboration with colleagues at Yale, we have developed a public-facing project that integrates music composition, music theory, electrical engineering, and environmental science. My collaborator Matthew Suttor is a professor in the Yale School of Drama who has been working on the intersection of musical composition and animal sounds (listen to Matthew creating a song with crying baby mice). We quickly connected and decided to collaborate given our mutual obsession with collective animal behavior and its similarities to musical theory! As I explained to Matthew, pigeons in a flock don’t want to be too far apart or too close together, and they all want to move at similar speeds. “Nature is musical; those are all rules of harmony!” Matthew excitedly said to me. We began exploring the sounds that can emerge from the collective movement and orientation of fish and pigeons. We were lucky enough to find Jonathan Koss, a PhD candidate in electric engineering at Yale, to aid us in incorporating cutting edge technology, such as mini computers and computer vision that can identify animal species by images or sounds using deep learning.
Our project, currently coined ‘Collective Pulse,’ aims to uncover hidden dimensions of nature by utilizing advances in technology and artificial intelligence. We have recently showcased our work in two art galleries and presented our work as a recorded scientific talk. Our vision is to create a global collective of musicians, artists, naturalists, and scientists working together to sonify the natural world in new ways. Listening to both the silence and sounds of the natural world is primordial to human existence, and yet, something that is rapidly being threatened by human activities. We hope to bring the natural world to people's ears, eyes, and hearts through this project.
Recently, we showcased our project at two art galleries in New Haven, featuring work that blends animal-generated art and music, including a collection of artwork created by local school children and artists as well as a demo of pigeons singing to gregorian chants and termites learning how to jam. We emphasize that we did not train the animals to perform music! Rather, we use artificial intelligence and biological and musical theory to allow music to be created out of the data captured by animal collectives. As a biologist studying the ecology and evolution of species, it has been rewarding working in the context of environmental humanities. When communicating science, I have learned that a picture can indeed be worth more than a thousand words, but that sounds can be worth a thousand pictures. Collaborating with artists, musicians, and designers has taught me the importance of avoiding jargon (as we all may mean different things with the same words), that exploratory research and curiosity fosters creativity, and that having an open mind to sit back and let other people explore their ideas in the context of this project can be extremely rewarding. It also taught me about the subjectivity of interpretation when looking at data-generated figures in terms of the emotions that they may evoke on the observer. One of my favorite memories when installing this art gallery was the amount of people collaborating on this project: staff at a cancer hospital providing us with origami paper cranes and painted rocks with messages around the beauty of nature; artists generating art work for this exhibit; scientists generating figures we call static art; and musicians creating live music to generate a collective pulse among the audience.
What is next for us? We recently installed bird spy microphones on an urban farm and made farm music, and played our music, including trees breathing in carbon at an electronic music festival in Durham, North Carolina. We are also exploring how to create more sophisticated translations from nature to musical sounds by taking a deep dive into music theory and biological principles. Finally, and most importantly, we are exploring how to teach this to the next generation of environmental humanists, scientists, artists and musicians.
Thank you for listening!
Diego, Matthew & Jonathan
Diego Ellis Soto: Diego is a Uruguayan PhD candidate in Ecology at Yale University and a NASA FINESST fellow. Working at the intersection of technology and conservation, he researches how animals move across the world under increasing human threats and a changing climate. As a music producer, Diego blends sounds from his research travels, biological concepts and technologies, with electronic music, for example by making eight termites jam together or migrating wildebeest tuning their hooves to electronica!
Matthew Suttor: With a background in computer music and over twenty years of experience teaching at Yale University, Matthew’s operas, theatre, and dance works have been presented at festivals and galleries such as Bard SummerScape, BAM Next Wave Festival, Guggenheim Works & Process, New Zealand Festival of the Arts, and the Mozart Prague Festival, and his concert, film, and TV music has been performed, screened, and broadcast throughout Australasia, Europe, and the Americas.
Jonathan Koss: Jonathan is a PhD candidate in electrical engineering at Yale University focusing on machine learning research for brain imaging and medical applications. Outside of this, he enjoys bringing AI and engineering to conservation and environmental projects such as with Collective Pulse.