About This Project

Advances on audio ML have spurred improvements in bioacoustics and open possibilities for automatically detecting and decoding the meaning of animal calls (1, 2, 3). However, state of the art models have not seen the same successes on the full range of vocalizations produced by various species, in particular the Egyptian fruit-bat. By leveraging the challenge of decoding a bat repertoire, we will develop more holistic models.

Ask the Scientists

What is the context of this research?

The application of machine learning in bioacoustic research hasled to significant breakthroughs in our understanding of animal behavior, ecology, and even conservation efforts. Traditional bioacoustics often involves manual tagging and categorization of all vocalizations, which necessitates expert-level knowledge of the species’ vocal behavior and can be both time-consuming and subject to human error. Deep neural networks can efficiently process large audio datasets, including unfamiliar data, reducing the need for manual curation. As such, our understanding of many species' vocal patterns is limited (1, 2, 3). Current models, designed mainly for human speech, often miss nuances in animal vocalizations. An unsupervised, broader framework to decode animal "words" (call types) is crucial.

What is the significance of this project?

Integrating expert traditional bioacoustics knowledge with speech processing ML, we aim to create an unsupervised model that can discern call-types across animal vocal repertoires. Such advances would unlock our ability to understand intricate animal communication, and develop more effective conservation strategies (1, 2, 3). Validating this model using a songbird species with a well-known vocal repertoire and structure will not only attest to its accuracy, but may also highlight subtle vocal structures previously missed by supervised analyses. Finally, our unsupervised model will be used to reveal the vocal structure and ontogeny of a bat species for which we have extensive datasets. Using data from both adults and young bats, we will be able to identify how bats learn their own language.

What are the goals of the project?

Building on the HuBERT-based AVES model and inspired by non-parametric instance discrimination tactics, our first objective is to develop an unsupervised model architecture for call-type discovery. By incorporating feature spaces specific to species and harnessing data augmentation, we intend to create a relevant latent space for pinpointing animal vocal patterns. Next, our model will be tested and refined using a zebra finch dataset with vocalizer and call-type annotations (1, 2). Finally, we aim to adapt our model to explore the vocal repertoire of the Egyptian Fruit Bat. Using our extensive dataset of freely interacting bats (young and adults identified through audio-data loggers), our focus is discerning call-types and how they emerge over development in that species.