Listening to whales: A multimodal approach using hydrophones, drones, and AI to unravel killer whale communication

$5,650
Goal
Approved
  • $0
    pledged
  • 0%
    funded
  • Private
    Not Launched

About This Project

This collaborative project pioneers a multimodal approach to decode orca communication by developing research protocols that combine drone video, hydrophone recordings, and AI, applicable across populations. We ask how calls coordinate behaviours and relationships, predicting that specific call patterns correspond to distinct behavioural contexts and are altered under elevated noise conditions. This work will reveal which behaviours are most disrupted by underwater noise.

Ask the Scientists

Join The Discussion

What is the context of this research?

Orcas are among the most socially complex mammals on the planet. Call repertoires differ between populations and vocal traditions are passed down through generations, an extraordinary example of animal culture. Despite decades of research, we still don’t know what many sounds mean or how whales use them to coordinate activities like hunting and prey sharing, or to maintain social bonds. While most work has focused on how noise interferes with echolocation, relatively little is known about how it disrupts the communication that underpins cooperation and culture. Our project brings together marine biologists and AI researchers to pair synchronized drone footage, underwater recordings, and advanced analytical tools to reveal how orcas use sound to share information and maintain complex relationships in a noisy ocean. We will test the hypothesis that specific call patterns correspond to distinct social and behavioural contexts and are modified when noise interferes with communication.

What is the significance of this project?

Acoustic communication is the glue that holds killer whale societies together. By revealing how these culturally diverse whales use sound to coordinate, share food, and care for one another, this project will transform our understanding of one of the ocean’s most socially and cognitively sophisticated species. It also has critical conservation implications: noise from ships can mask or disrupt key social calls, undermining cooperation and survival. Linking specific calls to behaviours will help us identify which aspects of killer whale communication are most sensitive to disturbance and design evidence-based strategies that reduce noise impacts and protect the social bonds and cultural traditions that define their societies. In doing so, this work contributes to the growing conversation around more-than-human personhood, showing that species like killer whales possess communication systems that reflect advanced cognition and deep social connection.

What are the goals of the project?

Our goal is to understand how killer whales communicate and how noise pollution affects their communication. We’ll combine drone video, underwater recordings, and field observations to link sounds with behaviours and social interactions. Using AI tools, we’ll analyze these synchronized datasets to detect patterns that reveal how they organize and use their calls. We’ll then apply these models to , examine sequences and potential syntax to generate new hypotheses about what call combinations convey and how they function. Ultimately, we aim to transform how we study and protect killer whales. Our long-term goal is to develop standardized research methods that scientists worldwide can use to study communication in other killer whale populations. The species is highly cultural, and studying them across regions greatly further our understanding of the diversity and commonalities of their vocal systems, and how they adapt to changing environments.

Budget

Please wait...

This funding would support a portion of the annotation of acoustic and aerial datasets collected during our successful 2025 field pilot. This is an essential step toward producing the high-quality training set needed to develop machine-learning models that detect and classify orca calls. ESP, our primary partner, has already contributed $15K toward these annotations, underscoring the collaborative investment in this foundational stage. Without this work, subsequent analyses and AI development cannot proceed.

This $5K contribution will help bridge immediate funding gaps, ensuring momentum between field seasons, critical to advance this multi-year project. Additional complementary funding is being sought through other sources to complete the full annotation phase and advance 2026 fieldwork in BC and Iceland, and analysis.

Project Timeline

This funding will support the first phase of annotation and analysis of multimodal datasets collected during the 2025 pilot field season. Work will focus on aligning acoustic and drone data, conducting quality checks, and generating ground-truthed, labelled training sets for machine-learning development that will enable comparative analyses. Deliverables will directly inform the 2026 fieldwork phase.

Dec 20, 2025

Finalize alignment of 2025 acoustic and drone datasets; establish synchronization workflow.

Jan 15, 2026

Conduct preliminary quality checks and adjustments; begin systematic annotation of paired behavioral and acoustic data sets using Boris and Raven.

Feb 28, 2026

Generate ground-truthed, labelled training sets for machine-learning development and share preliminary outputs  with collaborators.

Apr 30, 2026

Share final labelled data sets,  visualizations and insights with collaborators to guide 2026 fieldwork.

Meet the Team

Valeria Vergara
Valeria Vergara
Dr.

Affiliates

1. Raincoast Conservation Foundation 2. University of Windsor
View Profile

Team Bio

Dr. Amy Rowley's Dr. Amy Rowley and Marie-Ana Mikus lead the data annotation and analysis for this project. Amy, an experienced behavioural ecologist, brings over a decade of fieldwork with Northern Resident killer whales, guiding behavioural interpretation and ethogram design. Marie-Ana contributes five years of drone-based behavioural analysis and ten years of acoustic expertise across cetaceans and otters.


Valeria Vergara

I am a behavioural ecologist and bioacoustician fascinated by how animals use sound to navigate their social and ecological worlds, and what that reveals about their intelligence, culture, and resilience.

For over two decades, I have studied the communication systems of cetaceans and the impacts of underwater noise on their ability to communicate effectively, with a primary focus on beluga whales and later on killer whales. My doctoral research at the University of British Columbia was the first to document how beluga calves develop their vocal repertoires and to identify individually distinctive contact calls used for mother–calf contact and group cohesion, vital in the underwater world where vision is limited. Since then, I have led long-term studies integrating underwater acoustics, behavioural observations, and, more recently, drone-based aerial imagery to understand how whales use sound to maintain social bonds, coordinate behaviour, and navigate increasingly noisy oceans.

My fieldwork has taken me from the Canadian Arctic and Hudson Bay to the St. Lawrence Estuary, British Columbia’s coastal waters, Patagonia, and Colombia, studying belugas, narwhals, killer whales, humpbacks, and Guiana dolphins. This broad comparative perspective, spanning species, habitats, and conservation contexts, has deepened my understanding of how acoustic communication reflects both ecological pressures and social complexity.

I co-direct the Cetacean Conservation Research Program at Raincoast Conservation Foundation and serve as an Adjunct Professor at the University of Windsor, roles that combine field research, mentorship, and collaborations with academic, Indigenous, government, and NGO partners. My research blends behavioural ecology, acoustics, and conservation science to inform strategies that reduce underwater noise and protect the acoustic habitats that whales depend on.

Lab Notes

Nothing posted yet.

Additional Information

This synchronized clip was filmed and recorded on Aug 31 2025 during our pilot, showing 4 Northern Resident A50 matriline individuals, with clear bubble-streams revealing who is vocalizing. Fainter calls come from three additional pod members 800 meters ahead.

This synchronized clip was filmed and recorded on Sept 3 2025 during our pilot, showing five Bigg’s killer whales from the T109B matriline sharing a sea lion they recently captured. Notable variable whistles are heard throughout the coordinated feeding event.

This figure integrates multiple data streams (drone flights, voice notes, and hydrophone recordings) across the pilot season, showing when and how intensively different modalities overlapped, and illustrating the volume of data currently under analysis.


Project Backers

  • 0Backers
  • 0%Funded
  • $0Total Donations
  • $0Average Donation
Please wait...