Project Results

We are excited to share that the work resulting from this experiment.com funding was published in the Computer Vision for Animal Behavior Tracking and Modeling Workshop at CVPR 2022, and was presented at the workshop poster session.

In the paper, we introduce methods for generating aggregate "ground truths" for fine-grained (i.e. multi-class) seal counting from the collected annotations, as well as a counting method that can utilize the aggregate information. Our method improves results by 8% over a comparable baseline, indicating the potential for algorithms to learn fine-grained counting using crowd-sourced supervision.

The paper can be found on ArXiv here, and a digital copy of the poster can be found here.

About This Project

Seals are important indicators for the health of our oceans, but it is often difficult to monitor their populations, especially in remote regions. Time-lapse cameras have made this monitoring more feasible, however the scale of data produced by these tools is prohibitively large. This project will enable automated AI-based analysis of this data via the publication of a large high-quality dataset of images collected of seal populations on remote islands near Antarctica over the last ten years.

Ask the Scientists

What is the context of this research?

Methods in computer vision have shown impressive results in recent years, however the range of tasks supported is still limited. Notably, much existing work deals with the detection or classification of a few objects in a scene, or counting people in a crowd. The classification of seal sex, species, and age in imagery is extremely challenging, both for scientists and automated methods, and exposes several open research challenges in computer vision. Many indications of sex or species are highly contextual, and seals are often tightly clustered into groups or may blend in with background environments. This project aims to develop a high-quality dataset and benchmark to encourage the computer vision research community to focus on these high-impact challenges.

What is the significance of this project?

Seals are sentinels of changes within their ecosystem. Because they spend the majority of their life in water and are at the top of the food chain, variation in the size or structure of their populations may be representative of larger changes to the dynamic ocean ecosystems. Measuring these changes year-round can help to better understand how threats to the ecosystem disrupt the dynamics of resident wildlife. Automated analysis would enable this research to take place on a much larger scale than is currently possible.

Within computer vision, datasets and benchmarks are crucial for pushing the field forward. This project would help to align computer vision research with the study of the natural world and encourage the development of state-of-the-art algorithms which benefit the oceans.

What are the goals of the project?

For this project we will evaluate the efficacy of using AI and computer vision to automate the analysis and classification of seals. To do this, we will first expand upon the work of Seal Watch, a citizen science project which collected nearly 200,000 annotations of seals in imagery from South Georgia and the South Sandwich Islands, to create a high-quality computer vision dataset. We will then fine-tune and benchmark existing AI algorithms for counting and classifying seals, and publish the dataset and results in order to further engage the computer vision research community on this challenge. The requested funds will support annotation labor, cloud computing and storage costs, and researcher labor. All results, data, and models will be open sourced to the public.