TLDR: A new study systematically evaluated 22 AI methods for detecting “Out-of-Distribution” (OoD) data in plankton recognition, which is crucial for identifying unknown species or noise in marine monitoring. Using a new benchmark with various OoD scenarios, the ViM method emerged as the top performer, especially for very different unknown data, providing a practical guide for selecting robust AI algorithms in marine ecology.
Plankton, the tiny organisms drifting in our oceans, are vital for marine ecosystems, global carbon cycles, and supporting marine life. Monitoring their species, numbers, and distribution helps us understand marine health and even predict climate change. In recent years, automated plankton recognition using underwater imaging and deep learning has become a key tool for this monitoring.
However, these automated systems face a significant hurdle: encountering “Out-of-Distribution” (OoD) data. Imagine a model trained on common plankton species suddenly seeing a new, undiscovered species, or just a bubble or a piece of debris. Plankton have incredibly complex shapes, immense species diversity, and new species are constantly being found. This means models often encounter things they weren’t specifically trained to identify, leading to errors.
Traditionally, plankton recognition might treat unknown items as a single “background” class. But this “background” is vast and unpredictable. This is where Out-of-Distribution (OoD) detection comes in. It’s about teaching a model to recognize when a sample doesn’t belong to any of the known categories it was trained on.
Despite rapid progress in OoD detection methods in computer vision, the field of plankton recognition has lacked a comprehensive, large-scale evaluation of these advanced techniques. To bridge this gap, a recent study meticulously designed a new set of benchmarks to simulate various real-world scenarios of encountering unknown plankton or marine elements. This research, detailed in the paper “Benchmarking Out-of-Distribution Detection for Plankton Recognition: A Systematic Evaluation of Advanced Methods in Marine Ecological Monitoring”, is the first of its kind to systematically evaluate OoD detection in plankton recognition on a large scale.
The researchers built their benchmark using the DYB-PlanktonNet dataset, carefully dividing it into four categories:
In-Distribution (ID) data
Ecologically important species commonly found in Daya Bay.
Near-OoD data
Biologically related plankton species that are less common or less critical for routine monitoring, representing “novel-but-similar” encounters.
Far-OoD (Bubbles & Particles) data
Non-biological items like bubbles, debris, or fish eggs, simulating common noise in underwater images.
Also Read:
- New Benchmark and Framework for Marine Open-Vocabulary Segmentation
- Enhancing Arctic Sea Ice Prediction with Physics-Informed Machine Learning
Far-OoD (General) data
Completely unrelated images from general computer vision datasets (like CIFAR-10 or MNIST), to test extreme “unknown” scenarios.
They then evaluated twenty-two different OoD detection methods, categorized into distance-based, classification-based, and density-based approaches. The goal was to see how well each method could distinguish between known plankton and these various types of unknown data.
The extensive experiments revealed that the ViM method (Virtual Logit Matching) consistently outperformed other approaches across all benchmarks. It was particularly strong in identifying Far-OoD scenarios, showing significant improvements in key performance metrics. Distance-based methods like Mahalanobis also performed exceptionally well for Far-OoD (General) data, while density-based methods such as Energy and DICE were highly effective for the more subtle Near-OoD anomalies.
A key takeaway was that different detection methods specialize in different types of unknown data. Some, like ViM and KNN, showed strong overall generalization without needing extra training. Others were better suited for either very different (Far-OoD) or subtly different (Near-OoD) unknown samples. This highlights the importance of choosing the right detection strategy based on the specific characteristics of the unknown data expected in a real-world monitoring task.
This comprehensive evaluation provides a valuable guide for selecting algorithms in automated plankton recognition and lays a solid foundation for future research. It demonstrates that existing OoD detection methods can significantly enhance the reliability and safety of large-scale plankton monitoring, even when faced with diverse plankton shapes and complex environmental conditions. Future work will focus on refining these post-hoc methods, extracting even more fine-grained features to distinguish highly similar species, and developing methods for multi-label classification to handle diverse plankton communities.
