TLDR: This research introduces a novel method for classifying fish freshness using handcrafted features extracted from fish eye images. By systematically fusing color statistics, percentiles, histograms, and texture features (LBP, GLCM), the approach achieves high accuracy with classical machine learning models. The LightGBM classifier reached 77.56% accuracy in a standard setting, and an Artificial Neural Network achieved 97.16% with augmented data, significantly outperforming previous deep learning baselines. The study also found that contextual information from surrounding tissues is crucial for accurate classification, as segmenting the eye alone reduced performance. This method offers an interpretable and reliable solution for automated fish freshness assessment in the food industry.
Ensuring the freshness of fish is a critical challenge for the food industry, impacting everything from product quality and market value to consumer health. Traditional methods, which often rely on human sensory evaluation, are notoriously subjective, inconsistent, and difficult to standardize. These limitations highlight a pressing need for more objective and automated approaches to assess fish freshness.
A recent study by Phi-Hung Hoang, Nam-Thuan Trinh, Van-Manh Tran, and Thi-Thu-Hong Phan introduces an innovative method that leverages carefully designed ‘handcrafted features’ from fish eye images to classify freshness levels with remarkable accuracy. Their research, titled “Enhanced Fish Freshness Classification with Incremental Handcrafted Feature Fusion,” offers a robust and interpretable solution to this long-standing problem.
The core idea behind this research is to extract a comprehensive set of visual descriptors from fish eye images. Instead of relying on complex deep learning models that learn features automatically (and often in ways that are hard to understand), this approach focuses on features that are explicitly engineered to capture subtle signs of spoilage. These features fall into two main categories: color-based and texture-based.
Color-based features are crucial because the color of a fish eye changes as it loses freshness. The researchers extracted various color statistics, such as the average brightness, spread of colors, and the overall shape of color distribution, from multiple color spaces like RGB, CIELAB, and HSV. CIELAB is particularly interesting because it mimics human color perception, separating lightness from color components, which can be more effective in detecting subtle shifts. They also looked at color percentiles (e.g., the 5th or 95th percentile of color intensity) and color histograms, which provide a detailed distribution of colors across the image.
Texture-based features, on the other hand, capture surface irregularities and patterns on the fish eye. As fish spoil, their eyes can develop cloudiness, spots, or changes in surface smoothness. The study utilized Local Binary Patterns (LBP) to detect fine-scale variations and Gray-Level Co-occurrence Matrices (GLCM) to analyze broader spatial relationships between pixels. These texture features were specifically extracted from the ‘b*’ channel of the CIELAB color space, which represents the blue-yellow spectrum and is closely linked to biological decay.
A key innovation of this research is its ‘incremental feature fusion’ strategy. Rather than simply combining all extracted features at once, the researchers systematically added and evaluated groups of features in a stepwise manner. This allowed them to identify which combinations were most effective and complementary, ensuring that the final set of features was both compact and optimized for classification accuracy. For instance, they found that combining color statistics from BGR and CIELAB spaces, then adding color variance ratios, and finally incorporating GLCM and LBP features, progressively improved performance.
Once the optimal set of handcrafted features was identified, various classical machine learning models were trained and evaluated. These included Logistic Regression, K-Nearest Neighbors, Support Vector Machines, Artificial Neural Networks (ANN), Random Forests, Extra Trees, LightGBM, and CatBoost. The selection of these models was based on their ability to handle structured data and capture both simple and complex relationships.
The experimental results were highly impressive. In a standard training and testing setup, the LightGBM classifier, using the proposed handcrafted features, achieved an accuracy of 77.56%. This represents a significant improvement of over 14% compared to a previous deep learning baseline of 63.21% on the same Freshness of the Fish Eyes (FFE) dataset. Furthermore, when data augmentation was applied, an Artificial Neural Network (ANN) reached an astounding 97.16% accuracy, surpassing the prior best of 77.3% by nearly 20%.
Interestingly, the study also investigated the impact of segmenting the fish eye from its surrounding tissues. The researchers found that removing the background actually led to a substantial decline in classification accuracy. This suggests that subtle cues from the surrounding skin and scales, such as gradual color transitions or fine textures, provide important contextual information that is crucial for accurate freshness assessment. Their removal cannot be fully compensated by focusing solely on the isolated eye region.
While some handcrafted features like color statistics and GLCM were computationally intensive, others like color histograms and variance ratios were much faster. The final best-performing model successfully balanced these factors to achieve optimal accuracy. This research demonstrates that a well-designed, knowledge-driven approach using handcrafted features can not only compete with but often outperform complex deep learning models for specialized tasks like fish freshness assessment. It offers a robust, interpretable, and reliable solution with significant practical implications for food quality monitoring.
Also Read:
- New Benchmark and Framework for Marine Open-Vocabulary Segmentation
- Efficient AI for Super-Resolution Microscopy: Transforming Images and Validating Experiments
For more detailed information, you can refer to the full research paper available here.
