TLDR: This research addresses vehicle type recognition challenges due to skewed datasets by creating a 16-class corpus and balancing it with SMOTE oversampling and undersampling. It compares lightweight ensembles (Random Forest, AdaBoost, Voting Classifier) with a ResNet-style CNN. While the CNN achieved higher accuracy (79.19% test, 81.25% unseen), deep models still struggled with extremely rare classes like “Barge,” highlighting the need for more minority-class data and advanced loss functions.
Accurate identification of vehicle types is crucial for modern intelligent transportation and logistics systems. However, a significant challenge in this field is the severe class imbalance found in public datasets, where common vehicle types are abundant, while rare categories are scarcely represented. This imbalance often leads to poor performance in recognizing these less common vehicles.
To tackle this problem, researchers curated a comprehensive 16-class vehicle image corpus, comprising approximately 47,000 images. This dataset was created by merging images from Kaggle, ImageNet, and web-crawled sources. To address the inherent class imbalance, six balanced variants of the dataset were generated using techniques like SMOTE (Synthetic Minority Over-sampling Technique) and targeted undersampling.
The study then benchmarked two main approaches: lightweight ensemble models and a deep learning model. The ensemble models included Random Forest, AdaBoost, and a soft-voting combiner, all built upon features extracted using MobileNet-V2. These were compared against a configurable ResNet-style Convolutional Neural Network (CNN) that was trained with strong data augmentation and label smoothing.
Ensemble Learning Approach
For the ensemble models, MobileNet-V2 served as a feature extractor. The researchers applied SMOTE to boost underrepresented classes and undersampling to reduce dominant ones, creating various training set configurations. Through extensive hyperparameter tuning using grid search, the Random Forest model performed best when trained on the ‘smote_combined’ dataset variant, which benefited from both improved class balance and diverse external data. Similarly, the AdaBoost classifier, after multiple stages of grid search, achieved its highest accuracy on the ‘smote’ dataset, with the ‘smote-combined’ variant chosen for the final model. However, AdaBoost still struggled with extremely underrepresented classes such as Barge, Cart, and Limousine.
A Voting Classifier was then implemented, combining the best-performing Random Forest and AdaBoost models. This ensemble demonstrated more stable results across different datasets, mitigating individual model biases. When tested on an unseen set of 16 EE531 inference images, the Voting Classifier achieved an overall accuracy of 75%, correctly predicting 12 out of 16 samples. Despite this, misclassifications still occurred in visually similar classes like Cart and Boat.
Deep Learning Approach with ResNet-style CNN
The deep learning approach utilized a ResNet-inspired CNN architecture. The dataset for the CNN combined all three data sources and underwent extensive preprocessing, including normalization, resizing, and advanced data augmentation techniques like random horizontal flipping, cropping, color jittering, and random erasing. The CNN architecture was modular and flexible, allowing adjustments to its depth and complexity.
The training involved two phases. The first phase explored 18 different CNN configurations, identifying three strong performers. In the second phase, these selected models were refined with data augmentation and label smoothing, and two deeper architectures were introduced, including one mimicking ResNet-101. The ‘best_model’ checkpoint, derived from an extended training of the ResNet-101-like architecture, emerged as the top performer. This model showed steady improvement and strong convergence without significant overfitting.
On the full test set, the best CNN model achieved an accuracy of 79.19%. Furthermore, when evaluated on the same unseen EE531 inference batch, it reached an impressive 81.25% accuracy, correctly predicting 13 out of 16 samples. This confirmed the advantage of deep learning models in overall performance. However, a critical finding was the CNN’s complete failure on the ‘Barge’ class, achieving 0.00% accuracy. This starkly highlighted the persistent challenge posed by extreme class imbalance, even for highly optimized deep learning models.
Also Read:
- FedCLF: Enhancing Federated Learning for Connected Vehicle Networks
- Evaluating V2X Cooperative Perception Systems: Performance, Errors, and Vulnerabilities
Conclusion and Future Directions
In conclusion, this study underscores that a combination of careful data balancing and complementary model strategies can significantly enhance multi-class vehicle recognition. While the deep learning CNN model generally outperformed classical ensembles, the most underrepresented class, ‘Barge’, remained a significant failure point for all models. This indicates that rebalancing alone has its limits when facing extreme data scarcity.
The findings suggest that future research should prioritize collecting additional samples for minority classes, exploring cost-sensitive objectives like focal loss, and investigating hybrid ensemble–CNN pipelines. Such hybrid approaches could combine the interpretability of shallower models with the powerful representational capabilities of deep networks to achieve more robust and balanced performance across all vehicle categories. For more details, you can refer to the full research paper here.
