TLDR: A study on credit scoring models found that using ADASYN with a 1x multiplication factor (doubling the minority class) significantly improved predictive performance, achieving an optimal majority-to-minority class ratio of 6.6:1. This contradicts the common practice of aiming for a 1:1 balance, as higher augmentation levels led to diminishing returns and performance degradation. The research provides empirical evidence for an optimal “sweet spot” in data augmentation, offering practical guidelines for financial risk assessment.
Credit scoring models are vital for financial institutions, helping them assess loan applicants and manage risk. However, these models face a significant hurdle: class imbalance. In real-world scenarios, the number of defaulting customers is typically very low, often less than 10% of the total population. This severe imbalance makes it difficult for standard machine learning algorithms to accurately predict defaults, which are the very events lenders need to identify to prevent financial losses.
To combat this, techniques like SMOTE (Synthetic Minority Over-sampling Technique) and ADASYN (Adaptive Synthetic Sampling) have emerged. These methods generate synthetic data points for the minority class (defaulters), artificially increasing their representation in the training data. While widely used, a critical question has remained largely unanswered: what is the optimal amount of data augmentation? Many practitioners simply aim for a 1:1 balanced ratio, assuming more synthetic data is always better, without strong empirical evidence.
Uncovering the ‘Sweet Spot’ for Data Augmentation
A recent study, titled “Finding the Sweet Spot: Optimal Data Augmentation Ratio for Imbalanced Credit Scoring Using ADASYN” by Luis H. Chia, systematically investigated this question. The research aimed to evaluate different data augmentation scenarios to find the most effective strategy for imbalanced credit scoring. The study utilized the ‘Give Me Some Credit’ dataset, which contains 97,243 observations and a 7% default rate, representative of real-world consumer credit portfolios. The models were trained using XGBoost, a powerful machine learning algorithm, and evaluated on a separate, untouched test set of 29,173 real observations. Statistical significance was rigorously assessed using bootstrap testing with 1,000 iterations.
The key findings of this research challenge conventional wisdom. The study revealed that ADASYN with a 1× multiplication factor (effectively doubling the minority class) achieved optimal performance. This configuration resulted in an AUC (Area Under the Receiver Operating Characteristic Curve) of 0.6778 and a Gini coefficient of 0.3557. These figures represent statistically significant improvements of +0.77% in AUC and +3.00% in Gini coefficient compared to the baseline model (p=0.017, bootstrap test).
The Law of Diminishing Returns
Crucially, the study demonstrated a “law of diminishing returns” for synthetic oversampling. Higher multiplication factors, such as 2× and 3×, actually led to a degradation in performance. For instance, the 3× augmentation showed a -0.48% decrease in AUC. This suggests that while some augmentation is beneficial, excessive synthetic data can introduce noise, distort the true data distribution, and lead to overfitting to synthetic patterns rather than learning generalizable real-world relationships.
Another significant revelation was the optimal class imbalance ratio. The best-performing configuration maintained a majority-to-minority ratio of approximately 6.6:1, directly contradicting the common practice of balancing the dataset to a 1:1 ratio. This finding has substantial practical implications, guiding practitioners away from aggressive oversampling that can be counterproductive.
ADASYN’s Adaptive Advantage
When comparing different augmentation techniques at the optimal 1× multiplication factor, ADASYN consistently outperformed SMOTE and BorderlineSMOTE. ADASYN’s adaptive density-based weighting, which generates more synthetic samples in regions where the minority class is harder to learn, proved to be more effective than SMOTE’s uniform interpolation. BorderlineSMOTE, which focuses on instances near the decision boundary, showed intermediate performance, further supporting the idea that targeted augmentation is superior.
Also Read:
- Advancing Financial Inclusion for Malaysian MSMEs with Transactional Data
- Unlocking Better Generalization in Small Language Models Through Pattern-Guided Data Augmentation
Practical Takeaways for the Industry
The research provides clear, actionable guidelines for industry practitioners and researchers working with imbalanced datasets, particularly in credit scoring:
- For credit scoring tasks with approximately 7% default rates, use ADASYN with a 1× multiplication factor.
- Aim for a class ratio of roughly 6-7:1 (majority:minority) instead of a 1:1 balance.
- Avoid excessive hyperparameter tuning, as aggressive optimization can lead to severe overfitting and degrade performance on real test data.
- Do not mix multiple augmentation techniques, as ensemble approaches combining different generation strategies were found to perform worse than the single best method.
While this study was conducted on a single representative dataset, its rigorous methodology provides a reproducible framework for determining optimal augmentation ratios in other imbalanced domains. The findings suggest that finding the “sweet spot” in data augmentation is not just an academic exercise but a practical necessity for building robust and generalizable models in real-world applications. For more details, you can refer to the full research paper here.
