TLDR: A new framework, Multi-Granularity Knowledge Distillation (MGKD), significantly improves financial risk forecasting for new applicants by integrating historical in-service user behavior data into pre-service prediction models. It uses a teacher-student model approach with coarse-grained, fine-grained, and self-distillation strategies to transfer knowledge, leading to more accurate risk assessments. Tested on real-world datasets from Tencent Mobile Payment, MGKD demonstrated superior performance and robustness in identifying potential defaulters before service activation.
Financial institutions constantly strive to manage risk effectively, a process that typically involves two distinct stages: assessing risk before a service is activated (pre-service) and detecting potential defaults once the service is in use (in-service). Traditionally, these two phases have been modeled separately, often overlooking the valuable insights that in-service user behavior data could offer to improve early risk predictions.
A new research paper introduces a novel framework called Multi-Granularity Knowledge Distillation (MGKD) that aims to bridge this gap. The core idea is to enhance the accuracy of pre-service risk prediction by intelligently incorporating historical in-service user behavior data. This approach is particularly crucial for financial platforms that often deal with limited information about new applicants.
How MGKD Works
The MGKD framework operates on the principle of knowledge distillation, a technique where a ‘teacher’ model guides a ‘student’ model. In this context, the teacher model is trained using comprehensive historical in-service data, which provides a rich understanding of user behavior after service activation. This teacher model then shares its ‘knowledge’ with a student model, which is trained solely on pre-service data – the kind of information available for new applicants.
By using ‘soft labels’ derived from the in-service data, the teacher model helps the student model refine its risk predictions even before a service is activated. This means the student model learns from the patterns and behaviors observed in users who have already used the service, making its early assessments more informed.
The framework employs a multi-granularity distillation strategy, which involves transferring knowledge at different levels:
- Coarse-Grained Distillation: This focuses on aligning the overall representations or ‘understandings’ of users generated by both the teacher and student models. It ensures that the student model develops a similar high-level perspective on user characteristics as the more experienced teacher model.
- Fine-Grained Distillation: This level concentrates on aligning the actual risk predictions. The teacher’s nuanced predictions help the student model calibrate its own output, leading to more accurate probability assessments of default.
- Self-Distillation: To further enhance the student model’s performance, it also learns from its own past predictions during training, refining its understanding over time.
This comprehensive approach not only strengthens the model’s ability to represent potential defaulters but also facilitates the transfer of key behavioral patterns associated with them from the in-service teacher to the pre-service student model. Additionally, the framework includes a re-weighting strategy to address the common challenge of class imbalance, where default cases are a minority, ensuring the model doesn’t overlook them.
Also Read:
- Enhancing Traffic Prediction with Human Knowledge: A New Cross-Modal Fusion Model
- zkUnlearner: Advancing Verifiable Machine Unlearning with Precision and Security
Real-World Impact
The effectiveness of the MGKD framework was rigorously tested using large-scale, real-world datasets from Tencent Mobile Payment. The experimental results demonstrated significant improvements in key financial risk metrics like AUC, KS, and Recall@10 in both offline and online scenarios. This indicates that the framework is not only powerful in a controlled environment but also robust and generalizable when deployed in dynamic, real-world conditions with evolving user behaviors.
The study also included an ablation analysis, which confirmed that each component of the multi-granularity distillation strategy contributes positively to the overall performance, highlighting their complementary roles. Furthermore, a sensitivity analysis on hyperparameters showed that carefully tuning the distillation intensities is crucial for optimal results, as too much or too little knowledge transfer can impact performance.
In conclusion, the MGKD framework offers a practical and effective solution for financial institutions to enhance their pre-service risk assessment capabilities by intelligently leveraging in-service user behavior data. This integration provides a more comprehensive understanding of user risk, leading to more informed and accurate financial decisions. You can read the full paper here: Beyond the Pre-Service Horizon: Infusing In-Service Behavior for Improved Financial Risk Forecasting.
