TLDR: A study developed two AI systems for corporate credit assessment using non-financial data: a single-agent system (NAS) and a multi-agent debate system (KPD-MADS). While both significantly improved productivity over human experts, the KPD-MADS, which uses a structured debate among multiple LLM agents, demonstrated superior reasoning quality, explanatory adequacy, and practical applicability, showing that structured agent interaction can significantly improve financial AI reasoning.
In the complex world of corporate finance, assessing credit risk is a critical task. While financial numbers provide a clear picture, non-financial factors like a company’s management quality, governance, and market environment play an equally decisive role in determining loan repayment outcomes. However, these qualitative aspects are often hard to formalize and automate, making evidence-based reasoning a significant challenge for financial AI.
Traditional AI approaches in finance have largely focused on predicting outcomes based on numerical data. This leaves a gap in supporting the interpretive judgments that experienced loan evaluators make. The process of analyzing non-financial data is time-consuming and cognitively demanding for human experts, often leading to reliance on shortcuts that can compromise accuracy. Automating this analysis could bring substantial improvements to corporate loan service workflows.
Large Language Models (LLMs) have emerged as a promising tool, capable of integrating unstructured information and generating human-readable analyses. While LLMs have been explored for various financial tasks, their application in credit risk assessment has been limited, often focusing on structured financial indicators or using non-financial information merely as input variables for classification, rather than generating explanatory reasoning.
To address this crucial gap, a recent study introduces two innovative LLM-based systems designed to generate structured reasoning from non-financial evidence. The goal is not just to detect risk signals but to synthesize non-financial evidence into coherent and defensible credit reasoning. This research explores the potential of Agentic AI, which extends conventional LLMs with autonomous planning and multi-step reasoning capabilities.
Two Approaches to Automated Credit Assessment
The study developed and evaluated two distinct systems:
- Non-Adversarial System (NAS): This is a single-agent system that performs a bidirectional analysis (identifying both favorable and adverse signals) through a single-pass reasoning pipeline. It operates in three stages: data consolidation, reasoning based on prior knowledge, and analytical synthesis.
- Karl Popper Debate-based Multi-Agent System (KPD-MADS): This is a more sophisticated multi-agent system that formalizes adversarial verification through a ten-step structured interaction protocol. It involves six LLM agents organized into affirmative and negative teams, engaging in a structured debate to strengthen, challenge, and refine competing claims. The debate process is inspired by the Karl Popper debate, emphasizing critical rationalism and evidence-based argumentation.
Both systems operate on a shared knowledge framework, which includes a guideline prompt specifying how evidence should be evaluated across ten non-financial factor categories (e.g., industry growth outlook, market competition, government support, internal control risk). This framework ensures consistency and reflects established assessment principles. Company-related information is retrieved from a structured knowledge pool, supplemented by real-time web searches when necessary to ensure recency and transparency.
Evaluating Performance: Productivity and Quality
The systems were applied to three real corporate cases in South Korea and evaluated by experienced credit risk professionals. The evaluation focused on productivity, perceived report quality, and usability.
In terms of productivity, both systems demonstrated significant gains compared to human experts. While a human baseline for report preparation averaged 1,920 seconds per case, the NAS completed reports in an average of 11.55 seconds, and the KPD-MADS in 91.97 seconds. This highlights the immense efficiency potential of AI in this domain.
However, a clear trade-off emerged between generation latency and reasoning quality. The KPD-MADS, despite taking longer than the NAS, demonstrated superior reasoning quality. It received higher median ratings from professionals in explanatory adequacy (4.0 vs. 3.0), practical applicability (4.0 vs. 3.0), and usability (62.5 vs. 52.5). These ratings indicate that KPD-MADS outputs were considered professionally usable, while NAS outputs remained at a neutral level.
An objective measure, the Reasoning Elaboration Index (REI), further supported these findings. The KPD-MADS produced reports with a higher average REI (14.33 vs. 8.00), primarily due to its ability to develop deeper, multi-level arguments. While the NAS produced wider trees (more factors), it maintained a uniform single-level depth. The KPD-MADS, in contrast, selectively expanded reasoning depth to up to three levels when dealing with indicators that presented contradictory evidence, ambiguous signals, or multiple plausible explanations, demonstrating an adaptive mechanism for resolving analytical uncertainty.
Also Read:
- Enhancing LLM Multi-Agent Reasoning Through Strategic Self-Play
- AI Agents Learn and Adapt Through Dialogue to Tackle Complex Problems
The Power of Structured Debate
The study concludes that reasoning quality is significantly influenced by how reasoning is procedurally organized, not just by model size or fluency. The superior performance of the KPD-MADS is attributed to its structured debate protocol. By formalizing reasoning as an argumentative process, requiring claims to be supported by evidence, critically evaluated, and revised, the system enhances reasoning precision and filters unsupported assumptions.
This research marks a significant step forward in automated corporate credit risk assessment, demonstrating that LLM-based agentic systems can not only boost analytical efficiency but also produce qualitative reasoning that supports financial professionals’ decision-making. For more in-depth details, you can read the full research paper here.
Future research aims to explore variations in debate protocol design, enhance architectural specialization through agent roles, and extend the KPD-MADS framework to other high-stakes financial environments like insurance underwriting and project finance evaluation.
