TLDR: MLLM-CTBench is a new benchmark designed to rigorously evaluate how Multimodal Large Language Models (MLLMs) continually learn and adapt. It introduces a multidimensional evaluation that assesses both final answer accuracy and Chain-of-Thought reasoning quality, benchmarks various continual learning algorithms and training paradigms (including reinforcement learning vs. supervised fine-tuning), and uses 16 challenging datasets across six domains. Key findings include that stronger models forget less, reasoning degrades slower than answers, algorithm effectiveness depends on model and task order, and KL-divergence regularization is crucial for reinforcement learning to mitigate forgetting.
Multimodal Large Language Models (MLLMs) are advanced AI systems that can understand and generate content across different types of data, like text and images. For these models to remain effective in real-world situations, they need to continuously learn and adapt to new information and tasks without forgetting what they’ve already learned. This process is known as continual instruction tuning.
However, a significant challenge in this field has been the absence of a thorough and systematic way to evaluate how well MLLMs perform under continual learning. Existing benchmarks often fall short by focusing only on final answers, neglecting the crucial reasoning processes, or by using tasks that aren’t challenging enough for modern MLLMs.
To address these limitations, researchers have introduced MLLM-CTBench, a new comprehensive evaluation benchmark. This benchmark brings three key advancements to the table. First, it offers a multidimensional evaluation approach. Beyond just checking the accuracy of final answers, it also assesses the quality of the model’s Chain-of-Thought (CoT) reasoning. This is made possible by a specially trained CoT evaluator, which provides a more in-depth understanding of why models might forget information.
Second, MLLM-CTBench provides a comprehensive evaluation of various algorithms and training methods. It benchmarks eight different continual learning algorithms, categorized into four major types: regularization-based, replay-based, architecture-expansion-based, and model-fusion-based. Furthermore, it systematically compares reinforcement learning (RL) with traditional supervised fine-tuning (SFT) approaches, offering insights into which paradigms work best for continuous adaptation.
Third, the benchmark features carefully curated tasks. It selects and organizes 16 datasets from existing research, covering six challenging domains where MLLMs typically struggle. These include mathematical reasoning, optical character recognition (OCR) comprehension, and domain-specific knowledge in areas like science, medicine, arts, and economics. This diverse set of tasks ensures a rigorous test of an MLLM’s ability to learn continually.
Through extensive experiments using MLLM-CTBench, several important findings have emerged. It was observed that models with stronger general capabilities tend to be more resistant to forgetting during continual learning. This means that a more capable base model is likely to retain information better over time. Another key insight is that the intermediate reasoning steps, or ‘reasoning chains,’ degrade more slowly than the final answers. This supports the idea of ‘hierarchical forgetting,’ suggesting that factual knowledge might decay faster than the underlying procedural reasoning skills.
The effectiveness of different continual learning algorithms was also found to depend heavily on both the model’s inherent capability and the order in which tasks are presented. For instance, replay methods, which involve revisiting past data, are particularly beneficial for weaker models but show diminishing returns for stronger ones. In contrast, regularization-based methods, which constrain updates to important parameters, perform better with high-capacity models.
A significant finding related to reinforcement learning is that its advantage in mitigating forgetting, particularly with methods like GRPO (Generalized Reinforcement with Prompt Optimization), relies heavily on incorporating KL-divergence constraints. These constraints help maintain the stability of the model’s learning policy, acting as a form of implicit memory that preserves previously acquired reasoning skills. Without this constraint, reinforcement learning can even lead to more severe forgetting than supervised fine-tuning.
In practical terms, this research offers valuable guidance for designing and evaluating future continual learning algorithms for MLLMs. It highlights the importance of considering both final answer accuracy and the quality of reasoning processes, and it provides a robust framework for testing new methods. For more technical details, you can refer to the full research paper here.
Also Read:
- A New Toolkit for Multimodal AI: Introducing MCITlib for Continuous Learning
- LiLoRA: A New Approach to Efficient Continual Learning in Multimodal AI
The MLLM-CTBench sets a new standard for evaluating how Multimodal Large Language Models adapt and retain knowledge over time, paving the way for more robust and adaptable AI systems in real-world applications.
