TLDR: Ovis2.5 is a new multimodal large language model from Alibaba Group that significantly improves visual perception and reasoning. It processes images at their native resolution, avoiding detail loss, and introduces an optional ‘thinking mode’ for self-correction and deeper reasoning. Trained with a five-phase curriculum and efficient infrastructure, Ovis2.5 achieves state-of-the-art performance on various benchmarks, including complex visual tasks, mathematical problems, and video understanding, with both 9B and 2B parameter versions released.
Alibaba Group’s Ovis Team has unveiled Ovis2.5, a significant advancement in the field of multimodal large language models (MLLMs). This new model builds upon its predecessor, Ovis2, by introducing key innovations aimed at enhancing both visual perception and complex reasoning capabilities. [RESEARCH_PAPER_URL]
Smarter Vision with Native Resolution
One of the core improvements in Ovis2.5 is its approach to visual perception. Previous MLLMs often struggled with high-resolution images or visually dense content like charts because they processed images by splitting them into fixed-size sub-images. This method could lead to a loss of fine details and disrupt the overall global structure of the image. Ovis2.5 addresses this by integrating a ‘native-resolution vision transformer’ (NaViT). This allows the model to process images at their original, variable resolutions, preserving crucial details and the global layout, which is particularly important for tasks involving complex charts and diagrams.
Deeper Reasoning with a ‘Thinking Mode’
Beyond just seeing better, Ovis2.5 is designed to think more deeply. The model moves beyond simple, linear ‘chain-of-thought’ reasoning by incorporating a new ‘thinking mode’. This advanced capability, trained using data that encourages reflection, allows the model to perform self-checking and revision. Essentially, it learns to evaluate its own reasoning steps and refine its conclusions when necessary. Users can activate this optional mode during inference, trading a bit more processing time for significantly enhanced accuracy on challenging problems.
A Comprehensive Training Approach
To achieve these upgrades, Ovis2.5 underwent a comprehensive five-phase training curriculum. This progressive process starts with foundational visual and multimodal pretraining, moves through large-scale instruction tuning, and culminates in advanced alignment and reasoning enhancement using sophisticated techniques like DPO and GRPO. The training data itself is a rich mix, including specialized datasets for Optical Character Recognition (OCR), visual grounding (locating objects from descriptions), and complex reasoning tasks, including those designed to teach the ‘thinking-style’ approach.
Efficiency and Performance
Training such a powerful model efficiently required significant infrastructure advancements. The Ovis Team employed techniques like multimodal data packing, which minimizes wasted computation by combining shorter data samples, and a hybrid parallelism framework. These optimizations resulted in a remarkable 3 to 4 times end-to-end speedup in training.
The results speak for themselves. On the OpenCompass multimodal leaderboard, Ovis2.5-9B, the larger of the two released models, achieved an average score of 78.3. This marks a substantial improvement over its predecessor, Ovis2-8B, and positions it as a state-of-the-art open-source MLLM in the sub-40B parameter range. A smaller, more resource-efficient version, Ovis2.5-2B, also scored impressively at 73.9, setting a new benchmark for models of its size, ideal for on-device applications. Beyond aggregate scores, Ovis2.5 demonstrates leading performance across various specialized benchmarks, including STEM (Science, Technology, Engineering, and Mathematics) problems, complex chart analysis, visual grounding, and video understanding tasks.
Also Read:
- A New Path for Vision-Language AI: Text-to-Vision Mapping Without Alignment
- Advancing AI’s Visual Understanding with Region-Level Context
Looking Ahead
Ovis2.5 represents a significant step forward in multimodal AI, offering enhanced perception and reasoning capabilities. While the current version delivers impressive results, the Ovis Team is already looking towards future improvements, including scaling perception to 4K-level high-resolution images, handling longer video inputs with richer temporal reasoning, and integrating tool use for more action-augmented reasoning.
