TLDR: AndesVL is a new suite of efficient multimodal large language models (MLLMs) developed by OPPO AI Center, designed specifically for mobile devices. Ranging from 0.6 billion to 4 billion parameters, AndesVL achieves top-tier performance across various benchmarks, including text understanding, reasoning, multi-image comprehension, and GUI-related tasks. It incorporates innovative techniques like a 1+N LoRA architecture, Quantization-Aware LoRA Fine-Tuning (QALFT), and optimized decoding strategies, enabling significant speedups and memory reductions for on-device deployment on chips like MediaTek Dimensity 9500.
In an era where large language models (LLMs) and multimodal large language models (MLLMs) are pushing the boundaries of artificial intelligence, a significant challenge has emerged: bringing these powerful models to everyday mobile devices. Cloud-based MLLMs, with their enormous sizes reaching hundreds of billions of parameters, are simply too demanding for the limited memory, power, and computing capacity of smartphones and other edge devices.
Addressing this crucial gap, the OPPO AI Center has introduced AndesVL, a groundbreaking suite of mobile-side MLLMs. These models are specifically engineered for efficient deployment on edge devices, with parameter sizes ranging from a compact 0.6 billion to a robust 4 billion. AndesVL is built upon the foundation of Qwen3’s LLM and integrates various advanced visual encoders, making it a versatile and powerful tool for mobile AI.
Unpacking AndesVL’s Capabilities
AndesVL has demonstrated exceptional performance across a wide array of open-source benchmarks. It excels in diverse fields such as understanding text-rich images, complex reasoning and mathematical problems, comprehending multiple images simultaneously, general visual question answering (VQA), mitigating AI hallucinations, multilingual understanding, and even tasks related to graphical user interfaces (GUI). When compared to other state-of-the-art models of similar scale, AndesVL consistently achieves first-tier results.
The development of AndesVL involved a meticulous approach to model architecture, training pipelines, and data preparation. The models are designed to handle image inputs with arbitrary aspect ratios and resolutions, a critical feature for real-world mobile use. This is achieved through a Native Resolution ViT (NaViT)-based strategy, ensuring consistency between model inputs and original data without resorting to image cropping.
Innovative Training and Optimization
AndesVL’s training process is divided into distinct stages, including vision-language alignment, joint vision-language pre-training, and multi-task pre-training. This comprehensive approach allows the models to acquire extensive visual knowledge and develop robust multimodal capabilities. Furthermore, AndesVL offers two specialized versions: an instruction model for efficient task understanding and generation, and a thinking model designed for complex mathematical reasoning and planning.
For practical application on mobile devices, AndesVL introduces a novel 1+N Low-Rank Adaptation (LoRA) architecture. This allows for efficient task adaptation and model compression, enabling the base model to be adapted to various scenarios with minimal fine-tuning. To further enhance efficiency, a Quantization-Aware LoRA Fine-Tuning (QALFT) framework has been developed. This framework ensures that the model’s performance is preserved on edge devices while significantly boosting inference efficiency through fine-grained mixed-precision quantization.
Beyond its architecture and training, AndesVL incorporates advanced acceleration and compression strategies. These include a customized cache eviction algorithm called OKV, along with tailored speculative decoding and sparsification techniques. These innovations lead to remarkable on-device performance: a 6.7x peak decoding speedup ratio, up to 30.9% memory reduction, and an impressive 1.8 bits-per-weight when deploying AndesVL-4B on MediaTek Dimensity 9500 chips.
Also Read:
- Tiny-R1V: A Compact AI Model for Unified Multimodal Reasoning
- Unlocking Real-Time AI Perception for Endless Video Streams
Real-World Impact and Future Vision
The extensive experimental results confirm AndesVL’s comprehensive capabilities. It not only outperforms existing models of similar sizes across various benchmarks but also demonstrates particular strength in areas vital for mobile interaction, such as GUI understanding. The in-house AndesUI dataset, specifically developed for evaluating smartphone UI understanding, showcases AndesVL’s expertise in this domain.
The release of AndesVL models on Hugging Face signifies a major step towards making advanced multimodal AI accessible on edge devices. This work effectively bridges the gap between powerful cloud-based MLLMs and the practical limitations of mobile hardware, paving the way for a new generation of intelligent mobile applications. For more technical details, you can refer to the AndesVL Technical Report.
Looking ahead, the AndesVL team plans to explore even more optimal visual encoder solutions, develop superior post-training schemes, implement effective distillation techniques between large and small models, and ultimately work towards a unified mobile-side model that seamlessly integrates text, image, and speech modalities. These future directions promise to further enhance the capabilities and expand the application scope of mobile-side MLLMs.
