TLDR: InternVLA-M1 is a new robot policy framework that uses “spatially guided vision-language-action training” to enable robots to better understand instructions and perform complex tasks. It trains robots in two stages: first, learning “where to act” through spatial reasoning, and then “how to act” by generating specific movements. This approach, supported by massive datasets and a dual-system architecture, significantly improves robot performance in both simulated and real-world scenarios, especially in tasks requiring long-term planning and adaptability to new objects and situations.
A new framework called InternVLA-M1 is making strides in robot policy, aiming to create more scalable and general-purpose intelligent robots that can effectively follow instructions. Developed by Intern Robotics and Shanghai AI Laboratory, this unified system focuses on a novel approach: spatially guided vision-language-action training. The core idea is to use spatial understanding as a crucial link between human instructions and robot actions.
The InternVLA-M1 framework operates through a two-stage training pipeline. The first stage, known as spatial grounding pre-training, teaches the robot “where to act.” This involves aligning instructions with visual, location-agnostic positions, leveraging over 2.3 million pieces of spatial reasoning data. Essentially, the robot learns to identify relevant areas or objects in its environment based on a given command. Following this, the second stage, spatially guided action post-training, focuses on “how to act.” Here, the robot generates specific, embodiment-aware actions by using a technique called plug-and-play spatial prompting, translating its spatial understanding into physical movements.
This innovative training method has yielded impressive results. InternVLA-M1 has shown significant performance gains compared to its counterparts without spatial guidance. For instance, it achieved a 14.6% improvement on SimplerEnv Google Robot, a 17% increase on WidowX, and a 4.3% boost on LIBERO Franka. Beyond these benchmarks, the framework demonstrated enhanced spatial reasoning capabilities in tasks involving box, point, and trace prediction.
To further scale its instruction-following abilities, the researchers built a simulation engine that generated 244,000 generalizable pick-and-place episodes. This extensive synthetic data led to an average improvement of 6.2% across 200 tasks and over 3,000 objects. In real-world scenarios, such as clustered pick-and-place tasks, InternVLA-M1 improved by 7.3%, and with synthetic co-training, it achieved a remarkable 20.6% increase in success rates on unseen objects and novel configurations. For complex, long-horizon tasks that require intricate reasoning, the system surpassed existing works by over 10%, underscoring the effectiveness of spatially guided training for creating resilient and scalable generalist robots.
The model itself is a dual-system architecture, integrating a language head and an action head. The language head, powered by Qwen2.5-VL-3B-instruct, establishes instruction-to-visual grounding through spatial pre-training. The action head, built on a diffusion policy with a DINOv2 visual encoder, then uses these learned spatial priors to generate motor commands. With approximately 4.1 billion parameters, the system is designed for robust instruction following in diverse and complex environments, both simulated and real.
The training process involves dual-supervision, where batches from both multimodal and action data types are processed jointly. This ensures that perception and control are learned in a co-adapted manner. The spatial grounding pre-training stage focuses on enhancing spatial reasoning and planning for robotics by combining internet-scale multimodal corpora with robot-specific datasets. The spatially guided action post-training stage then jointly optimizes both the VLM and Action Expert on demonstration data, using spatial prompting and co-training with spatial grounding data to reinforce spatial reasoning and support efficient end-to-end optimization.
The extensive data used for InternVLA-M1 includes over 3 million multimodal training samples, with more than 2.3 million dedicated to spatial reasoning across categories like General QA, Box QA, Trajectory QA, and Point QA. A scalable synthetic data engine, built on GenManip and Isaac Sim, generated the InternData-M1 dataset, comprising 244,000 closed-loop pick-and-place samples, further enhancing the model’s ability to generalize.
Evaluations across various benchmarks, including SimplerEnv, LIBERO, large-scale simulated pick-and-place, and real-world cluttered-scene pick-and-place, consistently demonstrated InternVLA-M1’s superior performance. Its ability to handle unseen objects, new backgrounds, and paraphrased instructions highlights its strong generalization capabilities. Furthermore, in long-horizon and reasoning-intensive manipulation tasks, the framework proved highly adaptable to physical interference and dynamic task replanning, showcasing its resilience in complex, real-world scenarios.
Also Read:
- X-VLA: Unifying Robot Control Across Diverse Platforms with Soft Prompts
- ManiAgent: Orchestrating Robot Actions with AI Agents
This research highlights spatially guided training as a unifying principle for scalable and reliable generalist robots, bridging high-level multimodal reasoning with low-level robotic execution. For more in-depth information, you can read the full research paper here.
