TLDR: VLA-0 is a novel Vision-Language-Action (VLA) model that achieves state-of-the-art robot manipulation by representing actions directly as text. Unlike traditional approaches that modify Vision-Language Models (VLMs) with action tokens or heads, VLA-0 uses the VLM’s native text generation capabilities without any architectural changes. Through a careful training and testing recipe, including action decoding, ensemble prediction, and masked action augmentation, VLA-0 outperforms more complex VLA models on the LIBERO benchmark and in real-world tasks, even surpassing some models trained with large-scale robotic data.
Vision-Language-Action (VLA) models are at the forefront of enabling generalist robot manipulation, allowing robots to understand visual information, interpret language instructions, and perform physical actions. Traditionally, building these sophisticated models has involved complex modifications to existing Vision-Language Models (VLMs), such as adding special action tokens to their vocabulary or introducing dedicated action generation heads. These approaches often come with trade-offs, like limiting action resolution or compromising the VLM’s original language understanding capabilities.
A new research paper from NVIDIA, titled VLA-0: Building State-of-the-Art VLAs with Zero Modification, explores a surprisingly simple yet highly effective alternative: representing robot actions directly as text. Authored by Ankit Goyal, Hugo Hadfield, Xuning Yang, Valts Blukis, and Fabio Ramos, this work introduces VLA-0, a model that achieves state-of-the-art performance without altering the underlying VLM’s architecture, vocabulary, or adding new neural network layers.
The Simplicity of VLA-0
The core idea behind VLA-0 is to leverage the VLM’s native text generation capabilities to predict actions. Instead of discretizing continuous actions into special tokens or using complex generative heads, VLA-0 asks the VLM to output actions as numerical strings – for example, a sequence of space-separated integers representing end-effector coordinates or joint angles. This approach maintains the integrity of the VLM, allowing it to retain its powerful pre-trained language understanding and grounding capabilities.
This method offers several advantages. It avoids the limitations of discrete token VLAs, which can restrict action resolution and corrupt the VLM’s vocabulary. It also bypasses the need for additional neural networks and fine-tuning complexities associated with generative action head VLAs, which can sometimes degrade the VLM’s language understanding.
Unlocking High Performance
While the concept is simple, achieving state-of-the-art performance with VLA-0 relies on a carefully designed training and testing recipe. The researchers highlight three key components:
- Action Decoding: Continuous action values are normalized to a fixed integer range (e.g., 0-1000), and the VLM is trained to generate these integers for each action dimension. This allows for arbitrary action resolution without modifying the model’s vocabulary.
- Ensemble Prediction: During inference, VLA-0 predicts a sequence of future actions. For the current timestep, it averages multiple predictions made at different points in time, leading to more stable and reliable actions.
- Masked Action Augmentation: During training, random characters in the target action string are masked out. This forces the VLM to rely more on visual observations and task instructions to infer actions, rather than simply auto-completing a numerical sequence.
Impressive Results in Simulation and Real-World
VLA-0 was rigorously evaluated on the LIBERO benchmark, a popular platform for VLA models, and in real-world robotic tasks. The results were striking:
- In simulation, VLA-0 outperformed all existing VLA models trained with the same amount of robotic action data, including prominent methods like π0.5-KI, OpenVLA-OFT, and SmolVLA.
- Even more surprisingly, VLA-0, without any large-scale robotics-specific pretraining, surpassed the performance of many well-known models that *were* pretrained on extensive robotic datasets, such as π0.5-KI, π0, GR00T-N1, and MolmoAct.
- In real-world experiments using the SO-100 robot, VLA-0 demonstrated superior performance over SmolVLA, a model specifically pretrained on a large-scale real-world dataset, across various manipulation tasks like reorienting a block and picking and placing objects.
These findings suggest that the prevailing trend of adding complexity to VLAs might not always be necessary. VLA-0 demonstrates that a focused approach on leveraging the VLM’s inherent capabilities through simple text-based action representation, combined with smart training and inference techniques, can yield highly competitive and even superior results.
Also Read:
- X-VLA: Unifying Robot Control Across Diverse Platforms with Soft Prompts
- Unlocking Data Scaling for Self-Driving VLAs Through World Modeling
Future Directions
The researchers acknowledge limitations and point to future work, including exploring VLA-0’s performance with large-scale action data and improving inference speed through optimization techniques like quantization and distillation. Nevertheless, VLA-0 presents a compelling case for simplicity in the design of next-generation Vision-Language-Action models.
