TLDR: NVIDIA has introduced a suite of new generative AI models and software libraries, including the Cosmos Reason vision language model, aimed at enabling robots and AI agents to process information and reason with human-like understanding. This development, unveiled at SIGGRAPH, signifies a major step in physical AI, allowing robots to leverage prior knowledge, physics, and common sense for more complex tasks.
NVIDIA has announced significant breakthroughs in artificial intelligence, unveiling new generative AI models and software libraries designed to imbue robots and vision AI agents with human-like thought processes. These innovations, showcased at SIGGRAPH, the premier computer graphics conference, are set to redefine the capabilities of physical AI.
At the forefront of these advancements is Cosmos Reason, a novel reasoning vision language model. This model empowers robots and AI agents to ‘reason like humans using prior knowledge, physics understanding and common sense,’ as detailed in NVIDIA’s announcement. This capability is crucial for expanding the range of physical feats robots can accomplish, from navigating challenging terrains for emergency response to performing delicate tasks like picking fruit without damage or assembling microscopic electronic components with precision.
The development of these advanced physical AI systems hinges on the creation of high-fidelity, physically accurate 3D environments. These lifelike virtual settings are essential for training sophisticated physical AI systems, such as humanoid robots, ensuring that the skills acquired in simulation translate effectively to the real world. NVIDIA’s research integrates neural graphics, synthetic data generation, physics-based simulation, reinforcement learning, and AI reasoning to achieve this.
Sanja Fidler, vice president of AI research at NVIDIA, emphasized the symbiotic relationship between AI and simulation, stating, “AI is advancing our simulation capabilities, and our simulation capabilities are advancing AI systems. There’s an authentic and powerful coupling between the two fields, and it’s a combination that few have.”
Also Read:
- Second NSF Workshop Highlights Generative AI’s Transformative Role in Scientific Discovery
- Avaamo Unveils Advanced Task-Specific AI Agents to Revolutionize Enterprise Operations
Beyond Cosmos Reason, NVIDIA is also rolling out other key software libraries for physical AI, including NVIDIA Omniverse NuRec 3D Gaussian splatting libraries for large-scale world reconstruction and updates to the NVIDIA Metropolis platform for vision AI. The company is also advancing its NVIDIA Nemotron reasoning models, further solidifying its commitment to the convergence of AI and graphics research to power robotics, autonomous vehicles, and content creation.
