TLDR: A new research paper evaluates 17 leading Multimodal Large Language Models (MLLMs) on a set of daily composite tasks in a simulated home environment, inspired by early childhood development. The tasks cover object understanding, spatial intelligence, and social activity. The study found that current MLLMs perform poorly across all domains, indicating a significant gap between their capabilities and the requirements for Artificial General Intelligence (AGI), particularly in complex embodied reasoning and social interaction.
The quest for Artificial General Intelligence (AGI) continues to be a central focus in AI research. Unlike traditional AI, AGI is expected to handle complex, composite tasks that demand a broad spectrum of capabilities. While Multimodal Large Language Models (MLLMs) have shown impressive advancements in perception and interaction, their ability to tackle these multifaceted tasks, especially within dynamic environments, has remained largely unexplored.
A recent research paper, titled “Evaluating Multimodal Large Language Models with Daily Composite Tasks in Home Environments,” delves into this critical area. Authored by Zhenliang Zhang, Yuxi Wang, Hongzhao Xie, Shiyun Zhao, Mingyuan Liu, Yujie Lu, Xinyi He, Zhenku Cheng, and Yujia Peng, this work introduces a novel framework to assess the general abilities of MLLMs. You can find the full paper here.
The Challenge with Current AI Evaluations
Traditional AI evaluation methods, such as the Turing Test or task-specific benchmarks like ImageNet or COCO, have limitations when it comes to AGI. The Turing Test, while foundational, relies heavily on human judgment and can be ‘beaten’ by carefully designed response algorithms rather than genuine intelligence. Task-oriented benchmarks, on the other hand, often lead to over-specialization and overfitting, where models perform well in controlled settings but fail in complex, real-world scenarios.
Even more recent multi-task evaluation platforms, like MMBench and FlagEval, despite offering broad testing, lack the crucial element of embodied evaluation. This means they don’t fully assess how an AI agent interacts physically and socially within a dynamic environment, which is considered essential for true general intelligence.
A New Approach: Daily Composite Tasks in a Simulated Home
The researchers argue that evaluating AGI requires an embodied environment where an agent must continuously adapt to a dynamic world involving both physical and social interactions. To address this, they designed a set of composite tasks inspired by common daily activities observed in early childhood development. These tasks are set within a dynamic, simulated home environment and cover three core domains:
- Object Understanding: Tasks like counting objects with specific attributes or selecting appropriate gifts for different scenarios.
- Spatial Intelligence: Involving activities such as building blocks to match a target shape or solving jigsaw puzzles.
- Social Activity: Tasks like setting a dining table for guests, tidying up a room, or packing a suitcase for a trip.
These tasks were designed with three key principles: alignment with child development milestones, requiring composite abilities within an embodied environment, and being ability-oriented to systematically assess a range of cognitive and interactive skills.
How MLLMs Were Evaluated
The study evaluated 17 leading proprietary and open-source MLLMs. To integrate these models into the embodied task system, standard perception and action modules were developed, creating a perception-reasoning-action loop within a high-fidelity 3D simulator. The MLLM agents received visual input and scene descriptions, then generated natural-language reasoning traces and executable API calls to interact with the environment. This approach, using only prompt engineering without additional memory systems or external tools, aimed to directly assess the foundational capabilities of the MLLMs.
Key Findings: A Substantial Gap Remains
The results consistently showed poor performance across all three domains for the MLLM agents. The highest average score achieved by any model was only 25.36 out of 100 (Gemini-Pro-2.5). While proprietary models generally outperformed open-source ones, the performance gap was not substantial, indicating a systemic challenge rather than one confined to specific model families.
Most models performed relatively better in object understanding tasks compared to spatial intelligence and social activity. For instance, OpenAI o3 excelled in counting objects, and GPT-5 led in selecting gifts. However, spatial reasoning and social interaction tasks proved particularly challenging, with even the top-performing models in these categories (e.g., GPT-5 in setting tables, Gemini-Flash-2.5 in tidying rooms) achieving limited scores. This suggests that while MLLMs show promise in perceptual recognition, they still lack the robust multimodal integration and embodied reasoning necessary for general-purpose intelligence.
Also Read:
- EngiBench: A New Standard for Assessing AI in Engineering Challenges
- Foundation Models Navigate Virtual Worlds: New Strategies for Reinforcement Learning
Implications for Future AGI Development
The study highlights that current MLLMs might rely on surface-level correlations across language, perception, and motion, rather than developing a deep, grounded conceptual understanding of objects, space, and social dynamics. The inherent variability and randomness in task execution in real-world scenarios further amplify their lack of robustness.
This research provides a preliminary but significant framework for evaluating the general capabilities of embodied agents. It underscores critical limitations that future work must address, focusing on advancing multimodal perception, embodied reasoning, and social understanding to bring MLLMs closer to real-world AGI applications.
