TLDR: A new research paper argues that the term ‘agent’ in AI has become too diluted, causing confusion in research, evaluation, and policy. It proposes a framework that defines minimum requirements for an AI system to be considered an agent (environmental impact, goal-directed behavior, state awareness) and characterizes systems along five core dimensions: environmental interaction, goal complexity, temporal coherence, learning and adaptation, and autonomy. This framework aims to provide a more precise vocabulary for describing and evaluating AI systems, fostering clearer communication and more effective policy development.
The term ‘agent’ in artificial intelligence has become increasingly ambiguous, leading to significant challenges in how AI research is communicated, how systems are evaluated, and how policies are developed. This paper highlights that the broad characterizations of ‘agent’ have been modified over time to fit various research disciplines, resulting in a confusing array of definitions. With the widespread adoption of large language model (LLM) systems, this ambiguity has only intensified, with terms like ‘agent’ and ‘agentic’ often used interchangeably.
The lack of a clear definition makes it difficult to evaluate agent-based research, as evaluation criteria are not aligned across different interpretations. This also hinders the reproducibility of AI agent research. Furthermore, the confusion extends beyond the research community, impacting public perception and policy development. Different organizations and public figures, from Anthropic to Andrew Ng, offer varied definitions, contributing to a lack of precision that can influence how AI systems are understood and regulated.
A New Framework for Defining AI Agents
To address this critical need for clarity, the paper proposes a new framework for redefining what constitutes an ‘agent’ in AI. It introduces the term ‘agenticness’ to describe the degree to which a system exhibits agent-like characteristics, intentionally avoiding the philosophical weight of ‘agency’. The framework is built on historical analysis and contemporary usage patterns, aiming to provide precise vocabulary while preserving the term’s multifaceted nature.
For a system to be considered an agent, it must meet three minimum requirements:
-
Active and Measurable Environmental Impact: The system must be capable of taking actions that meaningfully and persistently alter its environment, with these changes being observable or measurable. Simple input-output systems like basic chatbots do not qualify.
-
Goal-Directed Behavior: The system must operate in service of defined objectives, adapting based on the environment and pursuing goals through multi-step planning and execution, not just optimization. Purely reactive systems are excluded.
-
State Awareness: The system must maintain and update a representation of its environmental state that influences its decisions, and this awareness must persist between interactions. Stateless systems, such as models processing each input independently, do not qualify.
Once these minimum requirements are met, AI systems can be characterized along a spectrum across five core dimensions of ‘agenticness’:
-
Environmental Interaction Sophistication: This measures a system’s ability to perceive, understand, and manipulate its environment, ranging from predefined actions in structured settings to complex action composition in unstructured environments with sophisticated tool use.
-
Goal-Directed Behavior Complexity: This dimension assesses a system’s capacity to form, understand, and pursue objectives, from basic adaptive goals within a narrow domain to abstract goal formation and complex adaptive planning across multiple interdependent objectives.
-
Temporal Coherence: This refers to a system’s ability to maintain consistent operation over time through state awareness and memory, ranging from basic short-term memory to advanced state management with hierarchical memory structures and complex temporal reasoning.
-
Learning and Adaptation: This encompasses a system’s capacity to improve performance and adjust to new situations, from basic parameter updating in defined scenarios to continuous learning, knowledge synthesis, and meta-learning capabilities.
-
Autonomy: This characterizes a system’s ability to operate without constant external guidance, including handling errors and unexpected situations, ranging from bounded autonomous operation with basic error handling to self-directed operation with clever error handling and recovery.
These dimensions are interconnected, meaning a system’s capabilities in one area often influence others. For instance, higher temporal coherence enables more complex environmental modeling, and advanced learning capabilities can lead to more sophisticated goal structures.
Applying the Framework to AI Systems
The framework clarifies why certain common AI systems, like simple chatbots, basic classification systems, and static expert systems, do not qualify as agents because they lack environmental impact, goal-directed behavior, or persistent state awareness.
Conversely, the paper illustrates how the framework applies to more advanced systems:
-
Smallville Generative Agents: These exhibit intermediate to advanced levels across most dimensions, showcasing sophisticated environmental interaction, goal-directed behavior, and temporal coherence within their sandbox environment.
-
LLM-based Personal Assistants: These typically demonstrate intermediate levels in environmental interaction, goal-directed behavior, and temporal coherence, but may be at a threshold level for learning and adaptation, and intermediate for autonomy.
-
Standard Vacuum Cleaning Robots: These are often at the threshold for environmental interaction and goal-directed behavior, intermediate for temporal coherence and autonomy, but weakest in learning and adaptation.
-
Theoretical Autonomous Scientific Research Systems: These represent the highest levels of agenticness across all dimensions, capable of complex experimental interaction, abstract goal formation, multi-layer state representation, continuous learning, and self-directed research.
Edge cases like Large Language Models with tool access and recommendation systems are also discussed, emphasizing that their classification as agents depends heavily on specific implementation details and whether they meet the minimum requirements and exhibit the core dimensions.
Also Read:
- Unpacking AI Learning: A Year-Long Simulation Reveals How LLMs Acquire Knowledge and Develop Cognition
- Navigating the Future of AI: A Look at Agent Registry Solutions
Moving Forward with Clarity
The paper acknowledges potential counterarguments, such as the risk of oversimplification or the practical challenges of implementing new standards. However, it argues that the benefits of clearer terminology—improved research communication, evaluation, and policy development—outweigh these challenges.
The authors recommend that researchers explicitly specify which dimensions of agenticness their work addresses and to what degree. They also urge benchmark developers to align their evaluations with these dimensions and call for standards organizations and regulatory bodies to adopt more precise terminology for AI governance. This structured approach aims to foster more rigorous research and support effective policies that can adapt to the continuous evolution of AI capabilities. For more details, you can read the full paper here.
