TLDR: A new research paper argues that the true challenge for AI is not just interpretability or explainability, but ‘systematicity’ – the ability to form a consistent, coherent, comprehensive, principled, and parsimonious body of thought. The paper distinguishes this ‘hard systematicity challenge’ from older notions, outlining five crucial functions (constitutive, hermeneutic, epistemological, critical, didactic) that systematicity serves for both human and artificial cognition. It concludes that a dynamic, context-sensitive approach to systematicity is needed to guide future AI development, ensuring models are intelligent in ways that meet diverse human needs.
Concerns about artificial intelligence (AI) often revolve around its perceived lack of interpretability or explainability. We frequently ask, ‘How did the AI reach this conclusion?’ or ‘Why should I trust this output?’ However, a new research paper titled Explainability Through Systematicity: The Hard Systematicity Challenge for Artificial Intelligence by Matthieu Queloz argues that these questions are just one part of a much larger ideal we hold for AI: systematicity.
What is Systematicity?
The paper suggests that the core issue is whether AI exhibits ‘systematicity of thought.’ This isn’t just about AI being able to combine ideas like building blocks (a concept known as ‘compositionality’). Instead, it’s about AI striving towards an integrated body of thought that is consistent, coherent, comprehensive, principled, and parsimonious. Think of it as building a well-organized, logical library of knowledge, rather than just having a collection of individual books.
This broader idea of systematicity, which the author calls ‘macrosystematicity,’ has been overshadowed by a narrower, older challenge to AI known as the ‘systematicity challenge.’ This challenge, posed by Jerry Fodor and his colleagues, questioned whether neural networks could handle the basic compositionality of thought, like understanding that if ‘John loves Mary’ is possible, then ‘Mary loves John’ should also be possible. While AI has made significant strides in overcoming this original challenge, the paper argues that a ‘hard systematicity challenge’ remains: getting AI to achieve this richer, more integrated form of thought.
Why Does Systematicity Matter for AI?
The paper identifies five key reasons, or ‘rationales,’ why systematicity is crucial, and why these reasons apply to AI just as they do to human cognition:
1. Constitutive Function: For us to even perceive an AI as ‘thinking’ or ‘cognizing’ the world, its outputs need to show a basic aversion to contradictions and incoherence. If an AI constantly contradicts itself without any attempt to resolve it, it stops looking like a coherent intelligence and more like a random pattern generator.
2. Hermeneutic Function (Understanding): We understand something better when we can see how it connects to other ideas. For AI outputs to be truly interpretable, they need to be systematically integrated. This means showing how a conclusion is supported by other information, or what its implications are. It’s about seeing the ‘web of reasons’ behind an AI’s statement.
3. Epistemological Function (Acceptability): Systematicity helps us decide if a piece of information is trustworthy. In science, for example, a theory is accepted not just because it fits some data, but because it’s consistent, coherent, and comprehensive within a larger scientific framework. For AI, this means we want its outputs to be justifiable within a broader, systematic body of knowledge.
4. Critical Function: Systematization allows for critical scrutiny. By pushing an AI to explicitly connect its outputs to general principles and show how they are consistently applied, we can more easily spot biases, inconsistencies, or arbitrary decisions. This is vital for ensuring fairness and accountability in AI systems, especially in sensitive applications.
5. Didactic Function (Learning and Persuasion): Well-organized, systematic information is easier to teach, learn, remember, and find persuasive. AI models that can present information in a structured, principled way will be far more effective as educational tools or knowledge interfaces, allowing users to explore topics with varying degrees of depth and connection.
Also Read:
- A New Framework for Understanding Intelligence: The Power of Generative Fidelity
- Transparent AI Collaboration: SynLang and the Philosophy of Shared Intelligence
The Current State and Future of AI Systematicity
While modern large language models (LLMs) are impressive, they still struggle with macrosystematicity, especially over longer conversations or when maintaining a consistent ‘perspective.’ They might contradict themselves across different interactions because their primary training objective is often just to predict the next word, not to be systematically coherent. However, ongoing research is exploring ways to improve this, such as fine-tuning models for consistency, integrating them with reliable databases (Retrieval-Augmented Generation), and using internal ‘self-consistency’ checks.
Ultimately, the paper argues for a dynamic and context-sensitive understanding of systematicity. Not every AI model needs to be maximally systematic all the time. The degree of systematicity required depends on ‘which AI model’ is being used, ‘which function of systematization’ needs to be discharged, and ‘for which human agents’ (e.g., developers, students, judges). By tailoring the demand for systematicity to these specific needs, we can guide the development of AI that is not just explainable, but truly intelligent in a way that aligns with our deepest cognitive ideals.
