TLDR: The research paper “Natural Latents: Latent Variables Stable Across Ontologies” by John Wentworth and David Lorell introduces a mathematical framework to explain how different agents (like humans and AI) can guarantee translatability of their internal concepts, even if their models of the world differ. The key lies in ‘Natural Latents,’ which are latent variables that satisfy two conditions: ‘Mediation’ (they make observations independent) and ‘Redundancy’ (they can be determined from any single observation). The paper proves that an agent’s latent is translatable to another’s if and only if it is a natural latent, providing a robust foundation for communication, scientific theory development, and understanding why different minds converge on similar concepts.
Imagine two different minds, perhaps a human and an AI, or even two different AI systems, trying to understand the same world. They might build very different internal models, using different ‘latent variables’ – the hidden concepts or factors they use to make sense of things. Yet, in practice, we often see that these different minds can still communicate and even converge on similar understandings. Think about how easily babies learn new words, how AI models develop human-interpretable features, or how different neural networks can be successfully merged. This suggests a deep underlying compatibility, but the mathematical reasons for this have been elusive.
A new research paper, “Natural Latents: Latent Variables Stable Across Ontologies” by John Wentworth and David Lorell, tackles this fundamental problem. The authors set out to find the conditions under which one agent can guarantee that its internal concepts (latent variables) are directly translatable into another agent’s concepts, even if their internal models are structured differently. The crucial assumption is that both agents agree on what they observe in the world, even if their explanations for those observations differ.
The Problem of Indeterminacy and Practical Convergence
The challenge lies in what philosophers call the ‘problem of indeterminacy.’ Different models can make identical predictions about the world while using radically different internal structures. This makes it difficult to guarantee that one model’s hidden concepts will map cleanly onto another’s. However, as the paper highlights, empirical evidence strongly suggests that a high degree of convergence of internal concepts is not just possible, but often the default outcome across humans, between humans and AI, and among different AI systems.
Introducing Natural Latents: The Key to Translatability
The paper introduces a mathematical framework to address this, centered on two key conditions for latent variables: Mediation and Redundancy. When a latent variable satisfies both, it’s called a ‘Natural Latent.’
-
Mediation: A latent variable ‘mediates’ between different observations if, once you know the value of that latent, the observations become independent of each other. For example, if you know the bias of a coin, then each individual coin flip is independent of the others. The bias acts as the mediator.
-
Redundancy: A latent variable is ‘redundant’ over a set of observations if you can determine its value from any single one of those observations. For instance, if you’re looking at different small patches of a picture of a bike, you can likely determine the bike’s color from any one of those patches. The color is redundantly represented across the patches.
A ‘Natural Latent’ is a variable that is both a mediator and a redund. The paper offers an intuitive example: the temperature of an ideal gas. Each small chunk of the gas has the same temperature (redundancy), and the low-level states of different chunks are independent given that temperature (mediation).
Guaranteed Translatability: The Core Theorem
The central finding of the paper is Theorem 2, which states that an agent (Alice) can guarantee her latent variable is a function of another agent’s (Bob’s) latent variable – meaning they are translatable – if and only if Alice’s latent is a ‘natural latent’ over the observed variables. This holds true even when allowing for approximation errors, making the theory robust for real-world applications.
This means that if Alice designs her model to use natural latents, she can be confident that her internal concepts will be understandable and mappable to Bob’s, as long as Bob’s latents also mediate the observed interactions. Natural latents are shown to be ‘minimal’ among mediators (they contain just enough information to explain interactions) and ‘maximal’ among redunds (they capture all the information that is consistently available across observations).
Intuition and Practical Examples
The paper provides several intuitive examples to solidify the concept:
-
Ideal Gas: As mentioned, temperature is a natural latent. You can estimate it from any part of the gas, and knowing the temperature makes the parts’ low-level states independent.
-
Biased Die: The underlying bias of a die is a natural latent over many rolls. You can estimate the bias from a subset of rolls, and knowing the bias makes the rolls independent.
-
Timescale Separation in Markov Chains: In systems that evolve over time, components that are conserved over long periods can act as natural latents, mediating between initial and later states.
Also Read:
- Unpacking How Language Models Simulate Reality: A Coin Toss Perspective
- Unveiling the Interpretive Lenses of AI: A New Framework for Understanding Semantic Models
Implications for AI, Science, and Communication
The concept of natural latents has profound implications:
-
Robust Human-AI Communication: By identifying and using natural latents, humans and AI systems can establish a common ground for understanding, ensuring that their internal concepts are genuinely translatable.
-
Forward-Compatible Scientific Theories: Scientists can use natural latents to build models whose concepts are more likely to carry over and remain relevant in future, more advanced theories.
-
Convergence of Concepts: The properties of natural latents – being minimal mediators and maximal redunds – suggest why different intelligent systems, when optimized for prediction or robustness, might naturally converge on similar internal representations.
In essence, this research provides a crucial mathematical foothold for understanding how robust translation of concepts is possible across different ‘ontologies’ or ways of structuring knowledge. It offers a path for designing AI systems that are inherently more interpretable and compatible with human understanding, and for building scientific theories that are more resilient to future paradigm shifts.
