TLDR: Mark Burgess’s research paper introduces a novel method for detecting ‘intentionality’ in data, particularly text, using a ‘Tiny Language Model’ based on Promise Theory. The approach analyzes the ‘work’ invested in forming patterns and their repetition rates to distinguish purposeful signals from ambient background noise. This low-computational-cost method avoids extensive training and complex reasoning, offering a pragmatic way to build knowledge representations and index information.
In the vast and complex world of artificial intelligence and cognitive science, understanding the concept of ‘intent’ has long been a challenging frontier. While philosophers like Searle have explored intentionality in depth, its practical application in technology has often been overlooked. A new research paper by Mark Burgess, titled “On The Role of Intentionality in Knowledge Representation Analyzing Scene Context for Cognitive Agents with a Tiny Language Model,” delves into this very topic, proposing a novel and surprisingly simple approach to identifying intentionality in data.
Burgess introduces a framework rooted in Promise Theory’s model of Semantic Spacetime, which he effectively uses as a “Tiny Language Model.” Unlike large, computationally intensive language models that require extensive training and reasoning, this approach aims to detect latent intentionality at a very low computational cost. The core idea revolves around analyzing “process coherence” and identifying “anomalous multi-scale anomalies” in data streams. Essentially, it looks for patterns that stand out from the ambient background, assessing the ‘work’ done to form them.
The paper distinguishes between ‘intent’ (directed efforts towards a goal) and ‘intentionality’ (the general capacity to project or adopt such intentions). In a cognitive scenario, intentionality isn’t about deciphering what an actor might be thinking or feeling, but rather determining if their behavior is distinguishable from its background. Does a scene unfold randomly, or in a more purposeful manner? This method seeks to answer that question.
A key heuristic for intent, as described, is related to thermodynamic work. Singleton or spurious events have low intentionality. Events that repeat suggest greater causal intent, but there’s a ‘Goldilocks’ zone: patterns that repeat too often become mere padding or background noise. The most intentional content lies in the middle range, where effort is invested to make a signal stand out without becoming habitual.
The paper introduces a “symbolic fractionation method” for analyzing data streams, particularly narrative texts. This involves dividing the stream into elementary sequences (n-grams) and assessing their intentionality. By measuring the distance between n-gram occurrences and looking for significant pauses, the method can distinguish between contextual (frequently repeated, ambient) and intentional (unique, original, anomalous) fragments. For instance, in Darwin’s “Origin of Species,” common phrases like “conditions of life” might be ambient, while less frequent but significant phrases like “origin of species” or “struggle for existence” are identified as highly intentional.
What makes this approach particularly compelling is its efficiency. It doesn’t require vast training corpuses or complex probabilistic calculations. Instead, it relies on basic principles of process dynamics, making it accessible even to systems with limited memory capacity. This low-brow approach to cognition, as the author puts it, can cheaply build robust knowledge graphs that can then be queried using the Semantic Spacetime model, sidestepping the need for complex ontologies.
The practical applications are significant, especially in knowledge representation and indexing. By identifying the most significant, intentional parts of a text, a machine can effectively distill core concepts and themes, providing entry points for understanding longer narratives. This is crucial for projects like the SSTorytime project, which aims to represent knowledge from narrative texts. For more details, you can read the full research paper here.
Also Read:
- Beyond Mimicry: Unpacking How Large Language Models Develop Understanding
- Enhancing AI Safety: Small Models Outperform Large Counterparts with Synthetic Data and Adversarial Training
In conclusion, Burgess’s work offers a refreshing perspective on intentionality, moving beyond complex linguistic and philosophical debates to a pragmatic, energetic explanation. By focusing on the ‘work’ and ‘coherence’ of patterns in data, it provides a powerful, low-cost tool for cognitive agents to discern purpose and build knowledge, even with a tiny language model.
