TLDR: This research paper introduces “core and periphery” principles, derived from the Law of Requisite Variety, to explain how biological and artificial intelligence systems adapt and scale. The “core” represents stable, invariant system elements, while the “periphery” represents dynamic, context-interacting elements. The paper provides empirical evidence from DNA/neurons, homeostasis/homeodynamics, and a CNN experiment, showing these principles apply across intelligent systems. It also defines core-dominant and periphery-dominant systems, offering new insights for engineering resilient intelligent systems by strategically designing their stable and adaptive components.
In the realm of engineering, traditional methods often rely on breaking down complex problems into smaller, manageable parts. This approach works well for many systems, but it faces significant challenges when applied to intelligent systems, especially when considering how intelligence scales as a system property. A recent research paper introduces a fresh perspective: the “core and periphery” principles, a conceptual framework rooted in abstract systems theory and the Law of Requisite Variety. This framework offers a new way to understand and engineer both biological and artificial intelligence.
The core idea behind this research is derived from Ashby’s Law of Requisite Variety, a fundamental principle from early cybernetics. This law states that for a system to remain stable, its internal “variety” – essentially, the number of distinct elements or states it can exhibit – must be greater than or equal to the variety present in its environment or context. If a system’s variety doesn’t match the complexity of its context, it struggles to achieve precise and predictable outcomes. The paper expands on this by defining “core” as the invariant elements of a system that remain constant over time, largely decoupled from external changes. Conversely, the “periphery” consists of elements that are dynamic and change in response to the system’s context, representing its direct interaction with the environment.
The distinction between core and periphery is crucial for understanding how intelligent systems adapt and scale. Adaptation, according to the authors, is primarily a function of the periphery, which is coupled with the system’s context. New information and changes from the environment first enter through the periphery. As the system adapts, these changes can eventually influence and integrate into the more stable core. This dynamic interplay suggests that while the core provides stability, the periphery drives the system’s ability to learn and evolve.
When it comes to scaling intelligence, the paper argues that current approaches in Artificial Intelligence and Machine Learning often focus on scaling inputs and outputs, which represents only a limited aspect of system variety. True scaling of intelligence, particularly when the context’s complexity grows indefinitely, necessitates a shift towards engineering systems where the unbounded variety of the periphery plays a dominant role. This means embracing “closed system phenomena,” where the system’s boundaries with its context become less distinct, allowing for deeper interaction and adaptation.
The researchers provide compelling empirical evidence from both biological and engineered systems to support their core and periphery precepts. In biological intelligence, they draw parallels between DNA and neurons. DNA, with its slow adaptation over generations and stable characteristics, can be seen as the core. Neurons, with their rapid self-reorganization, formation of new connections, and ability to adapt through learning, represent the dynamic periphery. Similarly, the body’s ability to maintain a stable internal environment (homeostasis) aligns with the core, while its dynamic equilibrium and adaptive responses (homeodynamics), like the immune system’s ability to respond to new threats, align with the periphery. Even in human cognition, stable “beliefs” are considered part of the core, while dynamic “interrogative attitudes” – the process of questioning and seeking new information – reside in the periphery.
To demonstrate these concepts in engineered intelligent systems, the paper describes an experiment using a Convolutional Neural Network (CNN). By training a ResNet-50 model on one dataset and then retraining it on a more complex one, the researchers observed how the weights in the network’s layers changed. Layers with minimal changes in their weights, indicating low entropy, were identified as core components, while layers exhibiting significant weight variations, showing high entropy, were characterized as periphery. This experiment suggests that even in relatively simple AI models, the patterns of core and periphery are evident, offering a potential pathway to design intelligent systems with desired outcomes by strategically structuring these components.
The paper further introduces the concepts of “core-dominant” and “periphery-dominant” systems. A core-dominant system has greater variety in its core, meaning its characteristics are more dependent on the stable behaviors of the core. These systems can be modeled primarily using an “open-view” paradigm, focusing on clear input-output relationships. In contrast, a periphery-dominant system has greater variety in its periphery, with its outcomes more influenced by changes in the context. These systems require a “closed-view” paradigm, acknowledging the strong coupling and dissolution of boundaries with the environment.
Also Read:
- Unlocking Deep Learning’s Success: The Power of Compositional Sparsity
- Unpacking Undecidability: A New Theory on Systemic Limits
These findings have significant implications for Systems Engineering (SE). The core and periphery principles offer a new lens through which to re-evaluate traditional decomposition methods, guiding engineers on when and how to model system elements based on their openness or closedness to behavior and structure. This framework could lead to better articulation of system architectures, subsystem specifications, and even inform how intelligent systems can be developed to be more resilient, by identifying which functions should be adaptable and which should remain constant. For a deeper dive into this groundbreaking research, you can access the full paper here.
