TLDR: This research paper introduces a semantic information theory for Large Language Models (LLMs), shifting the fundamental unit of analysis from ‘bits’ to ‘tokens’. It leverages concepts from rate-distortion function, directed information, and Granger causality to develop structure-agnostic measures for LLM pre-training, post-training, and inference phases. The paper defines token-level semantic embedding, proposes an information-theoretically optimal vectorization method, and provides a general definition of autoregressive LLMs, from which the Transformer architecture and its performance metrics are derived. It also discusses other architectures like Mamba/Mamba2 and LLaDA within this framework, offering a theoretical foundation for understanding LLMs’ capabilities and limitations.
Large Language Models (LLMs) have captivated the world with their impressive abilities, but understanding how they work remains a significant challenge. While much research focuses on experimental advancements, these often demand vast computational power and data. A new paper, “Forget BIT, It is All about TOKEN: Towards Semantic Information Theory for LLMs,” by Bo Bai, proposes a theoretical framework to open this ‘black box’ by introducing a semantic information theory where the fundamental unit is the ‘token’ rather than the traditional ‘bit’.
The inspiration for this shift comes from a reflection on the information age’s greatest invention. While the ‘bit’ was crucial for reliable information transmission, it lacks semantic meaning. This paper argues that for the AI age, especially with the rise of LLMs, the ‘token’ holds similar fundamental importance. Tokens, unlike bits, carry semantic meaning, which is essential for understanding how LLMs process and generate human-like language.
The work builds upon Shannon’s seminal 1948 information theory, which primarily addressed the ‘technical problem’ of accurately transmitting symbols (Level-A). However, Shannon himself acknowledged that his theory didn’t delve into the ‘semantic problem’ (Level-B) – how precisely symbols convey desired meaning – or the ‘effectiveness problem’ (Level-C) – how received meaning affects conduct. This new semantic information theory extends Shannon’s ideas to focus on tokens, aiming to explain the underlying principles of LLMs from a semantic perspective.
A key aspect of this theory is the representation of token-level semantics. Early efforts to represent and retrieve information using vectors paved the way for modern word embeddings. Techniques like Word2Vec, GloVe, and ELMo demonstrated that words could be represented as vectors in a way that captures meaningful semantic relationships. This vector representation has become the cornerstone of modern Natural Language Processing (NLP) and LLMs, allowing machines to process meaning computationally.
The paper models LLMs as ‘next-token predictors’ and introduces several information-theoretic measures to analyze their behavior across different phases:
Pre-training Phase
During pre-training, LLMs learn to predict the next token in a sequence. The paper defines a ‘directed rate-distortion function’ as a universal measure for this phase. This function helps quantify the minimum information needed from an input sequence to generate an expected output sequence with a certain level of ‘distortion’ (difference from the desired output). By minimizing this directed information, the LLM can filter out irrelevant information, potentially reducing issues like hallucinations.
Post-training Phase
After pre-training, LLMs undergo post-training, often involving reinforcement learning from human feedback (RLHF), to align their outputs with human preferences. For this, the paper introduces a ‘directed rate-reward function’. This measure helps evaluate how well the generated sequence aligns with human preferences, showing that LLMs approximate human-level Granger causality for next-token prediction after this phase.
Inference Phase
In the inference phase, where the LLM generates text based on an input, the ‘semantic information flow’ is analyzed. This flow, defined as the directed information density, helps understand how semantic information accumulates as the LLM generates each subsequent token. The paper also suggests that this framework provides a theoretical foundation for ‘prompt engineering’, which involves crafting inputs to guide LLMs effectively.
The Transformer Architecture
The widely used Transformer architecture, known for its attention mechanism, is shown to be a special case within this general framework of autoregressive LLMs (AR-LLMs). The attention mechanism, which assigns varying importance (semantic relevance) to different tokens in a sequence, is mathematically derived. The paper also discusses the Evidence Lower Bound (ELBO) for Transformer training and inference, and analyzes its generalization error bound and memory capacity.
Also Read:
- Understanding LLM Decisions: A New Look at Explainability with llmSHAP
- Unveiling the ‘Accuracy Cliff’: Why LLMs Struggle with Long Deterministic Tasks
Beyond Transformers
The framework also discusses other emerging LLM architectures like Mamba/Mamba2 and LLaDA. Mamba/Mamba2, which use discrete state space models, are identified as linear time-varying vector autoregression (TV-VAR) models, offering computational efficiency but potentially less semantic richness than Transformers due to the absence of a bilinear model for semantic relevance. LLaDA, a diffusion-based LLM, still relies on AR-LLM principles for masked token prediction.
This research provides a comprehensive theoretical framework for understanding LLMs through the lens of semantic information theory. By shifting the focus from ‘bits’ to ‘tokens’, it offers new tools to analyze LLM capabilities, explain experimental observations, and guide future research, particularly in addressing complex challenges like counterfactual reasoning and System 2 reasoning abilities. For more in-depth technical details, you can refer to the full research paper: Forget BIT, It is All about TOKEN: Towards Semantic Information Theory for LLMs.
