TLDR: CAST (Compositional Analysis via Spectral Tracking) is a new probe-free framework that analyzes Transformer layer functions by estimating direct transformation matrices and applying spectral analysis. It reveals distinct information processing strategies between encoder-only and decoder-only models: decoders exhibit compression-expansion cycles with three functional phases (feature extraction, compression, specialization), while encoders maintain consistent high-rank processing. The framework uses six interpretable metrics derived from singular value decomposition and incorporates kernel analysis to capture nonlinear dynamics, offering a transformation-centric view that complements existing interpretability methods.
Large language models (LLMs) have revolutionized many fields, but their inner workings often remain a mystery. Researchers are constantly looking for ways to peek inside these complex ‘black boxes’ to understand how they process information. A new framework called CAST, which stands for Compositional Analysis via Spectral Tracking, offers a fresh perspective by analyzing how information is transformed within each layer of a Transformer model.
Developed by Zihao Fu, Ming Liao, Chris Russell, and Zhenguang G. Cai, CAST is a ‘probe-free’ method. This means it doesn’t require additional training or auxiliary models, unlike some other interpretability techniques. Instead, it directly estimates the mathematical transformations happening between consecutive layers of an LLM and then uses spectral analysis to understand their characteristics.
How CAST Works
The CAST framework has two main components: linear transformation estimation and spectral analysis, complemented by kernel analysis for non-linear aspects.
First, CAST approximates the function of each Transformer layer as a linear transformation. While Transformer layers are known for their non-linear complexity, the researchers found that a linear approximation captures a substantial part of their behavior. This approximation is achieved by estimating a ‘transformation matrix’ for each layer using a technique called the Moore-Penrose pseudoinverse.
Once these transformation matrices are estimated, CAST applies spectral analysis, specifically Singular Value Decomposition (SVD). SVD breaks down each matrix into singular values, which essentially quantify the strength of information transformation in different directions. From these singular values, CAST derives six interpretable metrics:
- Effective Rank (ER): Measures the intrinsic dimensionality of the transformation, indicating whether a layer is expanding features or compressing information.
- Spectral Decay Rate (SDR): Quantifies how rapidly the importance of transformation directions decreases, revealing the layer’s compression strategy.
- Transformation Entropy (TE): Assesses the complexity of singular value distribution, showing if processing is multi-faceted or focused.
- Anisotropy Index (AI): Measures directional bias, indicating if a layer has specialized sensitivities to certain input patterns.
- Information Concentration (IC): Uses the Gini coefficient to quantify inequality in singular value distribution, revealing the degree of functional specialization.
- Residual Norm (RN): Measures the proportion of a layer’s output that cannot be explained by the linear transformation, indicating the degree of nonlinearity.
To further understand the non-linear dynamics, CAST also incorporates kernel analysis using Random Fourier Features (RFF) and Centered Kernel Alignment (CKA). This helps validate the linear approximations and reveals patterns of functional similarity between layers.
Key Discoveries Across LLM Architectures
The researchers applied CAST to four representative Transformer architectures: GPT-2, RoBERTa, Llama-3.2-1B, and DeepSeek-R1-Distill-Qwen-1.5B, using the WikiText-103 dataset. Their findings revealed fundamental differences in how encoder-only and decoder-only models process information:
- Decoder-only Models (GPT-2, Llama, DeepSeek-R1): These models, designed for sequential prediction, exhibit consistent ‘compression-expansion cycles.’ Early layers expand features, middle layers aggressively compress information (acting as an ‘information bottleneck’), and later layers re-expand for task-specific output generation. The effective rank drops sharply in the middle layers before recovering.
- Encoder-only Models (RoBERTa): In contrast, RoBERTa, a bidirectional encoder, maintains a consistently high effective rank throughout its layers. This suggests that encoders preserve comprehensive representations across their depth, suitable for various downstream tasks without committing to specific predictions.
Kernel analysis further supported these findings, demonstrating that layers can be clearly partitioned into three functional phases: feature extraction, compression, and specialization. The middle compression layers in decoder models also showed the strongest nonlinearity, indicating complex transformations during abstraction.
Also Read:
- Adaptive Transformers: Balancing Speed and Accuracy with Dynamic Computation
- Enhancing Model Merging: A Knowledge-Aware Approach to Refine Task Vectors
A Complementary View
CAST offers a unique ‘transformation-centric’ lens that complements existing interpretability methods. While techniques like the Logit Lens and Tuned Lens focus on how predictions evolve, DirectProbe examines representation geometry, and Attention Entropy quantifies attention patterns, CAST specifically measures the complexity and dynamics of the transformations themselves. This provides a more complete picture of how information flows and is processed within these powerful models.
The insights gained from CAST can guide practical applications such as model compression, efficient training strategies, and the design of future LLM architectures. By providing mathematical tools to understand Transformer computations, CAST opens new avenues for developing more transparent and interpretable language models. For more details, you can read the full research paper here.
