TLDR: LLM-Filter is a new framework that uses large language models (LLMs) for state estimation in dynamic systems. It converts noisy observations into text, uses “System-as-Prompt” (SaP) to guide a frozen LLM, and then projects the LLM’s output to state estimates. This approach significantly outperforms traditional and learning-based filters, demonstrating exceptional generalization to new and unseen systems without retraining, and exhibits scaling-law behavior with larger LLMs.
State estimation is a fundamental challenge across many scientific and engineering fields, from robotics to meteorology. It involves determining the hidden internal states of a dynamic system based on noisy external observations. Traditionally, methods like Kalman filters and particle filters have been used, but they often struggle with complex, high-dimensional, or non-Gaussian systems, or demand significant computational resources.
The Challenge of Generalization
More recently, learning-based filtering methods have emerged, offering improved accuracy and efficiency by training on specific datasets. However, a major drawback is their lack of generalization. These filters are typically designed for a particular task or system and require extensive retraining when applied to different environments or if the system characteristics change. This limitation makes them impractical for diverse, real-world applications.
Introducing LLM-Filter: A Generalist Approach
Inspired by the success of large language models (LLMs) in generalizing across various tasks in fields like control, researchers have introduced a novel framework called LLM-Filter. This groundbreaking approach leverages the powerful reasoning capabilities and vast pre-trained knowledge embedded within LLMs to perform state estimation. Unlike conventional learning-based filters, LLM-Filter is designed to be a “generalist filter,” capable of adapting to new or changed systems without needing to be retrained.
How LLM-Filter Works
The LLM-Filter operates through a carefully designed three-step process:
First, Observation Embedding: Noisy, continuous observations from the dynamic system are transformed into discrete “text prototypes” that the LLM can understand. This involves segmenting the observation data and embedding each segment into the LLM’s latent space, ensuring that crucial correlations between variables are preserved.
Second, In-context Inference with System-as-Prompt (SaP): This is a key innovation for generalization. Instead of retraining the LLM, LLM-Filter uses a unique prompting strategy called System-as-Prompt (SaP). SaP provides the LLM with task-specific instructions and examples, guiding it to understand the current estimation task and adapt to the specific system. The core layers of the pre-trained LLM remain frozen, preserving its vast knowledge base.
Third, State Projection: The outputs generated by the LLM, which are in its internal token embedding space, are then projected back into the system’s state space to produce the final state estimates.
Exceptional Performance and Generalization
Experiments demonstrate that LLM-Filter consistently outperforms state-of-the-art learning-based filters and online Bayes filters across a range of classical dynamical systems, including low-dimensional nonlinear systems (like Selkov, Oscillator, Hopf, Double Pendulum) and high-dimensional chaotic systems (like Lorenz96 and VL20). In some cases, it achieved an average improvement of over 17% compared to the best learning-based methods.
Its generalization capabilities are particularly impressive. LLM-Filter shows remarkable robustness to “model mismatch,” where the assumed observation noise differs from reality, maintaining accuracy with minimal degradation even as mismatch severity increases. Furthermore, in “cross-system” scenarios—where the model is trained on one system and tested on a completely different one—LLM-Filter delivers competitive performance, often outperforming methods specifically trained on the target system. This ability to generalize to unseen environments without retraining is a significant breakthrough.
Insights into LLM Behavior
The research also explored the inherent characteristics of LLMs within the filtering context. It confirmed a “scaling-law” behavior, where increasing the size of the underlying LLM (e.g., from GPT-2 to LLaMA-13B) generally leads to improved estimation accuracy, albeit with longer training times. An ablation study further solidified the critical role of LLMs, showing that LLM-Filter significantly outperforms traditional neural network architectures like MLPs, RNNs, and Transformers when used as the core component.
Also Read:
- Beyond Pre-Training: How Experience Scaling Enables Continuous Learning for Large Language Models
- LatentGuard: Enhancing LLM Safety Through Interpretable Internal Control
A Promising Future for State Estimation
LLM-Filter represents a significant step towards a generalist foundation model for state estimation. By effectively harnessing the pre-trained knowledge and reasoning abilities of large language models, it offers a powerful and adaptable solution for complex dynamic systems. While current generalization is limited to systems of the same dimensionality, future work aims to extend this to multi-source data and varying dimensions, paving the way for broader real-world applications. You can read the full research paper here: One Filters All: A Generalist Filter for State Estimation.
