TLDR: Researchers have developed a novel dictionary-free method called LSTM-enhanced Deep Koopman to identify linear models of complex nonlinear systems with input delays. This approach uses Long Short-Term Memory (LSTM) networks to capture historical dependencies and encode time delays, outperforming traditional Extended Dynamic Mode Decomposition (eDMD) when true system dynamics are unknown and achieving comparable results when dynamics are known, all while being more computationally efficient. The method offers a robust framework for prediction and control of challenging dynamical systems.
Understanding and controlling complex systems is a major challenge in many fields, especially when these systems are nonlinear and have time delays. Imagine trying to predict the water level in a tank where the inflow rate from a pump takes a few seconds to actually affect the water level. These delays, combined with the system’s inherent nonlinearity, make it very difficult to use traditional linear control methods.
A promising approach to tackle this complexity is the Koopman operator. This mathematical tool helps transform a complicated nonlinear system into a simpler, linear representation in a higher-dimensional space. Once a system is represented linearly, it becomes much easier to predict its behavior, estimate its states, and design effective controls.
Previous methods, like Extended Dynamic Mode Decomposition (eDMD), often rely on a ‘dictionary’ of predefined functions to approximate the Koopman operator. The problem with this is that you need prior knowledge of the system’s underlying dynamics to choose the right functions for your dictionary. If you don’t know the true dynamics, these methods can struggle significantly.
A new research paper, titled “Deep Dictionary-Free Method for Identifying Linear Model of Nonlinear System with Input Delay,” introduces an innovative solution to this problem. Authored by Patrik Val´abek, Marek Wadinger, Michal Kvasnica, and Martin Klau ˇco†from the Slovak University of Technology in Bratislava, this paper proposes an LSTM-enhanced Deep Koopman model. You can read the full paper here: Deep Dictionary-Free Method for Identifying Linear Model of Nonlinear System with Input Delay.
The key innovation of this new model is that it is ‘dictionary-free.’ Instead of relying on predefined functions, it uses deep neural networks, specifically Long Short-Term Memory (LSTM) layers, to learn the optimal transformations adaptively during training. LSTM layers are particularly good at capturing historical dependencies, which is crucial for systems with time delays. By processing the system’s past behavior, the LSTM layer efficiently encodes these time delays into a latent space, reducing the need for large, complex Koopman matrices that would be computationally expensive to handle.
The architecture of the LSTM-enhanced Deep Koopman model is built upon an autoencoder framework. It takes the history of the system’s states and inputs, processes them through an LSTM layer, and then combines the resulting hidden states with the current state. This combined information is then ‘lifted’ into a higher-dimensional space by an encoder network. In this lifted space, the system’s dynamics are represented linearly using Koopman matrices. Finally, a decoder network projects these lifted states back to the original system states.
To ensure accurate and stable learning, the model uses a comprehensive loss function. Unlike simpler methods that might only consider one-step predictions, this approach incorporates reconstruction loss, one-step output prediction loss, and crucially, multi-step output and latent trajectory prediction losses. This multi-step prediction capability is vital for accurately capturing the long-term dynamics and the effects of input delays.
The researchers tested their method on a simulated two-tank system with input delays, comparing its performance against eDMD. The results were compelling. When the true nonlinear dynamics of the system were unknown (a common scenario in real-world applications), the LSTM-enhanced Deep Koopman model significantly outperformed eDMD in prediction accuracy, showing more than a threefold improvement in Mean Absolute Error (MAE). It effectively captured the system’s dynamics without needing prior knowledge of the nonlinear terms.
Even when eDMD was given the advantage of knowing the true nonlinear dynamics (e.g., the square root of water levels), the LSTM-enhanced Deep Koopman model achieved comparable results, with only a 6% higher MAE. This demonstrates its robustness and effectiveness even against methods with perfect prior knowledge. Furthermore, the LSTM-enhanced Deep Koopman model achieved this with a smaller number of lifted states, resulting in an almost four times smaller Koopman matrix, which is more efficient to compute and store.
Also Read:
- Unlocking Chaotic Dynamics: A New Approach to Time Series Prediction
- DeepContour: AI-Driven Efficiency for Generalized Eigenvalue Problems
In conclusion, this dictionary-free method offers a powerful new way to identify linear representations of complex nonlinear systems with input delays. By leveraging deep learning, particularly LSTM networks, it overcomes the limitations of traditional Koopman operator methods, providing more accurate and consistent predictions, especially when the true system dynamics are unknown. This advancement paves the way for improved prediction, estimation, and control of challenging dynamical systems, with future work focusing on applying this model to even more complex systems and in model predictive control.
