TLDR: This research extends Preferential Subspace Identification (PSID), a method for modeling two time-series (e.g., neural activity and behavior), to enable optimal filtering and smoothing. Traditionally, PSID focused on predicting a secondary signal from past primary data. The new “PSID with filtering” learns an optimal Kalman update step for concurrent data, while “PSID with smoothing” uses a novel forward-backward algorithm on signal residuals. Validated on simulated data, these extensions accurately recover model parameters and achieve ideal performance in estimating the secondary signal, significantly enhancing the analysis of dynamic interactions in multivariate time-series.
Understanding and predicting complex systems, especially those involving multiple interacting signals like neural activity and behavior, is a significant challenge in many scientific fields. System identification methods aim to build models that describe how these signals relate to each other over time. One such powerful method is Preferential Subspace Identification, or PSID.
The Foundation: Preferential Subspace Identification (PSID)
Originally, PSID was designed to create a state-space model of a primary time-series (for example, neural activity) to predict a secondary time-series (like behavior). A key feature of PSID is its ability to prioritize learning the dynamics of the primary signal that are most relevant to the secondary signal. However, the initial PSID framework primarily focused on optimal prediction using only past data from the primary signal. This meant it could predict what would happen next, but it wasn’t optimized for estimating the secondary signal using concurrent data (filtering) or all available data (smoothing).
Expanding Capabilities: PSID with Filtering and Smoothing
A new research paper, titled “Preferential subspace identification (PSID) with forward-backward smoothing” by Omid G. Sani and Maryam M. Shanechi, addresses this limitation by extending PSID to enable optimal filtering and smoothing. This is a crucial advancement because, in many offline applications, incorporating more data – either concurrent or all available data – can lead to much better estimations.
The core insight of this work is that the presence of a secondary signal fundamentally changes how certain internal model parameters can be identified. In a single-signal scenario, filtering and smoothing can be trivial, as the best estimate of an observed signal at a given time is simply the observed value itself. However, when a secondary signal is involved, the problem becomes non-trivial, allowing for the identification of models that are optimal for estimating this secondary signal.
PSID with Filtering: Estimating with Concurrent Data
The first extension, “PSID with filtering,” focuses on optimally estimating the secondary signal at a given time step using all primary signal data up to and including that same time step. The researchers found that the presence of the secondary signal makes it possible to uniquely identify a model with an optimal Kalman update step, which is essential for filtering. Instead of directly learning a complex parameter called the Kalman gain (Kf), which is often not uniquely identifiable, the method learns a related parameter, CzKf. This parameter is sufficient for generating the optimal filtered estimate of the secondary signal. This is achieved through a technique called Reduced-Rank Regression (RRR), applied to the residuals (the unexplained parts) of the one-step-ahead predictions.
PSID with Smoothing: Leveraging All Available Data
The second major contribution is a novel “forward-backward PSID smoothing” algorithm. Inspired by established two-filter Kalman smoother formulations, this method works in two main stages:
- First, it applies the newly developed PSID with filtering in the forward time direction.
- Then, it calculates the residual (the error) of the filtered secondary signal.
- Finally, it applies PSID with filtering again, but this time in the reverse time direction, using these residuals as the new secondary signal.
The final smoothed estimate of the secondary signal is then obtained by summing the estimates from both the forward and backward passes. This approach effectively learns optimal filters for both directions directly from the data, leading to highly accurate estimations.
Also Read:
- Rethinking Time-Series Forecasting: The Power of Learnable Dynamics
- Gaze-Guided Robots: Enhancing Efficiency and Robustness with Human-Inspired Vision
Validation and Impact
The researchers rigorously validated their new methods using simulated data. They demonstrated that their approach successfully recovers the ground-truth model parameters for filtering and achieves optimal filtering and smoothing performance for decoding the secondary signal. This performance was shown to match the ideal performance that would be obtained from the true underlying model.
This work provides a principled and powerful framework for optimal linear filtering and smoothing in scenarios involving two interacting signals. It significantly expands the analytical toolkit for understanding dynamic interactions in multivariate time-series, with potential applications ranging from neuroscience (e.g., modeling neural-behavioral data for brain-machine interfaces) to other complex systems. You can read the full paper here.
