TLDR: DeepFHT is a novel survival analysis framework that integrates deep neural networks with first hitting time (FHT) distributions from stochastic process theory. It models time-to-event as the first passage of a latent diffusion process to an absorbing boundary, where a neural network maps input features to physically meaningful process parameters (initial condition, drift, diffusion). This approach provides closed-form survival functions, captures time-varying risk without assuming proportional hazards, and offers physics-based interpretability, allowing researchers to understand the relationship between input features and risk. DeepFHT achieves competitive predictive accuracy, outperforming traditional methods like Cox regression in scenarios with non-proportional hazards.
Survival analysis is a critical field across medicine, engineering, economics, and finance, focusing on predicting the time until a specific event occurs, such as disease progression, equipment failure, or financial default. Unlike standard prediction tasks, survival data often includes ‘censoring,’ where the exact event time isn’t observed for all subjects, making it a unique challenge.
Traditional methods, like the widely used Cox proportional hazards (CoxPH) model, have been foundational but come with limitations. The CoxPH model assumes that the instantaneous risk of an event for two individuals differs by a constant factor over time, known as the proportional hazards assumption. This assumption is often unrealistic in real-world scenarios. Additionally, the Cox model is typically linear, which can miss complex interactions between different input features.
To address these limitations, researchers have explored alternative approaches. Among these, First Hitting Time (FHT) models are particularly promising. FHT models conceptualize event times as the moment a latent (hidden) stochastic process, underlying the observed event, crosses a certain barrier. This framework naturally allows for time-varying risk, moving beyond the restrictive proportional hazards assumption.
However, existing FHT methods have faced a dilemma: some machine learning and deep neural network-based FHT models offer strong predictive performance but act as ‘black boxes,’ providing little insight into the underlying process. Conversely, parametric FHT models are more physically interpretable but often lack the expressiveness to capture complex data relationships.
Introducing DeepFHT: A Hybrid Approach
A new framework called DeepFHT aims to bridge this gap by combining the flexibility of deep neural networks with the physics-based interpretability of FHT distributions. DeepFHT represents the time-to-event as the first passage of a latent diffusion process to an absorbing boundary. The core innovation lies in using a deep neural network to map input variables (features) to physically meaningful parameters of this stochastic process. These parameters can include the initial condition, drift, and diffusion coefficient, within a chosen FHT process like Brownian motion (with or without drift).
This integration yields several significant advantages. Firstly, it provides closed-form survival and hazard functions, which are essential for understanding risk over time. Secondly, it inherently captures time-varying risk without needing the proportional hazards assumption. Most importantly, DeepFHT maintains a physics-based interpretable parameterization, offering clear insights into how input features influence the underlying risk process.
Also Read:
- Next Event Prediction: Enhancing AI’s Understanding of Patient Journeys in Electronic Health Records
- Fiaingen: Generating High-Quality Synthetic Financial Time Series with Graph-Based Methods
Performance and Interpretability
DeepFHT was rigorously tested against traditional Cox regression and other parametric survival models using both synthetic and real-world datasets. The results showed that DeepFHT achieves predictive accuracy on par with state-of-the-art approaches. Crucially, it significantly outperformed Cox regression in synthetic datasets specifically designed to violate the proportional hazards assumption and include nonlinear relationships, highlighting its robustness in complex scenarios.
The true strength of DeepFHT lies in its interpretability. By mapping each individual to a point in a latent parameter space, the model allows for a natural understanding of survival dynamics. Researchers can visualize how different parameters (like initial position, drift, and diffusion) influence the process. For instance, patients with longer survival times tend to cluster in regions of the parameter space associated with favorable initial conditions (further from the absorbing barrier) and lower diffusion or drift.
The paper illustrates this with examples: in the GBSG2 dataset, patients with low-grade tumors were mapped to low-risk regions (favorable initial conditions, weaker drift), while those with high-grade tumors clustered in high-risk regions (closer to the barrier, stronger drift). Similarly, in the Framingham dataset, individuals with higher blood pressure were found in high-risk areas of the parameter space. This alignment between clinical risk factors and model-derived parameters strongly supports DeepFHT’s physics-based interpretability.
DeepFHT represents a principled advancement in survival analysis, offering a powerful combination of deep learning’s flexibility and stochastic process theory’s interpretability. It provides a controlled alternative to black-box survival models, grounding predictions in a dynamic description that connects input features to the evolution of an underlying stochastic process. For more details, you can read the full research paper here.
