EKF-based system parameter estimation under high-dimensional low-sample-size data

Abstract

Dynamical Network Biomarkers (DNB) theory has recently emerged as a promising framework for the ultra-early detection of diseases, particularly when dealing with high-dimensional low-sample-size (HDLSS) data. Once such early warning signs are identified, timely intervention becomes essential to prevent the disease from progressing into irreversible states. From the perspective of control theory, such intervention aims to improve the system stability margin to avoid critical transitions, a process known as re-stabilization. Successful re-stabilization requires knowledge of the system parameters. However, the HDLSS nature of biological datasets poses significant challenges for precise system parameter identification. To address this issue, this study explores the application of the extended Kalman filter (EKF) and proposes a novel dual-loop EKF approach. In the inner-loop iteration, we simultaneously estimate both the system states and unknown parameters by augmenting them into a unified model and employing EKF with first-order linearization. Meanwhile, the outer loop iteratively refines these parameter estimates by reusing historical measurement data, eliminating the need for additional data collection. Numerical simulations on low- and high-dimensional systems demonstrate that the proposed dual-loop EKF method significantly improves parameter estimation accuracy compared to the traditional EKF method, highlighting its potential applicability in complex biological contexts.