Accounting for model errors in probabilistic linear identification of nonlinear PDE systems

Minisymposium presentation at 16th U.S. National Congress on Computational Mechanics. The slides can be found here.

System identification of dynamical systems is important in many applications for discovering dynamical properties and behavior of a given system from data. Recent work has focused on advancements in modeling complex systems through the model parameterization by increasingly sophisticated neural networks and through introducing model structure that enforces certain dynamical properties of the desired model, e.g., stability or energy conservation. Across all of these different contributions, however, nearly every method suffers from at least one of two common drawbacks: the method requires a large amount of data to train and/or the method’s performance drops dramatically when the training data are sparse/noisy and the models are mismatched with the truth. We propose to mitigate these issues through the use of an objective function that accounts for the uncertainty in the model that other methods struggle to manage. Our objective is derived from first principles utilizing a probabilistic model of the system dynamics such that the three primary sources of uncertainty present in every system identification problem are addressed: model, measurement, and parameter uncertainty. In contrast, the methods mentioned earlier most commonly rely on a least squares-based approach, which effectively bundles these three sources of uncertainty into one, often ignoring how they may interact. In fact, it can be provably shown that some of the most common objective functions do not account for at least one source of uncertainty, resulting in the method’s performance suffering greatly. The most notable difference in our modeling from least squares-based algorithms is the inclusion of process noise regardless of whether we are interested in learning a deterministic or stochastic system. We empirically show that directly accounting for these three sources of uncertainty significantly increases the robustness of our algorithm compared to approaches that exclude these uncertainties. In this work specifically, we present an objective function for learning linear subspace models. We demonstrate that this algorithm can effectively capture nonlinear dynamics through the approximation of the Koopman operator, can learn low-dimensional representations of high-dimensional PDE systems, and can even learn chaotic systems due to our robust management of uncertainty. Furthermore, we show that our method consistently outperforms existing least squares-based algorithms and is able to closely approximate a given dynamical system when the data are noisy and/or sparse beyond what most existing algorithms can handle.