These labels were automatically added by AI and may be inaccurate. For details, see About Literature Database.
Abstract
We propose an ensemble score filter (EnSF) for solving high-dimensional
nonlinear filtering problems with superior accuracy. A major drawback of
existing filtering methods, e.g., particle filters or ensemble Kalman filters,
is the low accuracy in handling high-dimensional and highly nonlinear problems.
EnSF attacks this challenge by exploiting the score-based diffusion model,
defined in a pseudo-temporal domain, to characterizing the evolution of the
filtering density. EnSF stores the information of the recursively updated
filtering density function in the score function, instead of storing the
information in a set of finite Monte Carlo samples (used in particle filters
and ensemble Kalman filters). Unlike existing diffusion models that train
neural networks to approximate the score function, we develop a training-free
score estimation that uses a mini-batch-based Monte Carlo estimator to directly
approximate the score function at any pseudo-spatial-temporal location, which
provides sufficient accuracy in solving high-dimensional nonlinear problems as
well as saves a tremendous amount of time spent on training neural networks.
High-dimensional Lorenz-96 systems are used to demonstrate the performance of
our method. EnSF provides surprising performance, compared with the
state-of-the-art Local Ensemble Transform Kalman Filter method, in reliably and
efficiently tracking extremely high-dimensional Lorenz systems (up to 1,000,000
dimensions) with highly nonlinear observation processes.