Neural Backward Filtering Forward Guiding
This addresses a specific problem in computational biology and statistics for researchers dealing with complex tree-structured data, representing an incremental improvement by combining existing techniques in a novel way.
The paper tackled the challenge of inference in non-linear continuous stochastic processes on trees with sparse observations, proposing Neural Backward Filtering Forward Guiding (NBFFG) to outperform baselines on synthetic benchmarks and reconstruct ancestral butterfly wing shapes in a high-dimensional phylogenetic task.
Inference in non-linear continuous stochastic processes on trees is challenging, particularly when observations are sparse (leaf-only) and the topology is complex. Exact smoothing via Doob's $h$-transform is intractable for general non-linear dynamics, while particle-based methods degrade in high dimensions. We propose Neural Backward Filtering Forward Guiding (NBFFG), a unified framework for both discrete transitions and continuous diffusions. Our method constructs a variational posterior by leveraging an auxiliary linear-Gaussian process. This auxiliary process yields a closed-form backward filter that serves as a ``guide'', steering the generative path toward high-likelihood regions. We then learn a neural residual--parameterized as a normalizing flow or a controlled SDE--to capture the non-linear discrepancies. This formulation allows for an unbiased path-wise subsampling scheme, reducing the training complexity from tree-size dependent to path-length dependent. Empirical results show that NBFFG outperforms baselines on synthetic benchmarks, and we demonstrate the method on a high-dimensional inference task in phylogenetic analysis with reconstruction of ancestral butterfly wing shapes.