End-to-End Probabilistic Framework for Learning with Hard Constraints
This addresses the challenge of integrating operational or physical constraints into machine learning models for domains like PDEs and time-series forecasting, offering a general framework that bridges these areas.
The paper tackles the problem of incorporating hard constraints into probabilistic forecasting by introducing ProbHardE2E, a framework that uses a differentiable probabilistic projection layer to enable end-to-end learning and uncertainty quantification, achieving robust distributional estimates without distributional assumptions.
We present ProbHardE2E, a probabilistic forecasting framework that incorporates hard operational/physical constraints, and provides uncertainty quantification. Our methodology uses a novel differentiable probabilistic projection layer (DPPL) that can be combined with a wide range of neural network architectures. DPPL allows the model to learn the system in an end-to-end manner, compared to other approaches where constraints are satisfied either through a post-processing step or at inference. ProbHardE2E optimizes a strictly proper scoring rule, without making any distributional assumptions on the target, which enables it to obtain robust distributional estimates (in contrast to existing approaches that generally optimize likelihood-based objectives, which are heavily biased by their distributional assumptions and model choices); and it can incorporate a range of non-linear constraints (increasing the power of modeling and flexibility). We apply ProbHardE2E in learning partial differential equations with uncertainty estimates and to probabilistic time-series forecasting, showcasing it as a broadly applicable general framework that connects these seemingly disparate domains.