A New PHO-rmula for Improved Performance of Semi-Structured Networks
This work addresses issues in SSNs for researchers and practitioners seeking interpretability and uncertainty quantification, but it is incremental as it builds on existing techniques to improve component identification.
The paper tackles the problem of suboptimal estimation and erroneous predictions in semi-structured neural networks (SSNs) by proposing a non-invasive post-hoc orthogonalization (PHO) method, which guarantees identifiability of model components and improves estimation and prediction quality, as demonstrated in numerical experiments, a benchmark comparison, and a real-world COVID-19 application.
Recent advances to combine structured regression models and deep neural networks for better interpretability, more expressiveness, and statistically valid uncertainty quantification demonstrate the versatility of semi-structured neural networks (SSNs). We show that techniques to properly identify the contributions of the different model components in SSNs, however, lead to suboptimal network estimation, slower convergence, and degenerated or erroneous predictions. In order to solve these problems while preserving favorable model properties, we propose a non-invasive post-hoc orthogonalization (PHO) that guarantees identifiability of model components and provides better estimation and prediction quality. Our theoretical findings are supported by numerical experiments, a benchmark comparison as well as a real-world application to COVID-19 infections.