Posterior Contraction for Sparse Neural Networks in Besov Spaces with Intrinsic Dimensionality
This provides theoretical justification for the practical effectiveness of Bayesian neural networks in high-dimensional, structured estimation problems, though it is an incremental theoretical analysis.
This work proves that sparse Bayesian neural networks achieve optimal posterior contraction rates for functions in anisotropic Besov spaces with intrinsic dimensionality, showing these rates depend on the intrinsic dimension rather than the ambient dimension to mitigate the curse of dimensionality. The analysis demonstrates that sparse or continuous shrinkage priors enable rate adaptation even when the true function's smoothness is unknown.
This work establishes that sparse Bayesian neural networks achieve optimal posterior contraction rates over anisotropic Besov spaces and their hierarchical compositions. These structures reflect the intrinsic dimensionality of the underlying function, thereby mitigating the curse of dimensionality. Our analysis shows that Bayesian neural networks equipped with either sparse or continuous shrinkage priors attain the optimal rates which are dependent on the intrinsic dimension of the true structures. Moreover, we show that these priors enable rate adaptation, allowing the posterior to contract at the optimal rate even when the smoothness level of the true function is unknown. The proposed framework accommodates a broad class of functions, including additive and multiplicative Besov functions as special cases. These results advance the theoretical foundations of Bayesian neural networks and provide rigorous justification for their practical effectiveness in high-dimensional, structured estimation problems.