Bayesian Convolutional Deep Sets with Task-Dependent Stationary Prior
This work addresses representation ambiguity in stationary stochastic process modeling for machine learning applications, but it is incremental as it builds on existing convolutional deep sets architecture.
The paper tackled the issue of ambiguous representations in convolutional deep sets when data is insufficient by introducing Bayesian convolutional deep sets with a task-dependent stationary prior, achieving improved performance on time-series and image datasets.
Convolutional deep sets are the architecture of a deep neural network (DNN) that can model stationary stochastic process. This architecture uses the kernel smoother and the DNN to construct the translation equivariant functional representations, and thus reflects the inductive bias of the stationarity into DNN. However, since this architecture employs the kernel smoother known as the non-parametric model, it may produce ambiguous representations when the number of data points is not given sufficiently. To remedy this issue, we introduce Bayesian convolutional deep sets that construct the random translation equivariant functional representations with stationary prior. Furthermore, we present how to impose the task-dependent prior for each dataset because a wrongly imposed prior forms an even worse representation than that of the kernel smoother. We validate the proposed architecture and its training on various experiments with time-series and image datasets.