Uncertainty-Aware Perceiver
This work tackles the problem of improving uncertainty calibration and performance for Perceiver models, but it is incremental as it builds directly on the existing Perceiver architecture.
The paper addresses the Perceiver model's lack of predictive uncertainty estimation and marginal performance improvements by introducing five Uncertainty-Aware Perceiver variants, which achieve considerable performance enhancements on CIFAR-10 and CIFAR-100 datasets.
The Perceiver makes few architectural assumptions about the relationship among its inputs with quadratic scalability on its memory and computation time. Indeed, the Perceiver model outpaces or is competitive with ResNet-50 and ViT in terms of accuracy to some degree. However, the Perceiver does not take predictive uncertainty and calibration into account. The Perceiver also generalizes its performance on three datasets, three models, one evaluation metric, and one hyper-parameter setting. Worst of all, the Perceiver's relative performance improvement against other models is marginal. Furthermore, its reduction of architectural prior is not substantial; is not equivalent to its quality. Thereby, I invented five mutations of the Perceiver, the Uncertainty-Aware Perceivers, that obtain uncertainty estimates and measured their performance on three metrics. Experimented with CIFAR-10 and CIFAR-100, the Uncertainty-Aware Perceivers make considerable performance enhancement compared to the Perceiver.