Axel Flinth

Axel Flinth, Pierre Weiss

We study the solutions of infinite dimensional linear inverse problems over Banach spaces. The regularizer is defined as the total variation of a linear mapping of the function to recover, while the data fitting term is a near arbitrary convex function. The first contribution is about the solu-tion's structure: we show that under suitable assumptions, there always exist an m-sparse solution, where m is the number of linear measurements of the signal. Our second contribution is about the computation of the solution. While most existing works first discretize the problem, we show that exacts solutions of the infinite dimensional problem can be obtained by solving two consecutive finite dimensional convex programs. These results extend recent advances in the understanding of total-variation reg-ularized problems.

Georg Bökman, Axel Flinth, Fredrik Kahl

Equivariance of linear neural network layers is well studied. In this work, we relax the equivariance condition to only be true in a projective sense. We propose a way to construct a projectively equivariant neural network through building a standard equivariant network where the linear group representations acting on each intermediate feature space are "multiplicatively modified lifts" of projective group representations. By theoretically studying the relation of projectively and linearly equivariant linear layers, we show that our approach is the most general possible when building a network out of linear layers. The theory is showcased in two simple experiments.

Axel Flinth, Sandra Keiper

This work treats the recovery of sparse, binary signals through box-constrained basis pursuit using biased measurement matrices. Using a probabilistic model, we provide conditions under which the recovery of both sparse and saturated binary signals is very likely. In fact, we also show that under the same condition, the solution of the boxed-constrained basis pursuit program can be found using boxed-constrained least-squares.

Axel Flinth, Ali Hashemi

We propose a scheme utilizing ideas from infinite dimensional compressed sensing for thermal source localization. Using the soft recovery framework of one of the authors, we provide rigorous theoretical guarantees for the recovery performance. In particular, we extend the framework in order to also include noisy measurements. Further, we conduct numerical experiments, showing that our proposed method has strong performance, in a wide range of settings. These include scenarios with few sensors, off-grid source positioning and high noise levels, both in one and two dimensions.

Axel Flinth

This paper describes a dual certificate condition on a linear measurement operator $A$ (defined on a Hilbert space $\mathcal{H}$ and having finite-dimensional range) which guarantees that an atomic norm minimization, in a certain sense, will be able to approximately recover a structured signal $v_0 \in \mathcal{H}$ from measurements $Av_0$. Put very streamlined, the condition implies that peaks in a sparse decomposition of $v_0$ are close the the support of the atomic decomposition of the solution $v^*$. The condition applies in a relatively general context - in particular, the space $\mathcal{H}$ can be infinite-dimensional. The abstract framework is applied to several concrete examples, one example being super-resolution. In this process, several novel results which are interesting on its own are obtained.

Miaowen Dong, Axel Flinth, Jan E. Gerken

Symmetries are important for many deep learning tasks, ranging from applications in the sciences to medical imaging. However, there is an ongoing debate about whether to impose symmetry constraints on the neural network architecture (yielding equivariant neural networks) or learn them from augmented training data. Although equivariant networks are well-studied theoretically, much less is known about data augmentation, since analyzing augmentation requires control over the training dynamics. Inspired by recent results that show that augmented infinite deep ensembles are exactly equivariant, we study data augmentation for Bayesian neural networks (BNNs) trained with variational inference. We focus on variational distributions in the exponential family and derive conditions under which exact equivariance is reached. We furthermore obtain bounds on the equivariance error and introduce three novel symmetrization techniques which boost the effect of data augmentation in this setting. We conduct extensive numerical experiments which show that one of our symmetrization methods (orbit expansion) outperforms the baseline in both equivariance and overall performance. Our code is available at github.com/dmw1998/augment-BNNs

Oskar Nordenfors, Axel Flinth

In many machine learning tasks, known symmetries can be used as an inductive bias to improve model performance. In this paper, we consider learning group equivariance through training with data augmentation. We summarize results from a previous paper of our own, and extend the results to show that equivariance of the trained model can be achieved through training on augmented data in tandem with regularization.

Lucas Brynte, Georg Bökman, Axel Flinth et al.

We characterize the class of image plane transformations which realize rigid camera motions and call these transformations `rigidity preserving'. In particular, 2D translations of pinhole images are not rigidity preserving. Hence, when using CNNs for 3D inference tasks, it can be beneficial to modify the inductive bias from equivariance towards translations to equivariance towards rigidity preserving transformations. We investigate how equivariance with respect to rigidity preserving transformations can be approximated in CNNs, and test our ideas on both 6D object pose estimation and visual localization. Experimentally, we improve on several competitive baselines.

Georg Bökman, Fredrik Kahl, Axel Flinth

In this paper, we are concerned with rotation equivariance on 2D point cloud data. We describe a particular set of functions able to approximate any continuous rotation equivariant and permutation invariant function. Based on this result, we propose a novel neural network architecture for processing 2D point clouds and we prove its universality for approximating functions exhibiting these symmetries. We also show how to extend the architecture to accept a set of 2D-2D correspondences as indata, while maintaining similar equivariance properties. Experiments are presented on the estimation of essential matrices in stereo vision.

Axel Flinth

This article provides a new type of analysis of a compressed-sensing based technique for recovering column-sparse matrices, namely minimization of the $\ell_{1,2}$-norm. Rather than providing conditions on the measurement matrix which guarantees the solution of the program to be exactly equal to the ground truth signal (which already has been thoroughly investigated), it presents a condition which guarantees that the solution is approximately equal to the ground truth. Soft recovery statements of this kind are to the best knowledge of the author a novelty in Compressed Sensing. Apart from the theoretical analysis, we present two heuristic proposes how this property of the $\ell_{1,2}$-program can be utilized to design algorithms for recovery of matrices which are sparse and have low rank at the same time.