Markus Grasmair

Markus Grasmair, Otmar Scherzer, Anne Vanhems

This paper considers the nonparametric regression model with an additive error that is dependent on the explanatory variables. As is common in empirical studies in epidemiology and economics, it also supposes that valid instrumental variables are observed. A classical example in microeconomics considers the consumer demand function as a function of the price of goods and the income, both variables often considered as endogenous. In this framework, the economic theory also imposes shape restrictions on the demand function, like integrability conditions. Motivated by this illustration in microeconomics, we study an estimator of a nonparametric constrained regression function using instrumental variables by means of Tikhonov regularization. We derive rates of convergence for the regularized model both in a deterministic and stochastic setting under the assumption that the true regression function satisfies a projected source condition including, because of the non-convexity of the imposed constraints, an additional smallness condition.

Klaus Frick, Markus Grasmair

We study the application of the Augmented Lagrangian Method to the solution of linear ill-posed problems. Previously, linear convergence rates with respect to the Bregman distance have been derived under the classical assumption of a standard source condition. Using the method of variational inequalities, we extend these results in this paper to convergence rates of lower order, both for the case of an a priori parameter choice and an a posteriori choice based on Morozov's discrepancy principle. In addition, our approach allows the derivation of convergence rates with respect to distance measures different from the Bregman distance. As a particular application, we consider sparsity promoting regularization, where we derive a range of convergence rates with respect to the norm under the assumption of restricted injectivity in conjunction with generalized source conditions of Hölder type.

Martin Bauer, Thomas Fidler, Markus Grasmair

This article is concerned with the representation of curves by means of integral invariants. In contrast to the classical differential invariants they have the advantage of being less sensitive with respect to noise. The integral invariant most common in use is the circular integral invariant. A major drawback of this curve descriptor, however, is the absence of any uniqueness result for this representation. This article serves as a contribution towards closing this gap by showing that the circular integral invariant is injective in a neighbourhood of the circle. In addition, we provide a stability estimate valid on this neighbourhood. The proof is an application of Riesz-Schauder theory and the implicit function theorem in a Banach space setting.

Markus Grasmair, Valeriya Naumova

Inspired by several real-life applications in audio processing and medical image analysis, where the quantity of interest is generated by several sources to be accurately modeled and separated, as well as by recent advances in regularization theory and optimization, we study the conditions on optimal support recovery in inverse problems of unmixing type by means of multi-penalty regularization. We consider and analyze a regularization functional composed of a data-fidelity term, where signal and noise are additively mixed, a non-smooth, convex, sparsity promoting term, and a quadratic penalty term to model the noise. We prove not only that the well-established theory for sparse recovery in the single parameter case can be translated to the multi-penalty settings, but we also demonstrate the enhanced properties of multi-penalty regularization in terms of support identification compared to sole $\ell^1$-minimization. We additionally confirm and support the theoretical results by extensive numerical simulations, which give a statistics of robustness of the multi-penalty regularization scheme with respect to the single-parameter counterpart. Eventually, we confirm a significant improvement in performance compared to standard $\ell^1$-regularization for compressive sensing problems considered in our experiments.

Markus Grasmair

In this paper we derive higher order convergence rates in terms of the Bregman distance for Tikhonov like convex regularisation for linear operator equations on Banach spaces. The approach is based on the idea of variational inequalities, which are, however, not imposed on the original Tikhonov functional, but rather on a dual functional. Because of that, the approach is not limited to convergence rates of lower order, but yields the same range of rates that is well known for quadratic regularisation on Hilbert spaces.

Markus Grasmair

We study the behaviour of Tikhonov regularisation on topological spaces with multiple regularisation terms. The main result of the paper shows that multi-parameter regularisation is well-posed in the sense that the results depend continuously on the data and converge to a true solution of the equation to be solved as the noise level decreases to zero. Moreover, we derive convergence rates in terms of a generalised Bregman distance using the method of variational inequalities. All the results in the paper, including the convergence rates, consider not only noise in the data, but also errors in the operator.

Markus Grasmair

In this paper we study Tikhonov regularization for the stable solution of an ill-posed non-linear operator equation. The operator we consider, which is related to an active contour model for image segmentation, is continuous, compact, but nowhere differentiable. Nevertheless we are able to derive convergence rates under different smoothness assumptions on the true solution by employing the method of variational or source inequalities. With this approach, we can prove up to linear convergence with respect to the norm.

Martin Ludvigsen, Markus Grasmair

The idea of adversarial learning of regularization functionals has recently been introduced in the wider context of inverse problems. The intuition behind this method is the realization that it is not only necessary to learn the basic features that make up a class of signals one wants to represent, but also, or even more so, which features to avoid in the representation. In this paper, we will apply this approach to the problem of source separation by means of non-negative matrix factorization (NMF) and present a new method for the adversarial training of NMF bases. We show in numerical experiments, both for image and audio separation, that this leads to a clear improvement of the reconstructed signals, in particular in the case where little or no strong supervision data is available.

Martin Ludvigsen, Markus Grasmair

The idea of adversarial learning of regularization functionals has recently been introduced in the wider context of inverse problems. The intuition behind this method is the realization that it is not only necessary to learn the basic features that make up a class of signals one wants to represent, but also, or even more so, which features to avoid in the representation. In this paper, we will apply this approach to the training of generative models, leading to what we call Maximum Discrepancy Generative Regularization. In particular, we apply this to problem of source separation by means of Non-negative Matrix Factorization (NMF) and present a new method for the adversarial training of NMF bases. We show in numerical experiments, both for image and audio separation, that this leads to a clear improvement of the reconstructed signals, in particular in the case where little or no strong supervision data is available.

Markus Grasmair, Timo Klock, Valeriya Naumova

For many algorithms, parameter tuning remains a challenging and critical task, which becomes tedious and infeasible in a multi-parameter setting. Multi-penalty regularization, successfully used for solving undetermined sparse regression of problems of unmixing type where signal and noise are additively mixed, is one of such examples. In this paper, we propose a novel algorithmic framework for an adaptive parameter choice in multi-penalty regularization with a focus on the correct support recovery. Building upon the theory of regularization paths and algorithms for single-penalty functionals, we extend these ideas to a multi-penalty framework by providing an efficient procedure for the construction of regions containing structurally similar solutions, i.e., solutions with the same sparsity and sign pattern, over the whole range of parameters. Combining this with a model selection criterion, we can choose regularization parameters in a data-adaptive manner. Another advantage of our algorithm is that it provides an overview on the solution stability over the whole range of parameters. This can be further exploited to obtain additional insights into the problem of interest. We provide a numerical analysis of our method and compare it to the state-of-the-art single-penalty algorithms for compressed sensing problems in order to demonstrate the robustness and power of the proposed algorithm.

Martin Bauer, Markus Eslitzbichler, Markus Grasmair

Motions of virtual characters in movies or video games are typically generated by recording actors using motion capturing methods. Animations generated this way often need postprocessing, such as improving the periodicity of cyclic animations or generating entirely new motions by interpolation of existing ones. Furthermore, search and classification of recorded motions becomes more and more important as the amount of recorded motion data grows. In this paper, we will apply methods from shape analysis to the processing of animations. More precisely, we will use the by now classical elastic metric model used in shape matching, and extend it by incorporating additional inexact feature point information, which leads to an improved temporal alignment of different animations.

Markus Grasmair, Markus Haltmeier, Otmar Scherzer

Although the \emph{residual method}, or \emph{constrained regularization}, is frequently used in applications, a detailed study of its properties is still missing. This sharply contrasts the progress of the theory of Tikhonov regularization, where a series of new results for regularization in Banach spaces has been published in the recent years. The present paper intends to bridge the gap between the existing theories as far as possible. We develop a stability and convergence theory for the residual method in general topological spaces. In addition, we prove convergence rates in terms of (generalized) Bregman distances, which can also be applied to non-convex regularization functionals. We provide three examples that show the applicability of our theory. The first example is the regularized solution of linear operator equations on $L^p$-spaces, where we show that the results of Tikhonov regularization generalize unchanged to the residual method. As a second example, we consider the problem of density estimation from a finite number of sampling points, using the Wasserstein distance as a fidelity term and an entropy measure as regularization term. It is shown that the densities obtained in this way depend continuously on the location of the sampled points and that the underlying density can be recovered as the number of sampling points tends to infinity. Finally, we apply our theory to compressed sensing. Here, we show the well-posedness of the method and derive convergence rates both for convex and non-convex regularization under rather weak conditions.