Stefan Kindermann

Na Li, Stefan Kindermann, Carmeliza Navasca

We study the convergence of the Regularized Alternating Least-Squares algorithm for tensor decompositions. As a main result, we have shown that given the existence of critical points of the Alternating Least-Squares method, the limit points of the converging subsequences of the RALS are the critical points of the least squares cost functional. Some numerical examples indicate a faster convergence rate for the RALS in comparison to the usual alternating least squares method.

Tapio Helin, Stefan Kindermann, Jonatan Lehtonen et al.

Adaptive optics (AO) is a technology in modern ground-based optical telescopes to compensate the wavefront distortions caused by atmospheric turbulence. One method that allows to retrieve information about the atmosphere from telescope data is so-called SLODAR, where the atmospheric turbulence profile is estimated based on correlation data of Shack--Hartmann wavefront measurements. This approach relies on a layered Kolmogorov turbulence model. In this article, we propose a novel extension of the SLODAR concept by including a general non-Kolmogorov turbulence layer close to the ground with an unknown power spectral density. We prove that the joint estimation problem of the turbulence profile above ground simultaneously with the unknown power spectral density at the ground is ill-posed and propose three numerical reconstruction methods. We demonstrate by numerical simulations that our methods lead to substantial improvements in the turbulence profile reconstruction, compared to standard SLODAR-type approach. Also, our methods can accurately locate local perturbations in non-Kolmogorov power spectral densities.

Bernd Hofmann, Stefan Kindermann, Peter Mathé

The authors study Tikhonov regularization of linear ill-posed problems with a general convex penalty defined on a Banach space. It is well known that the error analysis requires smoothness assumptions. Here such assumptions are given in form of inequalities involving only the family of noise-free minimizers along the regularization parameter and the (unknown) penalty-minimizing solution. These inequalities control, respectively, the defect of the penalty, or likewise, the defect of the whole Tikhonov functional. The main results provide error bounds for a Bregman distance, which split into two summands: the first smoothness-dependent term does not depend on the noise level, whereas the second term includes the noise level. This resembles the situation of standard quadratic Tikhonov regularization Hilbert spaces. It is shown that variational inequalities, as these were studied recently, imply the validity of the assumptions made here. Several examples highlight the results in specific applications.

Stefan Kindermann, Sergiy Pereverzyev, Andrey Pilipenko

The linear functional strategy for the regularization of inverse problems is considered. For selecting the regularization parameter therein, we propose the heuristic quasi-optimality principle and some modifications including the smoothness of the linear functionals. We prove convergence rates for the linear functional strategy with these heuristic rules taking into account the smoothness of the solution and the functionals and imposing a structural condition on the noise. Furthermore, we study these noise conditions in both a deterministic and stochastic setup and verify that for mildly-ill-posed problems and Gaussian noise, these conditions are satisfied almost surely, where on the contrary, in the severely-ill-posed case and in a similar setup, the corresponding noise condition fails to hold. Moreover, we propose an aggregation method for adaptively optimizing the parameter choice rule by making use of improved rates for linear functionals. Numerical results indicate that this method yields better results than the standard heuristic rule.

Stefan Kindermann

We study the convergence of the gradient descent method for solving ill-posed problems where the solution is characterized as a global minimum of a differentiable functional in a Hilbert space. The classical least-squares functional for nonlinear operator equations is a special instance of this framework and the gradient method then reduces to Landweber iteration. The main result of this article is a proof of weak and strong convergence under new nonlinearity conditions that generalize the classical tangential cone conditions.

Werner Zellinger, Stefan Kindermann, Sergei V. Pereverzyev

Estimating the ratio of two probability densities from finitely many observations of the densities is a central problem in machine learning and statistics with applications in two-sample testing, divergence estimation, generative modeling, covariate shift adaptation, conditional density estimation, and novelty detection. In this work, we analyze a large class of density ratio estimation methods that minimize a regularized Bregman divergence between the true density ratio and a model in a reproducing kernel Hilbert space (RKHS). We derive new finite-sample error bounds, and we propose a Lepskii type parameter choice principle that minimizes the bounds without knowledge of the regularity of the density ratio. In the special case of quadratic loss, our method adaptively achieves a minimax optimal error rate. A numerical illustration is provided.

Tapio Helin, Stefan Kindermann, Daniela Saxenhuber

Modern ground-based telescopes rely on a technology called adaptive optics (AO) in order to compensate for the loss of image quality caused by atmospheric turbulence. Next-generation AO systems designed for a wide field of view require a stable and high-resolution reconstruction of the refractive index fluctuations in the atmosphere. By introducing a novel Bayesian method, we address the problem of estimating an atmospheric turbulence strength profile and reconstructing the refractive index fluctuations simultaneously, where we only use wavefront measurements of incoming light from guide stars. Most importantly, we demonstrate how this method can be used for model optimization as well. We propose two different algorithms for solving the maximum a posteriori estimate: the first approach is based on alternating minimization and has the advantage of integrability into existing atmospheric tomography methods. In the second approach, we formulate a convex non-differentiable optimization problem, which is solved by an iterative thresholding method. This approach clearly illustrates the underlying sparsity-enforcing mechanism for the strength profile. By introducing a tuning/regularization parameter, an automated model reduction of the layer structure of the atmosphere is achieved. Using numerical simulations, we demonstrate the performance of our method in practice.

Uno Hämarik, Urve Kangro, Stefan Kindermann et al.

We study the choice of the regularisation parameter for linear ill-posed problems in the presence of data noise and operator perturbations, for which a bound on the operator error is known but the data noise-level is unknown. We introduce a new family of semi-heuristic parameter choice rules that can be used in the stated scenario. We prove convergence of the new rules and provide numerical experiments that indicate an improvement compared to standard heuristic rules.

Stefan Kindermann, Stepan Papacek

The design of an experiment, e.g., the setting of initial conditions, strongly influences the accuracy of the whole process of determining model parameters from data. We impose a sensitivity-based approach for choosing optimal design variables and study the optimization of the shape (and topology) of the initial conditions for an inverse problem of a diffusion parameter identification. Our approach, although case independent, is illustrated at the FRAP (Fluorescence Recovery After Photobleaching) experimental technique. The core idea resides in the maximization of a sensitivity measure, which depends on a specific experimental setting of initial conditions. By a numerical optimization, we find an interesting pattern of increasingly complicated (with respect to connectivity) optimal initial shapes. The proposed modification of the FRAP experimental protocol is rather surprising but entirely realistic and the resulting enhancement of the parameter estimate accuracy is significant.

Stefan Kindermann, Carmeliza Navasca

We study the least-squares (LS) functional of the canonical polyadic (CP) tensor decomposition. Our approach is based on the elimination of one factor matrix which results in a reduced functional. The reduced functional is reformulated into a projection framework and into a Rayleigh quotient. An analysis of this functional leads to several conclusions: new sufficient conditions for the existence of minimizers of the LS functional, the existence of a critical point in the rank-one case, a heuristic explanation of "swamping" and computable bounds on the minimal value of the LS functional. The latter result leads to a simple algorithm -- the Centroid Projection algorithm -- to compute suboptimal solutions of tensor decompositions. These suboptimal solutions are applied to iterative CP algorithms as initial guesses, yielding a method called centroid projection for canonical polyadic (CPCP) decomposition which provides a significant speedup in our numerical experiments compared to the standard methods.

Helmut Gfrerer, Simon Hubmer, Stefan Kindermann et al.

In this paper, we consider the efficient numerical minimization of Tikhonov functionals resulting from total-variation (TV) regularization of linear inverse problems. Since the TV penalty is non-smooth, this is typically done either via smooth approximations, which are inexact, or using non-smooth optimization techniques, which can often be numerically expensive, in particular for large-scale problems. Here, we present a numerically efficient minimization approach based on the recently proposed semismooth* Newton method, which employs a novel concept of graphical derivatives and exhibits locally superlinear convergence. The proposed approach is specifically tailored to TV regularization, suitable for large-scale inverse problems, and supported by strong mathematical convergence guarantees. Furthermore, we demonstrate its performance on two (large-scale) tomographic imaging problems and compare our results to those obtained via other state-of-the-art TV regularization approaches.

Simon Hubmer, Stefan Kindermann, Ekaterina Sherina

We consider the identification of a scalar coefficient in a PDE-based parameter estimation problem with contact constraints. The considered problem can be used as an idealized model of a membrane under forces, constrained by a barrier or indenter. More generally, it serves as a benchmark for the analysis of more complex contact problems and the development of corresponding reconstruction algorithms. In this paper, we discuss both the forward and inverse parameter estimation problems, as well as uniqueness and non-uniqueness issues caused by the contact constraints. Furthermore, we consider the design and implementation of reconstruction approaches which we test on numerical examples, illustrating both uniqueness and non-uniqueness as well as parameter identifyability.

Stefan Kindermann, Kemal Raik

We investigate the convergence theory of several known as well as new heuristic parameter choice rules for convex Tikhonov regularisation. The success of such methods is dependent on whether certain restrictions on the noise are satisfied. In the linear theory, such conditions are well understood and hold for typically irregular noise. In this paper, we extend the convergence analysis of heuristic rules using noise restrictions to the convex setting and prove convergence of the aforementioned methods therewith. The convergence theory is exemplified for the case of an ill-posed problem with a diagonal forward operator in $\ell^q$ spaces. Numerical examples also provide further insight.

Stefan Kindermann, Kemal Raik

We study the choice of the regularisation parameter for linear ill-posed problems in the presence of noise that is possibly unbounded but only finite in a weaker norm, and when the noise-level is unknown. For this task, we analyse several heuristic parameter choice rules, such as the quasi-optimality, heuristic discrepancy, and Hanke-Raus rules and adapt the latter two to the weakly bounded noise case. We prove convergence and convergence rates under certain noise conditions. Moreover, we analyse and provide conditions for the convergence of the parameter choice by the generalised cross-validation and predictive mean-square error rules.

Xiaofei Wang, Carmeliza Navasca, Stefan Kindermann

In this paper, we discuss the acceleration of the regularized alternating least square (RALS) algorithm for tensor approximation. We propose a fast iterative method using a Aitken-Stefensen like updates for the regularized algorithm. Through numerical experiments, the fast algorithm demonstrate a faster convergence rate for the accelerated version in comparison to both the standard and regularized alternating least squares algorithms. In addition, we analyze the global convergence based on the Kurdyka- Lojasiewicz inequality as well as show that the RALS algorithm has a linear local convergence rate.

Stefan Kindermann

We consider the linearized electrical impedance tomography problem in two dimensions on the unit disk. By a linearization around constant coefficients and using a trigonometric basis, we calculate the linearized Dirichlet-to-Neumann operator in terms of moments of the conduction coefficient of the problem. By expanding this coefficient into angular trigonometric functions and Legendre-Müntz polynomials in radial coordinates, we can find a lower-triangular representation of the parameter to data mapping. As a consequence, we find an explicit solution formula for the corresponding inverse problem. Furthermore, we also consider the problem with boundary data given only on parts of the boundary while setting homogeneous Dirichlet values on the rest. We show that the conduction coefficient is uniquely determined from incomplete data of the linearized Dirichlet-to-Neumann operator with an explicit solution formula provided.