Attila Lovas

Attila Lovas, Miklós Rásonyi

We study the mixing properties of an important optimization algorithm of machine learning: the stochastic gradient Langevin dynamics (SGLD) with a fixed step size. The data stream is not assumed to be independent hence the SGLD is not a Markov chain, merely a \emph{Markov chain in a random environment}, which complicates the mathematical treatment considerably. We derive a strong law of large numbers and a functional central limit theorem for SGLD.

Attila Lovas, Róbert Nagy, Elek Csobo et al.

We present a novel numerical algorithm developed to reconstuct pulsatile blood flow from ECG-gated CT angiography data. A block-based optimization method was constructed to solve the inverse problem corresponding to the Riccati-type ordinary differential equation that can be deduced from conservation principles and Hooke's law. Local flow rate for 5 patients was computed in 10cm long aorta segments that are located 1cm below the heart. The wave form of the local flow rate curves seems to be realistic. Our approach is suitable for estimating characteristics of pulsatile blood flow in aorta based on ECG gated CT scan thereby contributing to more accurate description of several cardiovascular lesions.

Attila Lovas, Iosif Lytras, Miklós Rásonyi et al.

Artificial neural networks (ANNs) are typically highly nonlinear systems which are finely tuned via the optimization of their associated, non-convex loss functions. In many cases, the gradient of any such loss function has superlinear growth, making the use of the widely-accepted (stochastic) gradient descent methods, which are based on Euler numerical schemes, problematic. We offer a new learning algorithm based on an appropriately constructed variant of the popular stochastic gradient Langevin dynamics (SGLD), which is called tamed unadjusted stochastic Langevin algorithm (TUSLA). We also provide a nonasymptotic analysis of the new algorithm's convergence properties in the context of non-convex learning problems with the use of ANNs. Thus, we provide finite-time guarantees for TUSLA to find approximate minimizers of both empirical and population risks. The roots of the TUSLA algorithm are based on the taming technology for diffusion processes with superlinear coefficients as developed in \citet{tamed-euler, SabanisAoAP} and for MCMC algorithms in \citet{tula}. Numerical experiments are presented which confirm the theoretical findings and illustrate the need for the use of the new algorithm in comparison to vanilla SGLD within the framework of ANNs.

Attila Lovas, Miklós Rásonyi

We prove the existence of limiting distributions for a large class of Markov chains on a general state space in a random environment. We assume suitable versions of the standard drift and minorization conditions. In particular, the system dynamics should be contractive on the average with respect to the Lyapunov function and large enough small sets should exist with large enough minorization constants. We also establish that a law of large numbers holds for bounded functionals of the process. Applications to queuing systems, to machine learning algorithms and to autoregressive processes are presented.