Ratna Khatri

Harbir Antil, Ratna Khatri, Mahamadi Warma

Very recently M. Warma has shown that for nonlocal PDEs associated with the fractional Laplacian, the classical notion of controllability from the boundary does not make sense and therefore it must be replaced by a control that is localized outside the open set where the PDE is solved. Having learned from the above mentioned result, in this paper we introduce a new class of source identification and optimal control problems where the source/control is located outside the observation domain where the PDE is satisfied. The classical diffusion models lack this flexibility as they assume that the source/control is located either inside or on the boundary. This is essentially due to the locality property of the underlying operators. We use the nonlocality of the fractional operator to create a framework that now allows placing a source/control outside the observation domain. We consider the Dirichlet, Robin and Neumann source identification or optimal control problems. These problems require dealing with the nonlocal normal derivative (that we shall call interaction operator). We create a functional analytic framework and show well-posedness and derive the first order optimality conditions for these problems. We introduce a new approach to approximate, with convergence rate, the Dirichlet problem with nonzero exterior condition. The numerical examples confirm our theoretical findings and illustrate the practicality of our approach.

Ratna Khatri, Anthony Kolshorn, Colin Olson et al.

This paper presents a novel, mathematically rigorous framework for autoencoder-type deep neural networks that combines optimal control theory and low-rank tensor methods to yield memory-efficient training and automated architecture discovery. The learning task is formulated as an optimization problem constrained by differential equations representing the encoder and decoder components of the network and the corresponding optimality conditions are derived via a Lagrangian approach. Efficient memory compression is enabled by approximating differential equation solutions on low-rank tensor manifolds using an adaptive explicit integration scheme. These concepts are combined to form OCTANE (Optimal Control for Tensor-based Autoencoder Network Emergence) -- a unified training framework that yields compact autoencoder architectures, reduces memory usage, and enables effective learning, even with limited training data. The framework's utility is illustrated with application to image denoising and deblurring tasks and recommendations regarding governing hyperparameters are provided.

Harbir Antil, Ratna Khatri, Rainald Löhner et al.

This paper introduces a novel algorithmic framework for a deep neural network (DNN), which in a mathematically rigorous manner, allows us to incorporate history (or memory) into the network -- it ensures all layers are connected to one another. This DNN, called Fractional-DNN, can be viewed as a time-discretization of a fractional in time nonlinear ordinary differential equation (ODE). The learning problem then is a minimization problem subject to that fractional ODE as constraints. We emphasize that an analogy between the existing DNN and ODEs, with standard time derivative, is well-known by now. The focus of our work is the Fractional-DNN. Using the Lagrangian approach, we provide a derivation of the backward propagation and the design equations. We test our network on several datasets for classification problems. Fractional-DNN offers various advantages over the existing DNN. The key benefits are a significant improvement to the vanishing gradient issue due to the memory effect, and better handling of nonsmooth data due to the network's ability to approximate non-smooth functions.

Harbir Antil, Zichao Di, Ratna Khatri

In this work we consider a generalized bilevel optimization framework for solving inverse problems. We introduce fractional Laplacian as a regularizer to improve the reconstruction quality, and compare it with the total variation regularization. We emphasize that the key advantage of using fractional Laplacian as a regularizer is that it leads to a linear operator, as opposed to the total variation regularization which results in a nonlinear degenerate operator. Inspired by residual neural networks, to learn the optimal strength of regularization and the exponent of fractional Laplacian, we develop a dedicated bilevel optimization neural network with a variable depth for a general regularized inverse problem. We also draw some parallels between an activation function in a neural network and regularization. We illustrate how to incorporate various regularizer choices into our proposed network. As an example, we consider tomographic reconstruction as a model problem and show an improvement in reconstruction quality, especially for limited data, via fractional Laplacian regularization. We successfully learn the regularization strength and the fractional exponent via our proposed bilevel optimization neural network. We observe that the fractional Laplacian regularization outperforms total variation regularization. This is specially encouraging, and important, in the case of limited and noisy data.