Kai-Liang Lu

Kai-Liang Lu, Avraham Chapman

To cluster, classify and represent are three fundamental objectives of learning from high-dimensional data with intrinsic structure. To this end, this paper introduces three interpretable approaches, i.e., segmentation (clustering) via the Minimum Lossy Coding Length criterion, classification via the Minimum Incremental Coding Length criterion and representation via the Maximal Coding Rate Reduction criterion. These are derived based on the lossy data coding and compression framework from the principle of rate distortion in information theory. These algorithms are particularly suitable for dealing with finite-sample data (allowed to be sparse or almost degenerate) of mixed Gaussian distributions or subspaces. The theoretical value and attractive features of these methods are summarized by comparison with other learning methods or evaluation criteria. This summary note aims to provide a theoretical guide to researchers (also engineers) interested in understanding 'white-box' machine (deep) learning methods.

Kai-liang Lu

In many scientific and engineering (e.g., physical, biochemical, medical) practices, data generated through expensive experiments or large-scale simulations, are often sparse and noisy. Physics-informed neural network (PINN) incorporates physical information and knowledge into network topology or computational processes as model priors, with the unique advantage of achieving strong generalization with small data. This study aims to investigate the performance characteristics of the soft-constrained PINN method to solving typical linear and nonlinear ordinary differential equations (ODEs) such as primer, Van der Pol and Duffing oscillators, especially the effectiveness, efficiency, and robustness to noise with minimal data. It is verified that the soft-constrained PINN significantly reduces the need for labeled data. With the aid of appropriate collocation points no need to be labeled, it can predict and also extrapolate with minimal data. First-order and second-order ODEs, no matter linear or nonlinear oscillators, require only one and two training data (containing initial values) respectively, just like classical analytic or Runge-Kutta methods, and with equivalent precision and comparable efficiency (fast training in seconds for scalar ODEs). Furthermore, it can conveniently impose a physical law (e.g., conservation of energy) constraint by adding a regularization term to the total loss function, improving the performance to deal with various complexities such as nonlinearity like Duffing. The DeepXDE-based PINN implementation is light code and can be efficiently trained on both GPU and CPU platforms. The mathematical and computational framework of this alternative and feasible PINN method to ODEs, can be easily extended to PDEs, etc., and is becoming a favorable catalyst for the era of Digital Twins.

Kai-Liang Lu

Compared with contact detection techniques, pavement crack identification with visual images via deep learning algorithms has the advantages of not being limited by the material of object to be detected, fast speed and low cost. The fundamental frameworks and typical model architectures of transfer learning (TL), encoder-decoder (ED), generative adversarial networks (GAN), and their common modules were first reviewed, and then the evolution of convolutional neural network (CNN) backbone models and GAN models were summarized. The crack classification, segmentation performance, and effect were tested on the SDNET2018 and CFD public data sets. In the aspect of patch sample classification, the fine-tuned TL models can be equivalent to or even slightly better than the ED models in accuracy, and the predicting time is faster; In the aspect of accurate crack location, both ED and GAN algorithms can achieve pixel-level segmentation and is expected to be detected in real time on low computing power platform. Furthermore, a weakly supervised learning framework of combined TL-SSGAN and its performance enhancement measures are proposed, which can maintain comparable crack identification performance with that of the supervised learning, while greatly reducing the number of labeled samples required.