Yaohua Liu

Risheng Liu, Yaohua Liu, Wei Yao et al.

Gradient methods have become mainstream techniques for Bi-Level Optimization (BLO) in learning fields. The validity of existing works heavily rely on either a restrictive Lower-Level Strong Convexity (LLSC) condition or on solving a series of approximation subproblems with high accuracy or both. In this work, by averaging the upper and lower level objectives, we propose a single loop Bi-level Averaged Method of Multipliers (sl-BAMM) for BLO that is simple yet efficient for large-scale BLO and gets rid of the limited LLSC restriction. We further provide non-asymptotic convergence analysis of sl-BAMM towards KKT stationary points, and the comparative advantage of our analysis lies in the absence of strong gradient boundedness assumption, which is always required by others. Thus our theory safely captures a wider variety of applications in deep learning, especially where the upper-level objective is quadratic w.r.t. the lower-level variable. Experimental results demonstrate the superiority of our method.

Risheng Liu, Yaohua Liu, Shangzhi Zeng et al.

In recent years, Bi-Level Optimization (BLO) techniques have received extensive attentions from both learning and vision communities. A variety of BLO models in complex and practical tasks are of non-convex follower structure in nature (a.k.a., without Lower-Level Convexity, LLC for short). However, this challenging class of BLOs is lack of developments on both efficient solution strategies and solid theoretical guarantees. In this work, we propose a new algorithmic framework, named Initialization Auxiliary and Pessimistic Trajectory Truncated Gradient Method (IAPTT-GM), to partially address the above issues. In particular, by introducing an auxiliary as initialization to guide the optimization dynamics and designing a pessimistic trajectory truncation operation, we construct a reliable approximate version of the original BLO in the absence of LLC hypothesis. Our theoretical investigations establish the convergence of solutions returned by IAPTT-GM towards those of the original BLO without LLC. As an additional bonus, we also theoretically justify the quality of our IAPTT-GM embedded with Nesterov's accelerated dynamics under LLC. The experimental results confirm both the convergence of our algorithm without LLC, and the theoretical findings under LLC.

Yawei Hui, Yaohua Liu

To address the SMC'17 data challenge -- "Data mining atomically resolved images for material properties", we first used the classic "blob detection" algorithms developed in computer vision to identify all atom centers in each STEM image frame. With the help of nearest neighbor analysis, we then found and labeled every atom center common to all the STEM frames and tracked their movements through the given time interval for both Molybdenum or Selenium atoms.

Yawei Hui, Yaohua Liu, Byung-Hoon Park

To address the SMC'18 data challenge, "Discovering Features in Sr$_{14}$Cu$_{24}$O$_{41}$", we have used the clustering algorithm "DBSCAN" to separate the diffuse scattering features from the Bragg peaks, which takes into account both spatial and photometric information in the dataset during in the clustering process. We find that, in additional to highly localized Bragg peaks, there exists broad diffuse scattering patterns consisting of distinguishable geometries. Besides these two distinctive features, we also identify a third distinguishable feature submerged in the low signal-to-noise region in the reciprocal space, whose origin remains an open question.

Yawei Hui, Yaohua Liu

Recent advancements in neutron and X-ray sources, instrumentation and data collection modes have significantly increased the experimental data size (which could easily contain 10$^{8}$ -- 10$^{10}$ data points), so that conventional volumetric visualization approaches become inefficient for both still imaging and interactive OpenGL rendition in a 3D setting. We introduce a new approach based on the unsupervised machine learning algorithm, Density-Based Spatial Clustering of Applications with Noise (DBSCAN), to efficiently analyze and visualize large volumetric datasets. Here we present two examples of analyzing and visualizing datasets from the diffuse scattering experiment of a single crystal sample and the tomographic reconstruction of a neutron scanning of a turbine blade. We found that by using the intensity as the weighting factor in the clustering process, DBSCAN becomes very effective in de-noising and feature/boundary detection, and thus enables better visualization of the hierarchical internal structures of the neutron scattering data.