Wei Yao

Yurun Zhang, Wei Yao, Yutian Lan et al.

The integration of renewable energy via power electronics is transforming power grids into low-inertia systems, heightening the risks of frequency insecurity and widespread outages. Therefore, frequency security assessment (FSA) methods are urgently needed to ensure the reliable system operation. Recently, knowledge-data fusion models attempt to address the limitations of knowledge-driven (accuracy) and data-driven (generalization) FSA methods. However, current methods remain confined to shallow knowledge-data integration due to challenges in representing heterogeneous knowledge and establishing interactive mechanisms. Here, by classifing FSA domain knowledge into physics-guided and physics-constrained categories, we propose a guided learning-constrained network (GL-CN) framework, which deeply integrates domain knowledge across both network architecture and training process. In this framework, a data-driven model with dual input channels combining graph convolutional networks (GCN) and multilayer perceptrons (MLP) is proposed to extract both nodal and system-level power system features. Furthermore, guided learning enhances model generalization through data augmentation in pre-training utilizing physics-guided knowledge, while constrained network encodes physics-constrained knowledge into the network architecture and loss function to ensure physics-consistent and robust predictions. Validated on Yunnan Provincial Power Grid in China, our method reduces FSA time from days to seconds compared to traditional simulation, achieving 98% accuracy, robustness against 39.0% knowledge error, and generalization for 40%-60% renewable penetration. This provides a solid solution for mitigating blackouts caused by frequency insecurity and offers a generalizable paradigm for broader cross-domain problems.

Wei Yao, Zhanke Zhou, Zhicong Li et al.

It has been observed that machine learning algorithms exhibit biased predictions against certain population groups. To mitigate such bias while achieving comparable accuracy, a promising approach is to introduce surrogate functions of the concerned fairness definition and solve a constrained optimization problem. However, it is intriguing in previous work that such fairness surrogate functions may yield unfair results and high instability. In this work, in order to deeply understand them, taking a widely used fairness definition--demographic parity as an example, we show that there is a surrogate-fairness gap between the fairness definition and the fairness surrogate function. Also, the theoretical analysis and experimental results about the gap motivate us that the fairness and stability will be affected by the points far from the decision boundary, which is the large margin points issue investigated in this paper. To address it, we propose the general sigmoid surrogate to simultaneously reduce both the surrogate-fairness gap and the variance, and offer a rigorous fairness and stability upper bound. Interestingly, the theory also provides insights into two important issues that deal with the large margin points as well as obtaining a more balanced dataset are beneficial to fairness and stability. Furthermore, we elaborate a novel and general algorithm called Balanced Surrogate, which iteratively reduces the gap to mitigate unfairness. Finally, we provide empirical evidence showing that our methods consistently improve fairness and stability while maintaining accuracy comparable to the baselines in three real-world datasets.

Wei Yao, Chengming Yu, Shangzhi Zeng et al.

This paper presents a new approach and algorithm for solving a class of constrained Bi-Level Optimization (BLO) problems in which the lower-level problem involves constraints coupling both upper-level and lower-level variables. Such problems have recently gained significant attention due to their broad applicability in machine learning. However, conventional gradient-based methods unavoidably rely on computationally intensive calculations related to the Hessian matrix. To address this challenge, we begin by devising a smooth proximal Lagrangian value function to handle the constrained lower-level problem. Utilizing this construct, we introduce a single-level reformulation for constrained BLOs that transforms the original BLO problem into an equivalent optimization problem with smooth constraints. Enabled by this reformulation, we develop a Hessian-free gradient-based algorithm-termed proximal Lagrangian Value function-based Hessian-free Bi-level Algorithm (LV-HBA)-that is straightforward to implement in a single loop manner. Consequently, LV-HBA is especially well-suited for machine learning applications. Furthermore, we offer non-asymptotic convergence analysis for LV-HBA, eliminating the need for traditional strong convexity assumptions for the lower-level problem while also being capable of accommodating non-singleton scenarios. Empirical results substantiate the algorithm's superior practical performance.

Risheng Liu, Zhu Liu, Wei Yao et al.

This work focuses on addressing two major challenges in the context of large-scale nonconvex Bi-Level Optimization (BLO) problems, which are increasingly applied in machine learning due to their ability to model nested structures. These challenges involve ensuring computational efficiency and providing theoretical guarantees. While recent advances in scalable BLO algorithms have primarily relied on lower-level convexity simplification, our work specifically tackles large-scale BLO problems involving nonconvexity in both the upper and lower levels. We simultaneously address computational and theoretical challenges by introducing an innovative single-loop gradient-based algorithm, utilizing the Moreau envelope-based reformulation, and providing non-asymptotic convergence analysis for general nonconvex BLO problems. Notably, our algorithm relies solely on first-order gradient information, enhancing its practicality and efficiency, especially for large-scale BLO learning tasks. We validate our approach's effectiveness through experiments on various synthetic problems, two typical hyper-parameter learning tasks, and a real-world neural architecture search application, collectively demonstrating its superior performance.