Xuan Wang

Jingqiu Zhou, Wang Xuan, Shaoshuai Mou et al.

This paper proposes distributed algorithms for multi-agent networks to achieve a solution in finite time to a linear equation $Ax=b$ where $A$ has full row rank, and with the minimum $l_1$-norm in the underdetermined case (where $A$ has more columns than rows). The underlying network is assumed to be undirected and fixed, and an analytical proof is provided for the proposed algorithm to drive all agents' individual states to converge to a common value, viz a solution of $Ax=b$, which is the minimum $l_1$-norm solution in the underdetermined case. Numerical simulations are also provided as validation of the proposed algorithms.

Xuan Wang, Shaoshuai Mou, Shreyas Sundaram

Consider a set of vectors in $\mathbb{R}^n$, partitioned into two classes: normal vectors and malicious vectors. The number of malicious vectors is bounded but their identities are unknown. The paper provides a way for achieving a resilient convex combination, which is a convex combination of only normal vectors. Compared with existing approaches based on Tverberg points, the proposed method based on the intersection of convex hulls has lower computational complexity. Simulations suggest that the proposed method can be applied to resilience for consensus-based distributed algorithms against Byzantine attacks.

Xuan Wang, Shaoshuai Mou, Brian. D. O. Anderson

This paper proposes a double-layered framework (or form of network) to integrate two mechanisms, termed consensus and conservation, achieving distributed solution of a linear equation. The multi-agent framework considered in the paper is composed of clusters (which serve as a form of aggregating agent) and each cluster consists of a sub-network of agents. By achieving consensus and conservation through agent-agent communications in the same cluster and cluster-cluster communications, distributed algorithms are devised for agents to cooperatively achieve a solution to the overall linear equation. These algorithms outperform existing consensus-based algorithms, including but not limited to the following aspects: first, each agent does not have to know as much as a complete row or column of the overall equation; second, each agent only needs to control as few as two scalar states when the number of clusters and the number of agents are sufficiently large; third, the dimensions of agents' states in the proposed algorithms do not have to be the same (while in contrast, algorithms based on the idea of standard consensus inherently require all agents' states to be of the same dimension). Both analytical proof and simulation results are provided to validate exponential convergence of the proposed distributed algorithms in solving linear equations.

Shaohuai Shi, Qing Yang, Yang Xiang et al.

To enable the pre-trained models to be fine-tuned with local data on edge devices without sharing data with the cloud, we design an efficient split fine-tuning (SFT) framework for edge and cloud collaborative learning. We propose three novel techniques in this framework. First, we propose a matrix decomposition-based method to compress the intermediate output of a neural network to reduce the communication volume between the edge device and the cloud server. Second, we eliminate particular links in the model without affecting the convergence performance in fine-tuning. Third, we implement our system atop PyTorch to allow users to easily extend their existing training scripts to enjoy the efficient edge and cloud collaborative learning. Experiments results on 9 NLP datasets show that our framework can reduce the communication traffic by 96 times with little impact on the model accuracy.

Xuan Wang, Jingqiu Zhou, Shaoshuai Mou et al.

A distributed discrete-time algorithm is proposed for multi-agent networks to achieve a common least squares solution of a group of linear equations, in which each agent only knows some of the equations and is only able to receive information from its nearby neighbors. For fixed, connected, and undirected networks, the proposed discrete-time algorithm results in each agents solution estimate to converging exponentially fast to the same least squares solution. Moreover, the convergence does not require careful choices of time-varying small step sizes.