Wenlian Lu

Bo Liu, Wenlian Lu, Licheng Jiao et al.

In this paper, we propose a new self-triggered consensus algorithm in networks of multi-agents. Different from existing works, which are based on the observation of states, here, each agent determines its next update time based on its coupling structure. Both centralized and distributed approaches of the algorithms have been discussed. By transforming the algorithm to a proper discrete-time systems without self delays, we established a new analysis framework to prove the convergence of the algorithm. Then we extended the algorithm to networks with switching topologies, especially stochastically switching topologies. Compared to existing works, our algorithm is easier to understand and implement. It explicitly provides positive lower and upper bounds for the update time interval of each agent based on its coupling structure, which can also be independently adjusted by each agent according to its own situation. Our work reveals that the event/self triggered algorithms are essentially discrete and more suitable to a discrete analysis framework. Numerical simulations are also provided to illustrate the theoretical results.

Xinjia Li, Boyu Chen, Wenlian Lu

The performance of federated learning in neural networks is generally influenced by the heterogeneity of the data distribution. For a well-performing global model, taking a weighted average of the local models, as done by most existing federated learning algorithms, may not guarantee consistency with local models in the space of neural network maps. In this paper, we propose a novel framework of federated learning equipped with the process of decentralized knowledge distillation (FedDKD) (i.e., without data on the server). The FedDKD introduces a module of decentralized knowledge distillation (DKD) to distill the knowledge of the local models to train the global model by approaching the neural network map average based on the metric of divergence defined in the loss function, other than only averaging parameters as done in literature. Numeric experiments on various heterogeneous datasets reveal that FedDKD outperforms the state-of-the-art methods with more efficient communication and training in a few DKD steps, especially on some extremely heterogeneous datasets.

Wenlian Lu, Fatihcan M. Atay

We study the stability of networks of multi-agent systems with local pinning strategies and two types of time delays, namely the transmission delay in the network and the pinning delay of the controllers. Sufficient conditions for stability are derived under specific scenarios by computing or estimating the dominant eigenvalue of the characteristic equation. In addition, controlling the network by pinning a single node is studied. Moreover, perturbation methods are employed to derive conditions in the limit of small and large pinning strengths.Numerical algorithms are proposed to verify stability, and simulation examples are presented to confirm the efficiency of analytic results.

Zhikun Xu, Yinghui Li, Ruixue Ding et al.

How to better evaluate the capabilities of Large Language Models (LLMs) is the focal point and hot topic in current LLMs research. Previous work has noted that due to the extremely high cost of iterative updates of LLMs, they are often unable to answer the latest dynamic questions well. To promote the improvement of Chinese LLMs' ability to answer dynamic questions, in this paper, we introduce CDQA, a Chinese Dynamic QA benchmark containing question-answer pairs related to the latest news on the Chinese Internet. We obtain high-quality data through a pipeline that combines humans and models, and carefully classify the samples according to the frequency of answer changes to facilitate a more fine-grained observation of LLMs' capabilities. We have also evaluated and analyzed mainstream and advanced Chinese LLMs on CDQA. Extensive experiments and valuable insights suggest that our proposed CDQA is challenging and worthy of more further study. We believe that the benchmark we provide will become one of the key data resources for improving LLMs' Chinese question-answering ability in the future.

Yinghui Li, Jiayi Kuang, Haojing Huang et al.

Leveraging mathematical Large Language Models (LLMs) for proof generation is a fundamental topic in LLMs research. We argue that the ability of current LLMs to prove statements largely depends on whether they have encountered the relevant proof process during training. This reliance limits their deeper understanding of mathematical theorems and related concepts. Inspired by the pedagogical method of "proof by counterexamples" commonly used in human mathematics education, our work aims to enhance LLMs' ability to conduct mathematical reasoning and proof through counterexamples. Specifically, we manually create a high-quality, university-level mathematical benchmark, CounterMATH, which requires LLMs to prove mathematical statements by providing counterexamples, thereby assessing their grasp of mathematical concepts. Additionally, we develop a data engineering framework to automatically obtain training data for further model improvement. Extensive experiments and detailed analyses demonstrate that CounterMATH is challenging, indicating that LLMs, such as OpenAI o1, have insufficient counterexample-driven proof capabilities. Moreover, our exploration into model training reveals that strengthening LLMs' counterexample-driven conceptual reasoning abilities is crucial for improving their overall mathematical capabilities. We believe that our work offers new perspectives on the community of mathematical LLMs.

Hengyuan Ma, Wenlian Lu, Jianfeng Feng

Combinatorial optimization problems are widespread but inherently challenging due to their discrete nature. The primary limitation of existing methods is that they can only access a small fraction of the solution space at each iteration, resulting in limited efficiency for searching the global optimal. To overcome this challenge, diverging from conventional efforts of expanding the solver's search scope, we focus on enabling information to actively propagate to the solver through heat diffusion. By transforming the target function while preserving its optima, heat diffusion facilitates information flow from distant regions to the solver, providing more efficient navigation. Utilizing heat diffusion, we propose a framework for solving general combinatorial optimization problems. The proposed methodology demonstrates superior performance across a range of the most challenging and widely encountered combinatorial optimizations. Echoing recent advancements in harnessing thermodynamics for generative artificial intelligence, our study further reveals its significant potential in advancing combinatorial optimization.

Hengyuan Ma, Yang Qi, Li Zhang et al.

Building robust, interpretable, and secure AI system requires quantifying and representing uncertainty under a probabilistic perspective to mimic human cognitive abilities. However, probabilistic computation presents significant challenges for most conventional artificial neural network, as they are essentially implemented in a deterministic manner. In this paper, we develop an efficient probabilistic computation framework by truncating the probabilistic representation of neural activation up to its mean and covariance and construct a moment neural network that encapsulates the nonlinear coupling between the mean and covariance of the underlying stochastic network. We reveal that when only the mean but not the covariance is supervised during gradient-based learning, the unsupervised covariance spontaneously emerges from its nonlinear coupling with the mean and faithfully captures the uncertainty associated with model predictions. Our findings highlight the inherent simplicity of probabilistic computation by seamlessly incorporating uncertainty into model prediction, paving the way for integrating it into large-scale AI systems.

Zhaofeng Liu, Wenlian Lu, Yingying Lang

This paper proposes a new event-based parameter switching method for the control tasks of cybersecurity in the context of preventive and reactive cyber defense dynamics. Our parameter switching method helps avoid excessive control costs as well as guarantees the dynamics to converge as our desired speed. Meanwhile, it can be proved that this approach is Zeno-free. A new estimation method with adaptive time windows is used to bridge the gap between the probability state and the sampling state. With the new estimation method, several practical experiments are given afterwards.

Yujuan Han, Wenlian Lu, Shouhuai Xu

Cybersecurity dynamics is a mathematical approach to modeling and analyzing cyber attack-defense interactions in networks. In this paper, we advance the state-of-the-art in characterizing one kind of cybersecurity dynamics, known as preventive and reactive cyber defense dynamics, which is a family of highly nonlinear system models. We prove that this dynamics in its general form with time-dependent parameters is globally attractive when the time-dependent parameters are ergodic, and is (almost) periodic when the time-dependent parameters have the stronger properties of being (almost) periodic. Our results supersede the state-of-the-art ones, including that the same type of dynamics but with time-independent parameters is globally convergent.

Yaoting Huang, Boyu Chen, Wenlian Lu et al.

Due to the complication of the presented problem, intelligent algorithms show great power to solve the parts logistics optimization problem related to the vehicle routing problem (VRP). However, most of the existing research to VRP are incomprehensive and failed to solve a real-work parts logistics problem. In this work, towards SAIC logistics problem, we propose a systematic solution to this 2-Dimensional Loading Capacitated Multi-Depot Heterogeneous VRP with Time Windows by integrating diverse types of intelligent algorithms, including, a heuristic algorithm to initialize feasible logistics planning schemes by imitating manual planning, the core Tabu Search algorithm for global optimization, accelerated by a novel bundle technique, heuristically algorithms for routing, packing and queuing associated, and a heuristic post-optimization process to promote the optimal solution. Based on these algorithms, the SAIC Motor has successfully established an intelligent management system to give a systematic solution for the parts logistics planning, superior than manual planning in its performance, customizability and expandability.

Changmao Cheng, Yanwei Fu, Yu-Gang Jiang et al.

Inspired by the recent neuroscience studies on the left-right asymmetry of the human brain in processing low and high spatial frequency information, this paper introduces a dual skipping network which carries out coarse-to-fine object categorization. Such a network has two branches to simultaneously deal with both coarse and fine-grained classification tasks. Specifically, we propose a layer-skipping mechanism that learns a gating network to predict which layers to skip in the testing stage. This layer-skipping mechanism endows the network with good flexibility and capability in practice. Evaluations are conducted on several widely used coarse-to-fine object categorization benchmarks, and promising results are achieved by our proposed network model.

Wenlian Lu, Ren Zheng, Tianping Chen

In this paper, we discuss the outer-synchronization of the asymmetrically connected recurrent time-varying neural networks. By both centralized and decentralized discretization data sampling principles, we derive several sufficient conditions based on diverse vector norms that guarantee that any two trajectories from different initial values of the identical neural network system converge together. The lower bounds of the common time intervals between data samples in centralized and decentralized principles are proved to be positive, which guarantees exclusion of Zeno behavior. A numerical example is provided to illustrate the efficiency of the theoretical results.

Ren Zheng, Xinlei Yi, Wenlian Lu et al.

In this paper, we investigate stability of a class of analytic neural networks with the synaptic feedback via event-triggered rules. This model is general and include Hopfield neural network as a special case. These event-trigger rules can efficiently reduces loads of computation and information transmission at synapses of the neurons. The synaptic feedback of each neuron keeps a constant value based on the outputs of the other neurons at its latest triggering time but changes at its next triggering time, which is determined by certain criterion. It is proved that every trajectory of the analytic neural network converges to certain equilibrium under this event-triggered rule for all initial values except a set of zero measure. The main technique of the proof is the Lojasiewicz inequality to prove the finiteness of trajectory length. The realization of this event-triggered rule is verified by the exclusion of Zeno behaviors. Numerical examples are provided to illustrate the efficiency of the theoretical results.

Ren Zheng, Wenlian Lu, Shouhuai Xu

The Internet is a man-made complex system under constant attacks (e.g., Advanced Persistent Threats and malwares). It is therefore important to understand the phenomena that can be induced by the interaction between cyber attacks and cyber defenses. In this paper, we explore the rich phenomena that can be exhibited when the defender employs active defense to combat cyber attacks. To the best of our knowledge, this is the first study that shows that {\em active cyber defense dynamics} (or more generally, {\em cybersecurity dynamics}) can exhibit the bifurcation and chaos phenomena. This has profound implications for cyber security measurement and prediction: (i) it is infeasible (or even impossible) to accurately measure and predict cyber security under certain circumstances; (ii) the defender must manipulate the dynamics to avoid such {\em unmanageable situations} in real-life defense operations.

Wenlian Lu, Shouhuai Xu, Xinlei Yi

Active cyber defense is one important defensive method for combating cyber attacks. Unlike traditional defensive methods such as firewall-based filtering and anti-malware tools, active cyber defense is based on spreading "white" or "benign" worms to combat against the attackers' malwares (i.e., malicious worms) that also spread over the network. In this paper, we initiate the study of {\em optimal} active cyber defense in the setting of strategic attackers and/or strategic defenders. Specifically, we investigate infinite-time horizon optimal control and fast optimal control for strategic defenders (who want to minimize their cost) against non-strategic attackers (who do not consider the issue of cost). We also investigate the Nash equilibria for strategic defenders and attackers. We discuss the cyber security meanings/implications of the theoretic results. Our study brings interesting open problems for future research.

Shouhuai Xu, Wenlian Lu, Hualun Li

The concept of active cyber defense has been proposed for years. However, there are no mathematical models for characterizing the effectiveness of active cyber defense. In this paper, we fill the void by proposing a novel Markov process model that is native to the interaction between cyber attack and active cyber defense. Unfortunately, the native Markov process model cannot be tackled by the techniques we are aware of. We therefore simplify, via mean-field approximation, the Markov process model as a Dynamic System model that is amenable to analysis. This allows us to derive a set of valuable analytical results that characterize the effectiveness of four types of active cyber defense dynamics. Simulations show that the analytical results are inherent to the native Markov process model, and therefore justify the validity of the Dynamic System model. We also discuss the side-effect of the mean-field approximation and its implications.

Ren Zheng, Wenlian Lu, Shouhuai Xu

The recently proposed {\em cybersecurity dynamics} approach aims to understand cybersecurity from a holistic perspective by modeling the evolution of the global cybersecurity state. These models describe the interactions between the various kinds of cyber defenses and the various kinds of cyber attacks. We study a particular kind of cybersecurity dynamics caused by the interactions between preventive and reactive defenses (e.g., filtering and malware detection) against push- and pull-based cyber attacks (e.g., malware spreading and "drive-by download" attacks). The dynamics was previously shown to be globally stable in a {\em special} regime of the parameter universe, but little is known beyond this special regime. In this paper, we resolve an open problem in this domain by proving that the dynamics is globally stable in the {\em entire} parameter universe (i.e., the dynamics always converges to a unique equilibrium). We discuss the cybersecurity meanings and implications of this theoretic result. We also prove that the dynamics converges {\em exponentially} to the equilibrium except for a special parameter regime, in which case the dynamics converges {\em polynomially}. Since it is often difficult to compute the equilibrium, we propose new bounds of the equilibrium and numerically show that these bounds are tighter than those proposed in the literature.

Xinlei Yi, Wenlian Lu, Tianping Chen

This paper mainly investigates consensus problem with pull-based event-triggered feedback control. For each agent, the diffusion coupling feedbacks are based on the states of its in-neighbors at its latest triggering time and the next triggering time of this agent is determined by its in-neighbors' information as well. The general directed topologies, including irreducible and reducible cases, are investigated. The scenario of distributed continuous monitoring is considered firstly, namely each agent can observe its in-neighbors' continuous states. It is proved that if the network topology has a spanning tree, then the event-triggered coupling strategy can realize consensus for the multi-agent system. Then the results are extended to discontinuous monitoring, i.e., self-triggered control, where each agent computes its next triggering time in advance without having to observe the system's states continuously. The effectiveness of the theoretical results are illustrated by a numerical example finally.

Yujuan Han, Wenlian Lu, Shouhuai Xu

Moving Target Defense (MTD) can enhance the resilience of cyber systems against attacks. Although there have been many MTD techniques, there is no systematic understanding and {\em quantitative} characterization of the power of MTD. In this paper, we propose to use a cyber epidemic dynamics approach to characterize the power of MTD. We define and investigate two complementary measures that are applicable when the defender aims to deploy MTD to achieve a certain security goal. One measure emphasizes the maximum portion of time during which the system can afford to stay in an undesired configuration (or posture), without considering the cost of deploying MTD. The other measure emphasizes the minimum cost of deploying MTD, while accommodating that the system has to stay in an undesired configuration (or posture) for a given portion of time. Our analytic studies lead to algorithms for optimally deploying MTD.

Shouhuai Xu, Wenlian Lu, Li Xu et al.

Theoretical modeling of computer virus/worm epidemic dynamics is an important problem that has attracted many studies. However, most existing models are adapted from biological epidemic ones. Although biological epidemic models can certainly be adapted to capture some computer virus spreading scenarios (especially when the so-called homogeneity assumption holds), the problem of computer virus spreading is not well understood because it has many important perspectives that are not necessarily accommodated in the biological epidemic models. In this paper we initiate the study of such a perspective, namely that of adaptive defense against epidemic spreading in arbitrary networks. More specifically, we investigate a non-homogeneous Susceptible-Infectious-Susceptible (SIS) model where the model parameters may vary with respect to time. In particular, we focus on two scenarios we call semi-adaptive defense and fully-adaptive} defense, which accommodate implicit and explicit dependency relationships between the model parameters, respectively. In the semi-adaptive defense scenario, the model's input parameters are given; the defense is semi-adaptive because the adjustment is implicitly dependent upon the outcome of virus spreading. For this scenario, we present a set of sufficient conditions (some are more general or succinct than others) under which the virus spreading will die out; such sufficient conditions are also known as epidemic thresholds in the literature. In the fully-adaptive defense scenario, some input parameters are not known (i.e., the aforementioned sufficient conditions are not applicable) but the defender can observe the outcome of virus spreading. For this scenario, we present adaptive control strategies under which the virus spreading will die out or will be contained to a desired level.