Wenlian Lu

Bo Liu, Wenlian Lu, Licheng Jiao et al.

In this paper, we propose a new self-triggered consensus algorithm in networks of multi-agents. Different from existing works, which are based on the observation of states, here, each agent determines its next update time based on its coupling structure. Both centralized and distributed approaches of the algorithms have been discussed. By transforming the algorithm to a proper discrete-time systems without self delays, we established a new analysis framework to prove the convergence of the algorithm. Then we extended the algorithm to networks with switching topologies, especially stochastically switching topologies. Compared to existing works, our algorithm is easier to understand and implement. It explicitly provides positive lower and upper bounds for the update time interval of each agent based on its coupling structure, which can also be independently adjusted by each agent according to its own situation. Our work reveals that the event/self triggered algorithms are essentially discrete and more suitable to a discrete analysis framework. Numerical simulations are also provided to illustrate the theoretical results.

Xinjia Li, Boyu Chen, Wenlian Lu

The performance of federated learning in neural networks is generally influenced by the heterogeneity of the data distribution. For a well-performing global model, taking a weighted average of the local models, as done by most existing federated learning algorithms, may not guarantee consistency with local models in the space of neural network maps. In this paper, we propose a novel framework of federated learning equipped with the process of decentralized knowledge distillation (FedDKD) (i.e., without data on the server). The FedDKD introduces a module of decentralized knowledge distillation (DKD) to distill the knowledge of the local models to train the global model by approaching the neural network map average based on the metric of divergence defined in the loss function, other than only averaging parameters as done in literature. Numeric experiments on various heterogeneous datasets reveal that FedDKD outperforms the state-of-the-art methods with more efficient communication and training in a few DKD steps, especially on some extremely heterogeneous datasets.

Wenlian Lu, Fatihcan M. Atay

We study the stability of networks of multi-agent systems with local pinning strategies and two types of time delays, namely the transmission delay in the network and the pinning delay of the controllers. Sufficient conditions for stability are derived under specific scenarios by computing or estimating the dominant eigenvalue of the characteristic equation. In addition, controlling the network by pinning a single node is studied. Moreover, perturbation methods are employed to derive conditions in the limit of small and large pinning strengths.Numerical algorithms are proposed to verify stability, and simulation examples are presented to confirm the efficiency of analytic results.

Yi Yu, Wenlian Lu, Boyu Chen

We propose theoretical analyses of a modified natural gradient descent method in the neural network function space based on the eigendecompositions of neural tangent kernel and Fisher information matrix. We firstly present analytical expression for the function learned by this modified natural gradient under the assumptions of Gaussian distribution and infinite width limit. Thus, we explicitly derive the generalization error of the learned neural network function using theoretical methods from eigendecomposition and statistics theory. By decomposing of the total generalization error attributed to different eigenspace of the kernel in function space, we propose a criterion for balancing the errors stemming from training set and the distribution discrepancy between the training set and the true data. Through this approach, we establish that modifying the training direction of the neural network in function space leads to a reduction in the total generalization error. Furthermore, We demonstrate that this theoretical framework is capable to explain many existing results of generalization enhancing methods. These theoretical results are also illustrated by numerical examples on synthetic data.

Yi Yu, Junzhuo Ma, Chenghuang Shen et al.

Adapting Large Language Models in complex technical service domains is constrained by the absence of explicit cognitive chains in human demonstrations and the inherent ambiguity arising from the diversity of valid responses. These limitations severely hinder agents from internalizing latent decision dynamics and generalizing effectively. Moreover, practical adaptation is often impeded by the prohibitive resource and time costs associated with standard training paradigms. To overcome these challenges and guarantee computational efficiency, we propose a lightweight adaptation framework comprising three key contributions. (1) Latent Logic Augmentation: We introduce Planning-Aware Trajectory Modeling and Decision Reasoning Augmentation to bridge the gap between surface-level supervision and latent decision logic. These approaches strengthen the stability of Supervised Fine-Tuning alignment. (2) Robust Noise Reduction: We construct a Multiple Ground Truths dataset through a dual-filtering method to reduce the noise by validating diverse responses, thereby capturing the semantic diversity. (3) Lightweight Adaptation: We design a Hybrid Reward mechanism that fuses an LLM-based judge with a lightweight relevance-based Reranker to distill high-fidelity reward signals while reducing the computational cost compared to standard LLM-as-a-Judge reinforcement learning. Empirical evaluations on real-world Cloud service tasks, conducted across semantically diverse settings, demonstrate that our framework achieves stability and performance gains through Latent Logic Augmentation and Robust Noise Reduction. Concurrently, our Hybrid Reward mechanism achieves alignment comparable to standard LLM-as-a-judge methods with reduced training time, underscoring the practical value for deploying technical service agents.

Zhichao Zhu, Yang Qi, Hengyuan Ma et al.

The Forward-Forward (FF) learning algorithm provides a bottom-up alternative to backpropagation (BP) for training neural networks, relying on a layer-wise "goodness" function with well-designed negative samples for contrastive learning. Existing goodness functions are typically defined as the sum of squared postsynaptic activations, neglecting correlated variability between neurons. In this work, we propose a novel goodness function termed dimensionality compression that uses the effective dimensionality (ED) of fluctuating neural responses to incorporate second-order statistical structure. Our objective minimizes ED for noisy copies of individual inputs while maximizing it across the sample distribution, promoting structured representations without the need to prepare negative samples.We demonstrate that this formulation achieves competitive performance compared to other non-BP methods. Moreover, we show that noise plays a constructive role that can enhance generalization and improve inference when predictions are derived from the mean of squared output, which is equivalent to making predictions based on an energy term. Our findings contribute to the development of more biologically plausible learning algorithms and suggest a natural fit for neuromorphic computing, where stochasticity is a computational resource rather than a nuisance. The code is available at https://github.com/ZhichaoZhu/StochasticForwardForward

Yi Yu, Guangquan Hu, Chenghuang Shen et al.

The increasing agentic capabilities of Large Language Models (LLMs) have enabled their deployment in real-world applications, such as cloud services, where customer-assistant interactions exhibit high technical complexity and long-horizon dependencies, making robustness and resolution efficiency critical for customer satisfaction. However, existing benchmarks for LLM-based agents largely rely on synthetic environments that fail to capture the diversity and unpredictability of authentic customer inputs, often ignoring the resolution efficiency essential for real-world deployment. To bridge this gap, we introduce CirrusBench, a novel evaluation framework distinguished by its foundation in real-world data from authentic cloud service tickets. CirrusBench preserves the intricate multi-turn logical chains and realistic tool dependencies inherent to technical service environments. Moving beyond execution correctness, we introduce novel Customer-Centric metrics to define agent success, quantifying service quality through metrics such as the Normalized Efficiency Index and Multi-Turn Latency to explicitly measure resolution efficiency. Experiments utilizing our framework reveal that while state-of-the-art models demonstrate strong reasoning capabilities, they frequently struggle in complex, realistic multi-turn tasks and fail to meet the high-efficiency standards required for customer service, highlighting critical directions for the future development of LLM-based agents in practical technical service applications. CirrusBench evaluation framework is released at: https://github.com/CirrusAI

Zhikun Xu, Yinghui Li, Ruixue Ding et al.

How to better evaluate the capabilities of Large Language Models (LLMs) is the focal point and hot topic in current LLMs research. Previous work has noted that due to the extremely high cost of iterative updates of LLMs, they are often unable to answer the latest dynamic questions well. To promote the improvement of Chinese LLMs' ability to answer dynamic questions, in this paper, we introduce CDQA, a Chinese Dynamic QA benchmark containing question-answer pairs related to the latest news on the Chinese Internet. We obtain high-quality data through a pipeline that combines humans and models, and carefully classify the samples according to the frequency of answer changes to facilitate a more fine-grained observation of LLMs' capabilities. We have also evaluated and analyzed mainstream and advanced Chinese LLMs on CDQA. Extensive experiments and valuable insights suggest that our proposed CDQA is challenging and worthy of more further study. We believe that the benchmark we provide will become one of the key data resources for improving LLMs' Chinese question-answering ability in the future.

Yinghui Li, Jiayi Kuang, Haojing Huang et al.

Leveraging mathematical Large Language Models (LLMs) for proof generation is a fundamental topic in LLMs research. We argue that the ability of current LLMs to prove statements largely depends on whether they have encountered the relevant proof process during training. This reliance limits their deeper understanding of mathematical theorems and related concepts. Inspired by the pedagogical method of "proof by counterexamples" commonly used in human mathematics education, our work aims to enhance LLMs' ability to conduct mathematical reasoning and proof through counterexamples. Specifically, we manually create a high-quality, university-level mathematical benchmark, CounterMATH, which requires LLMs to prove mathematical statements by providing counterexamples, thereby assessing their grasp of mathematical concepts. Additionally, we develop a data engineering framework to automatically obtain training data for further model improvement. Extensive experiments and detailed analyses demonstrate that CounterMATH is challenging, indicating that LLMs, such as OpenAI o1, have insufficient counterexample-driven proof capabilities. Moreover, our exploration into model training reveals that strengthening LLMs' counterexample-driven conceptual reasoning abilities is crucial for improving their overall mathematical capabilities. We believe that our work offers new perspectives on the community of mathematical LLMs.

Hengyuan Ma, Wenlian Lu, Jianfeng Feng

Combinatorial optimization problems are widespread but inherently challenging due to their discrete nature. The primary limitation of existing methods is that they can only access a small fraction of the solution space at each iteration, resulting in limited efficiency for searching the global optimal. To overcome this challenge, diverging from conventional efforts of expanding the solver's search scope, we focus on enabling information to actively propagate to the solver through heat diffusion. By transforming the target function while preserving its optima, heat diffusion facilitates information flow from distant regions to the solver, providing more efficient navigation. Utilizing heat diffusion, we propose a framework for solving general combinatorial optimization problems. The proposed methodology demonstrates superior performance across a range of the most challenging and widely encountered combinatorial optimizations. Echoing recent advancements in harnessing thermodynamics for generative artificial intelligence, our study further reveals its significant potential in advancing combinatorial optimization.

Hengyuan Ma, Yang Qi, Li Zhang et al.

Building robust, interpretable, and secure AI system requires quantifying and representing uncertainty under a probabilistic perspective to mimic human cognitive abilities. However, probabilistic computation presents significant challenges for most conventional artificial neural network, as they are essentially implemented in a deterministic manner. In this paper, we develop an efficient probabilistic computation framework by truncating the probabilistic representation of neural activation up to its mean and covariance and construct a moment neural network that encapsulates the nonlinear coupling between the mean and covariance of the underlying stochastic network. We reveal that when only the mean but not the covariance is supervised during gradient-based learning, the unsupervised covariance spontaneously emerges from its nonlinear coupling with the mean and faithfully captures the uncertainty associated with model predictions. Our findings highlight the inherent simplicity of probabilistic computation by seamlessly incorporating uncertainty into model prediction, paving the way for integrating it into large-scale AI systems.

Junya Chen, Zhe Gan, Xuan Li et al.

InfoNCE-based contrastive representation learners, such as SimCLR, have been tremendously successful in recent years. However, these contrastive schemes are notoriously resource demanding, as their effectiveness breaks down with small-batch training (i.e., the log-K curse, whereas K is the batch-size). In this work, we reveal mathematically why contrastive learners fail in the small-batch-size regime, and present a novel simple, non-trivial contrastive objective named FlatNCE, which fixes this issue. Unlike InfoNCE, our FlatNCE no longer explicitly appeals to a discriminative classification goal for contrastive learning. Theoretically, we show FlatNCE is the mathematical dual formulation of InfoNCE, thus bridging the classical literature on energy modeling; and empirically, we demonstrate that, with minimal modification of code, FlatNCE enables immediate performance boost independent of the subject-matter engineering efforts. The significance of this work is furthered by the powerful generalization of contrastive learning techniques, and the introduction of new tools to monitor and diagnose contrastive training. We substantiate our claims with empirical evidence on CIFAR10, ImageNet, and other datasets, where FlatNCE consistently outperforms InfoNCE.

Zhaofeng Liu, Wenlian Lu, Yingying Lang

This paper proposes a new event-based parameter switching method for the control tasks of cybersecurity in the context of preventive and reactive cyber defense dynamics. Our parameter switching method helps avoid excessive control costs as well as guarantees the dynamics to converge as our desired speed. Meanwhile, it can be proved that this approach is Zeno-free. A new estimation method with adaptive time windows is used to bridge the gap between the probability state and the sampling state. With the new estimation method, several practical experiments are given afterwards.

Yujuan Han, Wenlian Lu, Shouhuai Xu

Cybersecurity dynamics is a mathematical approach to modeling and analyzing cyber attack-defense interactions in networks. In this paper, we advance the state-of-the-art in characterizing one kind of cybersecurity dynamics, known as preventive and reactive cyber defense dynamics, which is a family of highly nonlinear system models. We prove that this dynamics in its general form with time-dependent parameters is globally attractive when the time-dependent parameters are ergodic, and is (almost) periodic when the time-dependent parameters have the stronger properties of being (almost) periodic. Our results supersede the state-of-the-art ones, including that the same type of dynamics but with time-independent parameters is globally convergent.

Yaoting Huang, Boyu Chen, Wenlian Lu et al.

Due to the complication of the presented problem, intelligent algorithms show great power to solve the parts logistics optimization problem related to the vehicle routing problem (VRP). However, most of the existing research to VRP are incomprehensive and failed to solve a real-work parts logistics problem. In this work, towards SAIC logistics problem, we propose a systematic solution to this 2-Dimensional Loading Capacitated Multi-Depot Heterogeneous VRP with Time Windows by integrating diverse types of intelligent algorithms, including, a heuristic algorithm to initialize feasible logistics planning schemes by imitating manual planning, the core Tabu Search algorithm for global optimization, accelerated by a novel bundle technique, heuristically algorithms for routing, packing and queuing associated, and a heuristic post-optimization process to promote the optimal solution. Based on these algorithms, the SAIC Motor has successfully established an intelligent management system to give a systematic solution for the parts logistics planning, superior than manual planning in its performance, customizability and expandability.

Boyu Chen, Wenlian Lu, Ernest Fokoue

Meta-learning is a promising method to achieve efficient training method towards deep neural net and has been attracting increases interests in recent years. But most of the current methods are still not capable to train complex neuron net model with long-time training process. In this paper, a novel second-order meta-optimizer, named Meta-learning with Hessian-Free(MLHF) approach, is proposed based on the Hessian-Free approach. Two recurrent neural networks are established to generate the damping and the precondition matrix of this Hessian-Free framework. A series of techniques to meta-train the MLHF towards stable and reinforce the meta-training of this optimizer, including the gradient calculation of $H$. Numerical experiments on deep convolution neural nets, including CUDA-convnet and ResNet18(v2), with datasets of CIFAR10 and ILSVRC2012, indicate that the MLHF shows good and continuous training performance during the whole long-time training process, i.e., both the rapid-decreasing early stage and the steadily-deceasing later stage, and so is a promising meta-learning framework towards elevating the training efficiency in real-world deep neural nets.

Changmao Cheng, Yanwei Fu, Yu-Gang Jiang et al.

Inspired by the recent neuroscience studies on the left-right asymmetry of the human brain in processing low and high spatial frequency information, this paper introduces a dual skipping network which carries out coarse-to-fine object categorization. Such a network has two branches to simultaneously deal with both coarse and fine-grained classification tasks. Specifically, we propose a layer-skipping mechanism that learns a gating network to predict which layers to skip in the testing stage. This layer-skipping mechanism endows the network with good flexibility and capability in practice. Evaluations are conducted on several widely used coarse-to-fine object categorization benchmarks, and promising results are achieved by our proposed network model.

Wenlian Lu, Ren Zheng, Tianping Chen

In this paper, we discuss the outer-synchronization of the asymmetrically connected recurrent time-varying neural networks. By both centralized and decentralized discretization data sampling principles, we derive several sufficient conditions based on diverse vector norms that guarantee that any two trajectories from different initial values of the identical neural network system converge together. The lower bounds of the common time intervals between data samples in centralized and decentralized principles are proved to be positive, which guarantees exclusion of Zeno behavior. A numerical example is provided to illustrate the efficiency of the theoretical results.

Ren Zheng, Xinlei Yi, Wenlian Lu et al.

In this paper, we investigate stability of a class of analytic neural networks with the synaptic feedback via event-triggered rules. This model is general and include Hopfield neural network as a special case. These event-trigger rules can efficiently reduces loads of computation and information transmission at synapses of the neurons. The synaptic feedback of each neuron keeps a constant value based on the outputs of the other neurons at its latest triggering time but changes at its next triggering time, which is determined by certain criterion. It is proved that every trajectory of the analytic neural network converges to certain equilibrium under this event-triggered rule for all initial values except a set of zero measure. The main technique of the proof is the Lojasiewicz inequality to prove the finiteness of trajectory length. The realization of this event-triggered rule is verified by the exclusion of Zeno behaviors. Numerical examples are provided to illustrate the efficiency of the theoretical results.

Ren Zheng, Wenlian Lu, Shouhuai Xu

The Internet is a man-made complex system under constant attacks (e.g., Advanced Persistent Threats and malwares). It is therefore important to understand the phenomena that can be induced by the interaction between cyber attacks and cyber defenses. In this paper, we explore the rich phenomena that can be exhibited when the defender employs active defense to combat cyber attacks. To the best of our knowledge, this is the first study that shows that {\em active cyber defense dynamics} (or more generally, {\em cybersecurity dynamics}) can exhibit the bifurcation and chaos phenomena. This has profound implications for cyber security measurement and prediction: (i) it is infeasible (or even impossible) to accurately measure and predict cyber security under certain circumstances; (ii) the defender must manipulate the dynamics to avoid such {\em unmanageable situations} in real-life defense operations.

Wenlian Lu, Shouhuai Xu, Xinlei Yi

Active cyber defense is one important defensive method for combating cyber attacks. Unlike traditional defensive methods such as firewall-based filtering and anti-malware tools, active cyber defense is based on spreading "white" or "benign" worms to combat against the attackers' malwares (i.e., malicious worms) that also spread over the network. In this paper, we initiate the study of {\em optimal} active cyber defense in the setting of strategic attackers and/or strategic defenders. Specifically, we investigate infinite-time horizon optimal control and fast optimal control for strategic defenders (who want to minimize their cost) against non-strategic attackers (who do not consider the issue of cost). We also investigate the Nash equilibria for strategic defenders and attackers. We discuss the cyber security meanings/implications of the theoretic results. Our study brings interesting open problems for future research.

Shouhuai Xu, Wenlian Lu, Hualun Li

The concept of active cyber defense has been proposed for years. However, there are no mathematical models for characterizing the effectiveness of active cyber defense. In this paper, we fill the void by proposing a novel Markov process model that is native to the interaction between cyber attack and active cyber defense. Unfortunately, the native Markov process model cannot be tackled by the techniques we are aware of. We therefore simplify, via mean-field approximation, the Markov process model as a Dynamic System model that is amenable to analysis. This allows us to derive a set of valuable analytical results that characterize the effectiveness of four types of active cyber defense dynamics. Simulations show that the analytical results are inherent to the native Markov process model, and therefore justify the validity of the Dynamic System model. We also discuss the side-effect of the mean-field approximation and its implications.

Shouhuai Xu, Wenlian Lu, Li Xu

Understanding the dynamics of computer virus (malware, worm) in cyberspace is an important problem that has attracted a fair amount of attention. Early investigations for this purpose adapted biological epidemic models, and thus inherited the so-called homogeneity assumption that each node is equally connected to others. Later studies relaxed this often-unrealistic homogeneity assumption, but still focused on certain power-law networks. Recently, researchers investigated epidemic models in {\em arbitrary} networks (i.e., no restrictions on network topology). However, all these models only capture {\em push-based} infection, namely that an infectious node always actively attempts to infect its neighboring nodes. Very recently, the concept of {\em pull-based} infection was introduced but was not treated rigorously. Along this line of research, the present paper investigates push- and pull-based epidemic spreading dynamics in arbitrary networks, using a Non-linear Dynamical Systems approach. The paper advances the state of the art as follows: (1) It presents a more general and powerful sufficient condition (also known as epidemic threshold in the literature) under which the spreading will become stable. (2) It gives both upper and lower bounds on the global mean infection rate, regardless of the stability of the spreading. (3) It offers insights into, among other things, the estimation of the global mean infection rate through localized monitoring of a small {\em constant} number of nodes, {\em without} knowing the values of the parameters.

Ren Zheng, Wenlian Lu, Shouhuai Xu

The recently proposed {\em cybersecurity dynamics} approach aims to understand cybersecurity from a holistic perspective by modeling the evolution of the global cybersecurity state. These models describe the interactions between the various kinds of cyber defenses and the various kinds of cyber attacks. We study a particular kind of cybersecurity dynamics caused by the interactions between preventive and reactive defenses (e.g., filtering and malware detection) against push- and pull-based cyber attacks (e.g., malware spreading and "drive-by download" attacks). The dynamics was previously shown to be globally stable in a {\em special} regime of the parameter universe, but little is known beyond this special regime. In this paper, we resolve an open problem in this domain by proving that the dynamics is globally stable in the {\em entire} parameter universe (i.e., the dynamics always converges to a unique equilibrium). We discuss the cybersecurity meanings and implications of this theoretic result. We also prove that the dynamics converges {\em exponentially} to the equilibrium except for a special parameter regime, in which case the dynamics converges {\em polynomially}. Since it is often difficult to compute the equilibrium, we propose new bounds of the equilibrium and numerically show that these bounds are tighter than those proposed in the literature.

Xinlei Yi, Wenlian Lu, Tianping Chen

This paper mainly investigates consensus problem with pull-based event-triggered feedback control. For each agent, the diffusion coupling feedbacks are based on the states of its in-neighbors at its latest triggering time and the next triggering time of this agent is determined by its in-neighbors' information as well. The general directed topologies, including irreducible and reducible cases, are investigated. The scenario of distributed continuous monitoring is considered firstly, namely each agent can observe its in-neighbors' continuous states. It is proved that if the network topology has a spanning tree, then the event-triggered coupling strategy can realize consensus for the multi-agent system. Then the results are extended to discontinuous monitoring, i.e., self-triggered control, where each agent computes its next triggering time in advance without having to observe the system's states continuously. The effectiveness of the theoretical results are illustrated by a numerical example finally.

Yujuan Han, Wenlian Lu, Shouhuai Xu

Moving Target Defense (MTD) can enhance the resilience of cyber systems against attacks. Although there have been many MTD techniques, there is no systematic understanding and {\em quantitative} characterization of the power of MTD. In this paper, we propose to use a cyber epidemic dynamics approach to characterize the power of MTD. We define and investigate two complementary measures that are applicable when the defender aims to deploy MTD to achieve a certain security goal. One measure emphasizes the maximum portion of time during which the system can afford to stay in an undesired configuration (or posture), without considering the cost of deploying MTD. The other measure emphasizes the minimum cost of deploying MTD, while accommodating that the system has to stay in an undesired configuration (or posture) for a given portion of time. Our analytic studies lead to algorithms for optimally deploying MTD.

Shouhuai Xu, Wenlian Lu, Li Xu et al.

Theoretical modeling of computer virus/worm epidemic dynamics is an important problem that has attracted many studies. However, most existing models are adapted from biological epidemic ones. Although biological epidemic models can certainly be adapted to capture some computer virus spreading scenarios (especially when the so-called homogeneity assumption holds), the problem of computer virus spreading is not well understood because it has many important perspectives that are not necessarily accommodated in the biological epidemic models. In this paper we initiate the study of such a perspective, namely that of adaptive defense against epidemic spreading in arbitrary networks. More specifically, we investigate a non-homogeneous Susceptible-Infectious-Susceptible (SIS) model where the model parameters may vary with respect to time. In particular, we focus on two scenarios we call semi-adaptive defense and fully-adaptive} defense, which accommodate implicit and explicit dependency relationships between the model parameters, respectively. In the semi-adaptive defense scenario, the model's input parameters are given; the defense is semi-adaptive because the adjustment is implicitly dependent upon the outcome of virus spreading. For this scenario, we present a set of sufficient conditions (some are more general or succinct than others) under which the virus spreading will die out; such sufficient conditions are also known as epidemic thresholds in the literature. In the fully-adaptive defense scenario, some input parameters are not known (i.e., the aforementioned sufficient conditions are not applicable) but the defender can observe the outcome of virus spreading. For this scenario, we present adaptive control strategies under which the virus spreading will die out or will be contained to a desired level.

Wenlian Lu, Fatihcan M. Atay, Jurgen Jost

We analyze stability of consensus algorithms in networks of multi-agents with time-varying topologies and delays. The topology and delays are modeled as induced by an adapted process and are rather general, including i.i.d.\ topology processes, asynchronous consensus algorithms, and Markovian jumping switching. In case the self-links are instantaneous, we prove that the network reaches consensus for all bounded delays if the graph corresponding to the conditional expectation of the coupling matrix sum across a finite time interval has a spanning tree almost surely. Moreover, when self-links are also delayed and when the delays satisfy certain integer patterns, we observe and prove that the algorithm may not reach consensus but instead synchronize at a periodic trajectory, whose period depends on the delay pattern. We also give a brief discussion on the dynamics in the absence of self-links.