Song Fang

Song Fang, Quanyan Zhu

In this short note, we investigate the feedback control of relativistic dynamics propelled by mass ejection, modeling, e.g., the relativistic rocket control or the relativistic (space-travel) flight control. As an extreme case, we also examine the control of relativistic photon rockets which are propelled by ejecting photons.

Song Fang, Quanyan Zhu

In this paper, we examine the fundamental performance limitations in the control of stochastic dynamical systems; more specifically, we derive generic $\mathcal{L}_p$ bounds that hold for any causal (stabilizing) controllers and any stochastic disturbances, by an information-theoretic analysis. We first consider the scenario where the plant (i.e., the dynamical system to be controlled) is linear time-invariant, and it is seen in general that the lower bounds are characterized by the unstable poles (or nonminimum-phase zeros) of the plant as well as the conditional entropy of the disturbance. We then analyze the setting where the plant is assumed to be (strictly) causal, for which case the lower bounds are determined by the conditional entropy of the disturbance. We also discuss the special cases of $p = 2$ and $p = \infty$, which correspond to minimum-variance control and controlling the maximum deviations, respectively. In addition, we investigate the power-spectral characterization of the lower bounds as well as its relation to the Kolmogorov-Szegö formula.

Song Fang, Quanyan Zhu

In this short note, we introduce the spectral-domain $\mathcal{W}_2$ Wasserstein distance for elliptical stochastic processes in terms of their power spectra. We also introduce the spectral-domain Gelbrich bound for processes that are not necessarily elliptical.

Song Fang, Quanyan Zhu

This short note is on a property of the Kullback-Leibler (KL) divergence which indicates that independent Gaussian distributions minimize the KL divergence from given independent Gaussian distributions. The primary purpose of this note is for the referencing of papers that need to make use of this property entirely or partially.

Song Fang, Quanyan Zhu

In this paper, we examine the fundamental performance limits of prediction, with or without side information. More specifically, we derive generic lower bounds on the $\mathcal{L}_p$ norms of the prediction errors that are valid for any prediction algorithms and for any data distributions. Meanwhile, we combine the entropic analysis from information theory and the innovations approach from prediction/estimation theory to characterize the conditions (in terms of, e.g., directed information or mutual information) to achieve the bounds. We also investigate the implications of the results in analyzing the fundamental limits of generalization in fitting (learning) problems from the perspective of prediction with side information, as well as the fundamental limits of recursive algorithms by viewing them as generalized prediction problems.

Song Fang, Quanyan Zhu

In this paper, we relate a feedback channel with any finite-order autoregressive moving-average (ARMA) Gaussian noises to a variant of the Kalman filter. In light of this, we obtain relatively explicit lower bounds on the feedback capacity for such colored Gaussian noises, and the bounds are seen to be consistent with various existing results in the literature. Meanwhile, this variant of the Kalman filter also leads to explicit recursive coding schemes with clear structures to achieve the lower bounds. In general, our results provide an alternative perspective while pointing to potentially tighter bounds for the feedback capacity problem.

Song Fang, Quanyan Zhu

In this paper, we utilize information theory to study the fundamental performance limitations of generic feedback systems, where both the controller and the plant may be any causal functions/mappings while the disturbance can be with any distributions. More specifically, we obtain fundamental $\mathcal{L}_p$ bounds on the control error, which are shown to be completely characterized by the conditional entropy of the disturbance, based upon the entropic laws that are inherent in any feedback systems. We also discuss the generality and implications (in, e.g., fundamental limits of learning-based control) of the obtained bounds.

Song Fang, Quanyan Zhu

Strictly speaking, Newton's second law of motion is only an approximation of the so-called relativistic dynamics, i.e., Einstein's modification of the second law based on his theory of special relativity. Although the approximation is almost exact when the velocity of the dynamical system is far less than the speed of light, the difference will become larger and larger (and will eventually go to infinity) as the velocity approaches the speed of light. Correspondingly, feedback control of such dynamics should also take this modification into consideration (though it will render the system nonlinear), especially when the velocity is relatively large. Towards this end, we start this note by studying the state-space representation of the relativistic dynamics. We then investigate on how to employ the feedback linearization approach for such relativistic dynamics, based upon which an additional linear controller may then be designed. As such, the feedback linearization together with the linear controller compose the overall relativistic feedback control law. We also provide discussions on, e.g., controllability, state feedback and output feedback, as well as PID control, in the relativistic setting.

Song Fang, Quanyan Zhu

In this paper, we obtain fundamental $\mathcal{L}_{p}$ bounds in sequential prediction and recursive algorithms via an entropic analysis. Both classes of problems are examined by investigating the underlying entropic relationships of the data and/or noises involved, and the derived lower bounds may all be quantified in a conditional entropy characterization. We also study the conditions to achieve the generic bounds from an innovations' viewpoint.

Song Fang, Quanyan Zhu

In this paper, we derive generic bounds on the maximum deviations in prediction errors for sequential prediction via an information-theoretic approach. The fundamental bounds are shown to depend only on the conditional entropy of the data point to be predicted given the previous data points. In the asymptotic case, the bounds are achieved if and only if the prediction error is white and uniformly distributed.

Song Fang, Karl Henrik Johansson, Mikael Skoglund et al.

In this paper, we introduce the concept of two-way coding, which originates in communication theory characterizing coding schemes for two-way channels, into control theory, particularly to facilitate the analysis and design of feedback control systems under injection attacks. Moreover, we propose the notion of attack decoupling, and show how the controller and the two-way coding can be co-designed to nullify the transfer function from attack to plant, rendering the attack effect zero both in transient phase and in steady state.

Song Fang, Mikael Skoglund, Karl Henrik Johansson et al.

In this paper, we obtain generic bounds on the variances of estimation and prediction errors in time series analysis via an information-theoretic approach. It is seen in general that the error bounds are determined by the conditional entropy of the data point to be estimated or predicted given the side information or past observations. Additionally, we discover that in order to achieve the prediction error bounds asymptotically, the necessary and sufficient condition is that the "innovation" is asymptotically white Gaussian. When restricted to Gaussian processes and 1-step prediction, our bounds are shown to reduce to the Kolmogorov-Szegö formula and Wiener-Masani formula known from linear prediction theory.

Song Fang, Hideaki Ishii, Jie Chen et al.

In this paper, we discover that the trace of the division of the optimal output estimation error covariance over the noise covariance attained by the Kalman-Bucy filter can be explicitly expressed in terms of the plant dynamics and noise statistics in a frequency-domain integral characterization. Towards this end, we examine the algebraic Riccati equation associated with Kalman-Bucy filtering using analytic function theory and relate it to the Bode integral. Our approach features an alternative, frequency-domain framework for analyzing algebraic Riccati equations and reduces to various existing related results.

Song Fang

This paper aims at obtaining universal laws and absolute lower bounds of feedback systems using information theory. The feedback system setup is that with causal plants and causal controllers. Three laws (entropy rate never decreases, generalized Bode integral, and absolute lower bound in variance minimization) are obtained, which are in entropy domain, frequency domain, and time domain, respectively. Those laws characterize the fundamental limitations of such systems imposed by the feedback mechanism. Two new notions, negentropy rate and Gaussianity-whiteness measure (joint Shannon-Wiener entropy), are proposed to facilitate the analysis. Topics such as whiteness-Gaussianity-variance decomposition, Gaussianing-whitening control (the maximum Gaussianity-whiteness measure principle), whitening control (spectrum/spectral flattening control), generalized Bode plot, and so on are also discussed. The special case of linear time-invariant feedback systems is considered in the end.

Song Fang

This paper aims at obtaining performance limitations of state estimation in terms of variance minimization (minimum variance estimation and filtering) using information theory. Two new notions, negentropy rate and Gaussianity-whiteness measure (joint Shannon-Wiener entropy), are proposed to facilitate the analysis. Topics such as Gaussianing-whitening filter (the maximum Gaussianity-whiteness measure principle) are also discussed.

Song Fang

In this paper, three laws are obtained for multiple-input multiple-output feedback systems, which are in entropy domain, frequency domain, and time domain, respectively. The system setup is that with causal plants and causal controllers. Those laws characterize the performance limitations of such systems imposed by the feedback mechanism. Some new notions are proposed to facilitate the analysis: negentropy rate, extended spectral flatness (extended Wiener entropy), Gaussianity-whiteness measure (joint Shannon-Wiener entropy), etc. Two approaches are adopted: the integrated approach and the divided approach. And 'uncertainty principles' are found in minimum variance control. Besides, performance limitations in minimum variance estimation and filtering are obtained. In the end, the special case of linear time-invariant feedback systems is discussed.