Hamza Anwar

Hamza Anwar, Quanyan Zhu

Owing to the recent advances in "Big Data" modeling and prediction tasks, variational Bayesian estimation has gained popularity due to their ability to provide exact solutions to approximate posteriors. One key technique for approximate inference is stochastic variational inference (SVI). SVI poses variational inference as a stochastic optimization problem and solves it iteratively using noisy gradient estimates. It aims to handle massive data for predictive and classification tasks by applying complex Bayesian models that have observed as well as latent variables. This paper aims to decentralize it allowing parallel computation, secure learning and robustness benefits. We use Alternating Direction Method of Multipliers in a top-down setting to develop a distributed SVI algorithm such that independent learners running inference algorithms only require sharing the estimated model parameters instead of their private datasets. Our work extends the distributed SVI-ADMM algorithm that we first propose, to an ADMM-based networked SVI algorithm in which not only are the learners working distributively but they share information according to rules of a graph by which they form a network. This kind of work lies under the umbrella of `deep learning over networks' and we verify our algorithm for a topic-modeling problem for corpus of Wikipedia articles. We illustrate the results on latent Dirichlet allocation (LDA) topic model in large document classification, compare performance with the centralized algorithm, and use numerical experiments to corroborate the analytical results.

Hamza Anwar, Quanyan Zhu

Sensing in complex systems requires large-scale information exchange and on-the-go communications over heterogeneous networks and integrated processing platforms. Many networked cyber-physical systems exhibit hierarchical infrastructures of information flows, which naturally leads to a multi-level tree-like information structure in which each level corresponds to a particular scale of representation. This work focuses on the multiscale fusion of data collected at multiple levels of the system. We propose a multiscale state-space model to represent multi-resolution data over the hierarchical information system and formulate a multi-stage dynamic zero-sum game to design a multi-scale $H_{\infty}$ robust filter. We present numerical experiments for one and two-dimensional signals and provide a comparative analysis of the minimax filter with the standard Kalman filter to show the improvement in signal-to-noise ratio (SNR).