Hagai Attias

Kensuke Sekihara, Hagai Attias, Julia P. Owen et al.

This paper examines the effectiveness of a sparse Bayesian algorithm to estimate multivariate autoregressive coefficients when a large amount of background interference exists. This paper employs computer experiments to compare two methods in the source-space causality analysis: the conventional least-squares method and a sparse Bayesian method. Results of our computer experiments show that the interference affects the least-squares method in a very severe manner. It produces large false-positive results, unless the signal-to-interference ratio is very high. On the other hand, the sparse Bayesian method is relatively insensitive to the existence of interference. However, this robustness of the sparse Bayesian method is attained on the scarifies of the detectability of true causal relationship. Our experiments also show that the surrogate data bootstrapping method tends to give a statistical threshold that are too low for the sparse method. The permutation-test-based method gives a higher (more conservative) threshold and it should be used with the sparse Bayesian method whenever the control period is available.

Hagai Attias

Current methods for learning graphical models with latent variables and a fixed structure estimate optimal values for the model parameters. Whereas this approach usually produces overfitting and suboptimal generalization performance, carrying out the Bayesian program of computing the full posterior distributions over the parameters remains a difficult problem. Moreover, learning the structure of models with latent variables, for which the Bayesian approach is crucial, is yet a harder problem. In this paper I present the Variational Bayes framework, which provides a solution to these problems. This approach approximates full posterior distributions over model parameters and structures, as well as latent variables, in an analytical manner without resorting to sampling methods. Unlike in the Laplace approximation, these posteriors are generally non-Gaussian and no Hessian needs to be computed. The resulting algorithm generalizes the standard Expectation Maximization algorithm, and its convergence is guaranteed. I demonstrate that this algorithm can be applied to a large class of models in several domains, including unsupervised clustering and blind source separation.