Michael P. Collins

Emily Sandford, David Kipping, Michael Collins

A planetary system consists of a host star and one or more planets, arranged into a particular configuration. Here, we consider what information belongs to the configuration, or ordering, of 4286 Kepler planets in their 3277 planetary systems. First, we train a neural network model to predict the radius and period of a planet based on the properties of its host star and the radii and period of its neighbors. The mean absolute error of the predictions of the trained model is a factor of 2.1 better than the MAE of the predictions of a naive model which draws randomly from dynamically allowable periods and radii. Second, we adapt a model used for unsupervised part-of-speech tagging in computational linguistics to investigate whether planets or planetary systems fall into natural categories with physically interpretable "grammatical rules." The model identifies two robust groups of planetary systems: (1) compact multi-planet systems and (2) systems around giant stars ($\log{g} \lesssim 4.0$), although the latter group is strongly sculpted by the selection bias of the transit method. These results reinforce the idea that planetary systems are not random sequences -- instead, as a population, they contain predictable patterns that can provide insight into the formation and evolution of planetary systems.

Zhiyun Lu, Dong Guo, Alireza Bagheri Garakani et al.

We study large-scale kernel methods for acoustic modeling and compare to DNNs on performance metrics related to both acoustic modeling and recognition. Measuring perplexity and frame-level classification accuracy, kernel-based acoustic models are as effective as their DNN counterparts. However, on token-error-rates DNN models can be significantly better. We have discovered that this might be attributed to DNN's unique strength in reducing both the perplexity and the entropy of the predicted posterior probabilities. Motivated by our findings, we propose a new technique, entropy regularized perplexity, for model selection. This technique can noticeably improve the recognition performance of both types of models, and reduces the gap between them. While effective on Broadcast News, this technique could be also applicable to other tasks.

Zhiyun Lu, Avner May, Kuan Liu et al.

The computational complexity of kernel methods has often been a major barrier for applying them to large-scale learning problems. We argue that this barrier can be effectively overcome. In particular, we develop methods to scale up kernel models to successfully tackle large-scale learning problems that are so far only approachable by deep learning architectures. Based on the seminal work by Rahimi and Recht on approximating kernel functions with features derived from random projections, we advance the state-of-the-art by proposing methods that can efficiently train models with hundreds of millions of parameters, and learn optimal representations from multiple kernels. We conduct extensive empirical studies on problems from image recognition and automatic speech recognition, and show that the performance of our kernel models matches that of well-engineered deep neural nets (DNNs). To the best of our knowledge, this is the first time that a direct comparison between these two methods on large-scale problems is reported. Our kernel methods have several appealing properties: training with convex optimization, cost for training a single model comparable to DNNs, and significantly reduced total cost due to fewer hyperparameters to tune for model selection. Our contrastive study between these two very different but equally competitive models sheds light on fundamental questions such as how to learn good representations.