Feature Graph Architectures
This work addresses the challenge of training deep networks for improved generalization, though it appears incremental as it builds on existing SVM methods with a new architectural approach.
The authors tackled the problem of deep network training by proposing feature graph architectures (FGA), which use structured initialization and training based on feature graphs to improve generalization; experimental results on deep SVMs showed robust and significant test set improvements over standard shallow SVMs across multiple datasets.
In this article we propose feature graph architectures (FGA), which are deep learning systems employing a structured initialisation and training method based on a feature graph which facilitates improved generalisation performance compared with a standard shallow architecture. The goal is to explore alternative perspectives on the problem of deep network training. We evaluate FGA performance for deep SVMs on some experimental datasets, and show how generalisation and stability results may be derived for these models. We describe the effect of permutations on the model accuracy, and give a criterion for the optimal permutation in terms of feature correlations. The experimental results show that the algorithm produces robust and significant test set improvements over a standard shallow SVM training method for a range of datasets. These gains are achieved with a moderate increase in time complexity.