Multidimensional Scaling, Sammon Mapping, and Isomap: Tutorial and Survey
It serves as an educational resource for researchers and practitioners in machine learning, offering a comprehensive overview of fundamental manifold learning methods, but it is incremental as it reviews existing techniques without introducing new ones.
This paper provides a tutorial and survey on Multidimensional Scaling (MDS), Sammon mapping, and Isomap, reviewing their theories and explaining various categories and extensions such as out-of-sample embedding and Nystrom approximation for big data.
Multidimensional Scaling (MDS) is one of the first fundamental manifold learning methods. It can be categorized into several methods, i.e., classical MDS, kernel classical MDS, metric MDS, and non-metric MDS. Sammon mapping and Isomap can be considered as special cases of metric MDS and kernel classical MDS, respectively. In this tutorial and survey paper, we review the theory of MDS, Sammon mapping, and Isomap in detail. We explain all the mentioned categories of MDS. Then, Sammon mapping, Isomap, and kernel Isomap are explained. Out-of-sample embedding for MDS and Isomap using eigenfunctions and kernel mapping are introduced. Then, Nystrom approximation and its use in landmark MDS and landmark Isomap are introduced for big data embedding. We also provide some simulations for illustrating the embedding by these methods.