Fisher and Kernel Fisher Discriminant Analysis: Tutorial
It serves as an educational resource for researchers and practitioners in machine learning, providing a comprehensive guide to FDA techniques, but it is incremental as it synthesizes existing knowledge without introducing new methods.
This tutorial paper explains Fisher Discriminant Analysis (FDA) and its kernel variant, covering theoretical foundations, practical examples, and comparisons with PCA, including simulations on the AT&T face dataset.
This is a detailed tutorial paper which explains the Fisher discriminant Analysis (FDA) and kernel FDA. We start with projection and reconstruction. Then, one- and multi-dimensional FDA subspaces are covered. Scatters in two- and then multi-classes are explained in FDA. Then, we discuss on the rank of the scatters and the dimensionality of the subspace. A real-life example is also provided for interpreting FDA. Then, possible singularity of the scatter is discussed to introduce robust FDA. PCA and FDA directions are also compared. We also prove that FDA and linear discriminant analysis are equivalent. Fisher forest is also introduced as an ensemble of fisher subspaces useful for handling data with different features and dimensionality. Afterwards, kernel FDA is explained for both one- and multi-dimensional subspaces with both two- and multi-classes. Finally, some simulations are performed on AT&T face dataset to illustrate FDA and compare it with PCA.