Eigenvalue and Generalized Eigenvalue Problems: Tutorial
It serves as an educational resource for learners in mathematics and machine learning, presenting established concepts without new contributions.
This tutorial explains eigenvalue and generalized eigenvalue problems, covering their definitions, optimization formulations, and solutions, with examples from machine learning such as PCA and Fisher discriminant analysis.
This paper is a tutorial for eigenvalue and generalized eigenvalue problems. We first introduce eigenvalue problem, eigen-decomposition (spectral decomposition), and generalized eigenvalue problem. Then, we mention the optimization problems which yield to the eigenvalue and generalized eigenvalue problems. We also provide examples from machine learning, including principal component analysis, kernel supervised principal component analysis, and Fisher discriminant analysis, which result in eigenvalue and generalized eigenvalue problems. Finally, we introduce the solutions to both eigenvalue and generalized eigenvalue problems.