An Entropy Weighted Nonnegative Matrix Factorization Algorithm for Feature Representation
This is an incremental improvement for researchers in machine learning and data analysis, addressing specific inaccuracies in NMF for tasks like image processing.
The paper tackles the problem of inaccurate feature representation in nonnegative matrix factorization (NMF) by proposing an entropy weighted NMF (EWNMF) that assigns optimizable weights to attributes, with experimental results demonstrating its feasibility and effectiveness.
Nonnegative matrix factorization (NMF) has been widely used to learn low-dimensional representations of data. However, NMF pays the same attention to all attributes of a data point, which inevitably leads to inaccurate representation. For example, in a human-face data set, if an image contains a hat on the head, the hat should be removed or the importance of its corresponding attributes should be decreased during matrix factorizing. This paper proposes a new type of NMF called entropy weighted NMF (EWNMF), which uses an optimizable weight for each attribute of each data point to emphasize their importance. This process is achieved by adding an entropy regularizer to the cost function and then using the Lagrange multiplier method to solve the problem. Experimental results with several data sets demonstrate the feasibility and effectiveness of the proposed method. We make our code available at https://github.com/Poisson-EM/Entropy-weighted-NMF.