Transformer-based dimensionality reduction
This work addresses dimensionality reduction for machine learning applications, but it appears incremental as it adapts an existing Transformer model to a known problem.
The authors tackled dimensionality reduction by proposing Transformer-DR, a new model based on Vision Transformer, and found it to be effective in tasks like data visualization, image reconstruction, and face recognition compared to existing methods.
Recently, Transformer is much popular and plays an important role in the fields of Machine Learning (ML), Natural Language Processing (NLP), and Computer Vision (CV), etc. In this paper, based on the Vision Transformer (ViT) model, a new dimensionality reduction (DR) model is proposed, named Transformer-DR. From data visualization, image reconstruction and face recognition, the representation ability of Transformer-DR after dimensionality reduction is studied, and it is compared with some representative DR methods to understand the difference between Transformer-DR and existing DR methods. The experimental results show that Transformer-DR is an effective dimensionality reduction method.