Transfer Learning with Point Transformers
This work addresses the problem of transfer learning effectiveness for 3D point cloud classification, but it is incremental as it applies an existing method to new data with limited impact.
The paper investigated whether transfer learning with Point Transformers improves classification on 3D MNIST after pre-training on ModelNet10, finding that transfer learned models did not outperform models trained from scratch due to large distribution differences between the datasets.
Point Transformers are near state-of-the-art models for classification, segmentation, and detection tasks on Point Cloud data. They utilize a self attention based mechanism to model large range spatial dependencies between multiple point sets. In this project we explore two things: classification performance of these attention based networks on ModelNet10 dataset and then, we use the trained model to classify 3D MNIST dataset after finetuning. We also train the model from scratch on 3D MNIST dataset to compare the performance of finetuned and from-scratch model on the MNIST dataset. We observe that since the two datasets have a large difference in the degree of the distributions, transfer learned models do not outperform the from-scratch models in this case. Although we do expect transfer learned models to converge faster since they already know the lower level edges, corners, etc features from the ModelNet10 dataset.