Transfer Learning of Tabular Data by Finetuning Large Language Models
This addresses the challenge of transfer learning for tabular data in AI, offering a more efficient solution for domains with heterogeneous features and small sample sizes, though it is incremental as it adapts existing LLM techniques to a specific data type.
The paper tackles the problem of limited success in deep learning for tabular data by proposing an end-to-end finetuning method using large language models (LLMs), which outperforms state-of-the-art methods on ten benchmark datasets with less than ten features and uses a fraction of the computational cost.
Despite the artificial intelligence (AI) revolution, deep learning has yet to achieve much success with tabular data due to heterogeneous feature space and limited sample sizes without viable transfer learning. The new era of generative AI, powered by large language models (LLM), brings unprecedented learning opportunities to diverse data and domains. This paper investigates the effectiveness of an LLM application programming interface (API) and transfer learning of LLM in tabular data classification. LLM APIs respond to input text prompts with tokenized data and instructions, whereas transfer learning finetunes an LLM for a target classification task. This paper proposes an end-to-end finetuning of LLM to demonstrate cross-data transfer learning on ten benchmark data sets when large pre-trained tabular data models do not exist to facilitate transfer learning. The proposed LLM finetuning method outperforms state-of-the-art machine and deep learning methods on tabular data with less than ten features - a standard feature size for tabular data sets. The transfer learning approach uses a fraction of the computational cost of other deep learning or API-based solutions while ensuring competitive or superior classification performance.