Randomized Geometric Algebra Methods for Convex Neural Networks
This provides a more stable transfer learning method for practitioners using LLM embeddings, though it appears incremental in combining existing concepts.
The paper tackles the problem of improving transfer learning robustness for large language models by introducing randomized geometric algebra methods that enable convex optimization of neural networks. The results demonstrate enhanced performance and stability across multiple text classification datasets using GPT-4 and BERT embeddings.
We introduce randomized algorithms to Clifford's Geometric Algebra, generalizing randomized linear algebra to hypercomplex vector spaces. This novel approach has many implications in machine learning, including training neural networks to global optimality via convex optimization. Additionally, we consider fine-tuning large language model (LLM) embeddings as a key application area, exploring the intersection of geometric algebra and modern AI techniques. In particular, we conduct a comparative analysis of the robustness of transfer learning via embeddings, such as OpenAI GPT models and BERT, using traditional methods versus our novel approach based on convex optimization. We test our convex optimization transfer learning method across a variety of case studies, employing different embeddings (GPT-4 and BERT embeddings) and different text classification datasets (IMDb, Amazon Polarity Dataset, and GLUE) with a range of hyperparameter settings. Our results demonstrate that convex optimization and geometric algebra not only enhances the performance of LLMs but also offers a more stable and reliable method of transfer learning via embeddings.