InfiR : Crafting Effective Small Language Models and Multimodal Small Language Models in Reasoning
This work addresses computational and privacy barriers for deploying AI systems on edge devices, though it appears incremental as it builds on existing model scaling approaches.
The paper tackles the problem of high computational demands and privacy concerns in large language models by developing efficient small language models and multimodal small language models that retain competitive reasoning abilities, achieving state-of-the-art performance while minimizing development costs.
Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) have made significant advancements in reasoning capabilities. However, they still face challenges such as high computational demands and privacy concerns. This paper focuses on developing efficient Small Language Models (SLMs) and Multimodal Small Language Models (MSLMs) that retain competitive reasoning abilities. We introduce a novel training pipeline that enhances reasoning capabilities and facilitates deployment on edge devices, achieving state-of-the-art performance while minimizing development costs. \InfR~ aims to advance AI systems by improving reasoning, reducing adoption barriers, and addressing privacy concerns through smaller model sizes. Resources are available at https://github. com/Reallm-Labs/InfiR.