Designing Large Foundation Models for Efficient Training and Inference: A Survey
It provides a comprehensive overview for researchers and practitioners aiming to make large language models more efficient and affordable, but it is incremental as a survey paper.
This survey examines efficient training and inference technologies for large foundation models, focusing on model and system design to reduce computational costs and improve accessibility.
This paper focuses on modern efficient training and inference technologies on foundation models and illustrates them from two perspectives: model and system design. Model and System Design optimize LLM training and inference from different aspects to save computational resources, making LLMs more efficient, affordable, and more accessible. The paper list repository is available at https://github.com/NoakLiu/Efficient-Foundation-Models-Survey.