Polaris: Scaling Up Instruction-Guided Image Generation Towards Millions of Personalized Style Needs
This work addresses the scalability and cost challenges of fine-tuning for personalized image generation by leveraging an existing ecosystem of models, offering a practical solution for users needing diverse styles.
Polaris introduces a retrieval framework that selects and integrates relevant fine-tuned models from a large library (6,500 checkpoints, 75,000 adapters) to fulfill diverse personalized image generation instructions without additional training, enabling scalable and controllable generation.
Users increasingly expect image generation models to quickly adapt to highly diverse and personalized requirements, such as producing images with distinctive styles or characteristics. Traditional approaches rely on fine-tuning, which is costly and difficult to scale. To cope with these limitations, the community has accumulated a growing library of fine-tuned modules and adapters, where each component targets specific generation needs and collectively serves as a foundation for handling new demands. This naturally raises a question: instead of repeatedly training new models, can we systematically exploit this expanding ecosystem to better fulfill user instructions? To this end, we present Polaris, an intelligent retrieval framework that automatically selects and integrates suitable models from the model library based on a user's instructions. The key insight is that harnessing such a massive and heterogeneous pool requires not only finding the most relevant modules among thousands of candidates, but also aligning them effectively for instruction-driven generation and editing. Polaris addresses this challenge by indexing over 6,500 checkpoints and 75,000 adapters, and retrieving the most relevant components given a user's input and instruction. In doing so, it delivers scalable, controllable, and well-aligned generation -- without any additional training.