CARLoS: Retrieval via Concise Assessment Representation of LoRAs at Scale
This addresses the challenge of unreliable discovery methods for LoRAs, offering a practical system with potential broader relevance for analysis, though it is incremental in improving retrieval over existing methods.
The paper tackles the problem of discovering generative components like LoRAs in an unstructured ecosystem by introducing CARLoS, a framework that characterizes LoRAs using a three-part representation based on CLIP embeddings, enabling efficient retrieval that outperforms textual baselines in evaluations.
The rapid proliferation of generative components, such as LoRAs, has created a vast but unstructured ecosystem. Existing discovery methods depend on unreliable user descriptions or biased popularity metrics, hindering usability. We present CARLoS, a large-scale framework for characterizing LoRAs without requiring additional metadata. Analyzing over 650 LoRAs, we employ them in image generation over a variety of prompts and seeds, as a credible way to assess their behavior. Using CLIP embeddings and their difference to a base-model generation, we concisely define a three-part representation: Directions, defining semantic shift; Strength, quantifying the significance of the effect; and Consistency, quantifying how stable the effect is. Using these representations, we develop an efficient retrieval framework that semantically matches textual queries to relevant LoRAs while filtering overly strong or unstable ones, outperforming textual baselines in automated and human evaluations. While retrieval is our primary focus, the same representation also supports analyses linking Strength and Consistency to legal notions of substantiality and volition, key considerations in copyright, positioning CARLoS as a practical system with broader relevance for LoRA analysis.