Tuning-less Object Naming with a Foundation Model
This work addresses the problem of scalable object naming without tuning for applications requiring real-time learning of new entities, though it appears incremental in its use of existing models and mechanisms.
The authors implemented a real-time object naming system that can learn new named entities without any training or fine-tuning by using an existing foundation model and an attention-based association mechanism. The system achieved one-shot object naming and was evaluated for how many objects it could handle.
We implement a real-time object naming system that enables learning a set of named entities never seen. Our approach employs an existing foundation model that we consider ready to see anything before starting. It turns seen images into relatively small feature vectors that we associate with index to a gradually built vocabulary without any training of fine-tuning of the model. Our contribution is using the association mechanism known from transformers as attention. It has features that support generalization from irrelevant information for distinguishing the entities and potentially enable associating with much more than indices to vocabulary. As a result, the system can work in a one-shot manner and correctly name objects named in different contents. We also outline implementation details of the system modules integrated by a blackboard architecture. Finally, we investigate the system's quality, mainly how many objects it can handle in this way.