Future Slot Prediction for Unsupervised Object Discovery in Surgical Video
This work addresses the problem of real-time interpretation of surgical video for healthcare applications, representing an incremental improvement by adapting existing methods to a specific domain.
The paper tackled the challenge of applying unsupervised object-centric slot attention to heterogeneous surgical videos, where adaptive slot count methods underperform, by proposing a dynamic temporal slot transformer (DTST) module for temporal reasoning and future slot prediction, achieving state-of-the-art performance on multiple surgical databases.
Object-centric slot attention is an emerging paradigm for unsupervised learning of structured, interpretable object-centric representations (slots). This enables effective reasoning about objects and events at a low computational cost and is thus applicable to critical healthcare applications, such as real-time interpretation of surgical video. The heterogeneous scenes in real-world applications like surgery are, however, difficult to parse into a meaningful set of slots. Current approaches with an adaptive slot count perform well on images, but their performance on surgical videos is low. To address this challenge, we propose a dynamic temporal slot transformer (DTST) module that is trained both for temporal reasoning and for predicting the optimal future slot initialization. The model achieves state-of-the-art performance on multiple surgical databases, demonstrating that unsupervised object-centric methods can be applied to real-world data and become part of the common arsenal in healthcare applications.