At FullTilt: Real-Time Open-Set 3D Macromolecule Detection Directly from Tilted 2D Projections
For cryo-ET researchers, FullTilt provides a practical solution for rapid, large-scale visual proteomics analysis by drastically accelerating inference and reducing computational demands.
FullTilt enables real-time open-set 3D macromolecule detection directly from 2D tilt-series, eliminating the need for 3D tomogram reconstruction and sliding-window inference. It achieves state-of-the-art zero-shot performance while reducing runtime and VRAM requirements by orders of magnitude.
Open-set 3D macromolecule detection in cryogenic electron tomography eliminates the need for target-specific model retraining. However, strict VRAM constraints prohibit processing an entire 3D tomogram, forcing current methods to rely on slow sliding-window inference over extracted subvolumes. To overcome this, we propose FullTilt, an end-to-end framework that redefines 3D detection by operating directly on aligned 2D tilt-series. Because a tilt-series contains significantly fewer images than slices in a reconstructed tomogram, FullTilt eliminates redundant volumetric computation, accelerating inference by orders of magnitude. To process the entire tilt-series simultaneously, we introduce a tilt-series encoder to efficiently fuse cross-view information. We further propose a multiclass visual prompt encoder for flexible prompting, a tilt-aware query initializer to effectively anchor 3D queries, and an auxiliary geometric primitives module to enhance the model's understanding of multi-view geometry while improving robustness to adverse imaging artifacts. Extensive evaluations on three real-world datasets demonstrate that FullTilt achieves state-of-the-art zero-shot performance while drastically reducing runtime and VRAM requirements, paving the way for rapid, large-scale visual proteomics analysis. All code and data will be publicly available upon publication.