Continuous Expert Assembly: Instance-Conditioned Low-Rank Residuals for All-in-One Image Restoration
For image restoration tasks with unknown, spatially non-uniform, and compositional degradations, CEA provides a more effective and efficient alternative to existing global conditioning and static expert routing methods.
CEA introduces a token-wise dynamic parameterization framework for all-in-one image restoration that uses cross-attention to synthesize instance-conditioned low-rank residuals, achieving improved average restoration quality on AIO-3, AIO-5, and CDD-11 benchmarks, particularly for spatially varying and compositional degradations, while maintaining efficiency.
Real-world image degradation is often unknown, spatially non-uniform, and compositional, requiring all-in-one restoration models to adapt a single set of weights to diverse local corruption patterns without test-time degradation labels. Existing methods typically modulate a shared backbone with global prompts or degradation descriptors, or route features through predefined expert pools. However, compact global conditioning can bottleneck localized degradation evidence, while static expert routing may produce homogeneous updates or rely on unstable sparse assignments. We propose \textbf{Continuous Expert Assembly} (CEA), a token-wise dynamic parameterization framework for all-in-one image restoration. CEA employs a lightweight \textbf{Cross-Attention Hyper-Adapter} to probe intermediate spatial features and synthesize instance-conditioned low-rank routing bases and residual directions. Each spatial token then assembles its own residual update via dense signed dot-product affinities over the generated rank-wise components, avoiding external prompts, static expert banks, and discrete Top- selection. The resulting assembly rule also admits a linear-attention perspective, making its dense token-wise routing behavior transparent. Experiments on AIO-3, AIO-5, and CDD-11 show that CEA improves average restoration quality over strong prompt-, descriptor-, and expert-based baselines, with the clearest gains on spatially varying and compositional degradations, while maintaining favorable parameter, FLOP, and runtime efficiency.