MOST: MR reconstruction Optimization for multiple downStream Tasks via continual learning
This work addresses the challenge of improving downstream task performance in medical imaging for researchers and practitioners, though it is incremental as it extends single-task optimization to multi-task scenarios.
The paper tackled the problem of optimizing MR reconstruction for multiple downstream tasks like segmentation by using continual learning to prevent performance degradation from error propagation and domain gaps, demonstrating that MOST outperformed baseline methods in comparative experiments.
Deep learning-based Magnetic Resonance (MR) reconstruction methods have focused on generating high-quality images but often overlook the impact on downstream tasks (e.g., segmentation) that utilize the reconstructed images. Cascading separately trained reconstruction network and downstream task network has been shown to introduce performance degradation due to error propagation and domain gaps between training datasets. To mitigate this issue, downstream task-oriented reconstruction optimization has been proposed for a single downstream task. Expanding this optimization to multi-task scenarios is not straightforward. In this work, we extended this optimization to sequentially introduced multiple downstream tasks and demonstrated that a single MR reconstruction network can be optimized for multiple downstream tasks by deploying continual learning (MOST). MOST integrated techniques from replay-based continual learning and image-guided loss to overcome catastrophic forgetting. Comparative experiments demonstrated that MOST outperformed a reconstruction network without finetuning, a reconstruction network with naïve finetuning, and conventional continual learning methods. The source code is available at: https://github.com/SNU-LIST/MOST.