Motion Style Transfer: Modular Low-Rank Adaptation for Deep Motion Forecasting
This addresses the challenge of adapting motion forecasting models to unseen agent types and scene contexts with limited data, representing an incremental improvement in transfer learning for domain-specific applications.
The paper tackles the problem of poor performance of deep motion forecasting models when training data is limited by proposing a transfer learning approach with a low-rank motion style adapter (MoSA) to efficiently adapt pre-trained models to new domains, showing that it outperforms prior methods on several benchmarks.
Deep motion forecasting models have achieved great success when trained on a massive amount of data. Yet, they often perform poorly when training data is limited. To address this challenge, we propose a transfer learning approach for efficiently adapting pre-trained forecasting models to new domains, such as unseen agent types and scene contexts. Unlike the conventional fine-tuning approach that updates the whole encoder, our main idea is to reduce the amount of tunable parameters that can precisely account for the target domain-specific motion style. To this end, we introduce two components that exploit our prior knowledge of motion style shifts: (i) a low-rank motion style adapter that projects and adjusts the style features at a low-dimensional bottleneck; and (ii) a modular adapter strategy that disentangles the features of scene context and motion history to facilitate a fine-grained choice of adaptation layers. Through extensive experimentation, we show that our proposed adapter design, coined MoSA, outperforms prior methods on several forecasting benchmarks.