Frequency-Domain Subspace Reuse and Integration for Multimodal Continual Instruction Tuning

抄録

Multimodal continual instruction tuning (MCIT) typically relies on attaching task-specific LoRA modules to solve catastrophic forgetting, yet sequentially growing LoRA parameters often suffer from capacity allocation and inter-task interference as tasks accumulate. Recent studies extended LoRA to the frequency domain to improve capacity allocation, but ignored task-specific subspaces that can be reused and integrated across tasks. As a result, subspaces remain redundant and underutilized in long and complex task sequences. To address these issues, we propose a Continual FreQuency LoRA (CFQ-LoRA). Specifically, we introduce a lightweight policy to reuse the similar LoRA subspace, improving scalability while maintaining strong retention under long task sequences. We also adopt input-dependent weights to aggregate and integrate existing task-specific LoRA subspaces for task-agnostic inference and improving subspace utilization. Experiments on two MCIT benchmarks show that our method consistent gains over state-of-the-art methods.