Higher-order interactions of multi-layer prompt
This addresses a fundamental gap in prompt-tuning for representation learning, offering a more integrated approach that enhances robustness and generalizability, though it is incremental as it builds on existing prompt-tuning paradigms.
The paper tackles the problem of prompt-tuning methods treating prompts as isolated components across network layers, which limits expressive power, by proposing a framework that models higher-order interactions among multi-layer prompts, resulting in consistent performance improvements over state-of-the-art baselines on eight benchmark datasets, especially in few-shot scenarios.
The "pre-train, prompt" paradigm has successfully evolved in representation learning. While current prompt-tuning methods often introduce learnable prompts, they predominantly treat prompts as isolated, independent components across different network layers. This overlooks the complex and synergistic higher-order interactions that exist between prompts at various hierarchical depths, consequently limiting the expressive power and semantic richness of the prompted model. To address this fundamental gap, we propose a novel framework that explicitly models the Higher-order Interactions of Multi-layer Prompt. Our approach conceptualizes prompts from different layers not as separate entities, but as a cohesive system where their inter-relationships are critical. We design an innovative interaction module that captures these sophisticated, non-linear correlations among multi-layer prompts, effectively modeling their cooperative effects. This allows the model to dynamically aggregate and refine prompt information across the network's depth, leading to a more integrated and powerful prompting strategy. Extensive experiments on eight benchmark datasets demonstrate that our method, by leveraging these higher-order interactions, consistently surpasses state-of-the-art prompt-tuning baselines. The performance advantage is particularly pronounced in few-shot scenarios, validating that capturing the intricate interplay between multi-layer prompts is key to unlocking more robust and generalizable representation learning.