Topology-Aware Layer Pruning for Large Vision-Language Models
This work addresses the computational and memory costs of large vision-language models for deployment in resource-constrained scenarios, offering a more effective pruning method that preserves critical layers.
The paper proposes a topology-aware layer pruning framework for large vision-language models that uses zigzag persistent homology to quantify inter-layer topological consistency, enabling adaptive pruning that preserves critical representational transitions. The method consistently outperforms existing pruning methods across a wide range of sparsity ratios on multimodal benchmarks.
Large Language Models (LLMs) have demonstrated strong capabilities in natural language understanding and reasoning, while recent extensions that incorporate visual inputs enable them to process multimodal information. Despite these advances, Large Vision-Language Models (LVLMs) incur substantial computational and memory costs, hindering deployment in resource-constrained scenarios. Existing layer pruning methods typically rely on local similarity metrics or static proxy signals, failing to capture the global and dynamic evolution of representations across model depth, which often leads to the removal of transition-critical layers. To address this limitation, we propose a topology-aware layer pruning framework for LVLMs. Specifically, we represent layer wise hidden states as point clouds and models their evolution using \textit{simplicial complexes}. By leveraging \textit{zigzag persistent homology}, we quantify inter-layer topological consistency and enable adaptive pruning that preserves critical representational transitions. Extensive experiments on diverse multimodal benchmarks demonstrate that the proposed framework consistently outperforms existing pruning methods across a wide range of sparsity ratios. Our code is available at https://github.com/zpc456/TopoVLM.