A Systematic Evaluation of Positional Bias in Multi-Video Summarization with MLLMs
For researchers and practitioners using MLLMs for multi-video understanding, this work highlights a previously understudied reliability issue that motivates more robust order-invariant systems.
This paper investigates positional bias in multi-video summarization using MLLMs, finding that summary quality varies with input slot position, with effects being domain- and model-dependent. Even increasing visual or generation budget does not uniformly remove the imbalance.
Multimodal Large Language Models (MLLMs) are increasingly used for video understanding, yet their reliability under multi-video inputs remains poorly understood. We study positional bias in multi-video summarization, where the quality of a per-video summary can change with the video's input slot even when the underlying content is unchanged. We construct a benchmark from ActivityNet and News videos, covering Cooking, Domestic, Leisure, and News settings with two- and four-video inputs. We evaluate nine open-source and proprietary MLLMs and measure position effects with three complementary metrics: Coverage, Directional Positional Bias (DPB), and Middle-Edge Gap (MEG). Our results show that positional effects are domain- and model-dependent: signed directional bias can be small even when middle positions underperform, and increasing visual or generation budget does not uniformly remove the imbalance. We further analyze prompt-level mitigation methods. Together, the results show that multi-video summarization remains sensitive to input protocol and position, motivating more robust order-invariant multimodal systems.