Adapting Binary Information Retrieval Evaluation Metrics for Segment-based Retrieval Tasks
This work addresses the need for standardized evaluation in segment-based retrieval, particularly for video hyperlinking, but is incremental as it reuses existing metrics.
The paper adapts existing binary information retrieval evaluation metrics, such as precision at n and mean average precision, for segment-based retrieval tasks, specifically applied to video hyperlinking.
This report describes metrics for the evaluation of the effectiveness of segment-based retrieval based on existing binary information retrieval metrics. This metrics are described in the context of a task for the hyperlinking of video segments. This evaluation approach re-uses existing evaluation measures from the standard Cranfield evaluation paradigm. Our adaptation approach can in principle be used with any kind of effectiveness measure that uses binary relevance, and for other segment-baed retrieval tasks. In our video hyperlinking setting, we use precision at a cut-off rank n and mean average precision.