IRJun 9, 2016
The Effects of Latency Penalties in Evaluating Push Notification Systems
arXiv:1606.03066v12 citations
Originality Synthesis-oriented
AI Analysis
This work addresses evaluation design issues for researchers and practitioners in information retrieval, but it is incremental as it builds on existing TREC frameworks.
The study analyzed how different latency penalties affect the evaluation of push notification systems, using data from the TREC 2015 Microblog track, to guide metric design for the TREC 2016 Real-Time Summarization track.
We examine the effects of different latency penalties in the evaluation of push notification systems, as operationalized in the TREC 2015 Microblog track evaluation. The purpose of this study is to inform the design of metrics for the TREC 2016 Real-Time Summarization track, which is largely modeled after the TREC 2015 evaluation design.