Tobias Hoßfeld

Huyen T. T. Tran, Nam Pham Ngoc, Tobias Hoßfeld et al.

Thanks to the abundance of Web platforms and broadband connections, HTTP Adaptive Streaming has become the de facto choice for multimedia delivery nowadays. However, the visual quality of adaptive video streaming may fluctuate strongly during a session due to bandwidth fluctuations. So, it is important to evaluate the quality of a streaming session over time. In this paper, we propose a model to estimate the cumulative quality for HTTP Adaptive Streaming. In the model, a sliding window of video segments is employed as the basic building block. Through statistical analysis using a subjective dataset, we identify three important components of the cumulative quality model, namely the minimum window quality, the last window quality, and the average window quality. Experiment results show that the proposed model achieves high prediction performance and outperforms related quality models. In addition, another advantage of the proposed model is its simplicity and effectiveness for deployment in real-time estimation. The source code of the proposed model has been made available to the public at https://github.com/TranHuyen1191/CQM.

Christian Sieber, Korbinian Hagn, Christian Moldovan et al.

Mobile video consumption is increasing and sophisticated video quality adaptation strategies are required to deal with mobile throughput fluctuations. These adaptation strategies have to keep the switching frequency low, the average quality high and prevent stalling occurrences to ensure customer satisfaction. This paper proposes a novel methodology for the design of machine learning-based adaptation logics named HASBRAIN. Furthermore, the performance of a trained neural network against two algorithms from the literature is evaluated. We first use a modified existing optimization formulation to calculate optimal adaptation paths with a minimum number of quality switches for a wide range of videos and for challenging mobile throughput patterns. Afterwards we use the resulting optimal adaptation paths to train and compare different machine learning models. The evaluation shows that an artificial neural network-based model can reach a high average quality with a low number of switches in the mobile scenario. The proposed methodology is general enough to be extended for further designs of machine learning-based algorithms and the provided model can be deployed in on-demand streaming scenarios or be further refined using reward-based mechanisms such as reinforcement learning. All tools, models and datasets created during the work are provided as open-source software.

Tobias Hossfeld, Poul E. Heegaard, Martin Varela et al.

In the context of QoE management, network and service providers commonly rely on models that map system QoS conditions (e.g., system response time, paket loss, etc.) to estimated end user QoE values. Observable QoS conditions in the system may be assumed to follow a certain distribution, meaning that different end users will experience different conditions. On the other hand, drawing from the results of subjective user studies, we know that user diversity leads to distributions of user scores for any given test conditions (in this case referring to the QoS parameters of interest). Our previous studies have shown that to correctly derive various QoE metrics (e.g., Mean Opinion Score (MOS), quantiles, probability of users rating "good or better", etc.) in a system under given conditions, there is a need to consider rating distributions obtained from user studies, which are often times not available. In this paper we extend these findings to show how to approximate user rating distributions given a QoS-to-MOS mapping function and second order statistics. Such a user rating distribution may then be combined with a QoS distribution observed in a system to finally derive corresponding distributions of QoE scores. We provide two examples to illustrate this process: 1) analytical results using a Web QoE model relating waiting times to QoE, and 2) numerical results using measurements relating packet losses to video stall pattern, which are in turn mapped to QoE estimates. The results in this paper provide a solution to the problem of understanding the QoE distribution in a system, in cases where the necessary data is not directly available in the form of models going beyond the MOS, or where the full details of subjective experiments are not available.

Tobias Hossfeld, Poul E. Heegaard, Martin Varela et al.

For the quantification of QoE, subjects often provide individual rating scores on certain rating scales which are then aggregated into Mean Opinion Scores (MOS). From the observed sample data, the expected value is to be estimated. While the sample average only provides a point estimator, confidence intervals (CI) are an interval estimate which contains the desired expected value with a given confidence level. In subjective studies, the number of subjects performing the test is typically small, especially in lab environments. The used rating scales are bounded and often discrete like the 5-point ACR rating scale. Therefore, we review statistical approaches in the literature for their applicability in the QoE domain for MOS interval estimation (instead of having only a point estimator, which is the MOS). We provide a conservative estimator based on the SOS hypothesis and binomial distributions and compare its performance (CI width, outlier ratio of CI violating the rating scale bounds) and coverage probability with well known CI estimators. We show that the provided CI estimator works very well in practice for MOS interval estimators, while the commonly used studentized CIs suffer from a positive outlier ratio, i.e., CIs beyond the bounds of the rating scale. As an alternative, bootstrapping, i.e., random sampling of the subjective ratings with replacement, is an efficient CI estimator leading to typically smaller CIs, but lower coverage than the proposed estimator.

Tobias Hossfeld, Poul E. Heegaard, Martin Varela et al.

This technical report formally defines the QoE metrics which are introduced and discussed in the article "QoE Beyond the MOS: An In-Depth Look at QoE via Better Metrics and their Relation to MOS" by Tobias Hoßfeld, Poul E. Heegaard, Martin Varela, Sebastian Möller, accepted for publication in the Springer journal "Quality and User Experience". Matlab scripts for computing the QoE metrics for given data sets are available in GitHub.