Perceived Performance of Webpages In the Wild: Insights from Large-scale Crowdsourcing of Above-the-Fold QoE
This work addresses the need for better benchmarks in web performance optimization for developers and researchers, though it is incremental as it builds on existing crowdsourcing methods.
The study tackled the problem of understanding which webpage loading metrics align with human perception of speed, finding that common metrics like onLoad and TTFB fail to match majority human perception (less than 60% match), and introduced a machine learning model that achieves 87 ± 2% accuracy in predicting user choices.
Clearly, no one likes webpages with poor quality of experience (QoE). Being perceived as slow or fast is a key element in the overall perceived QoE of web applications. While extensive effort has been put into optimizing web applications (both in industry and academia), not a lot of work exists in characterizing what aspects of webpage loading process truly influence human end-user's perception of the "Speed" of a page. In this paper we present "SpeedPerception", a large-scale web performance crowdsourcing framework focused on understanding the perceived loading performance of above-the-fold (ATF) webpage content. Our end goal is to create free open-source benchmarking datasets to advance the systematic analysis of how humans perceive webpage loading process. In Phase-1 of our "SpeedPerception" study using Internet Retailer Top 500 (IR 500) websites (https://github.com/pahammad/speedperception), we found that commonly used navigation metrics such as "onLoad" and "Time To First Byte (TTFB)" fail (less than 60% match) to represent majority human perception when comparing the speed of two webpages. We present a simple 3-variable-based machine learning model that explains the majority end-user choices better (with $87 \pm 2\%$ accuracy). In addition, our results suggest that the time needed by end-users to evaluate relative perceived speed of webpage is far less than the time of its "visualComplete" event.