Sixteen Years of Phishing User Studies: What Have We Learned?
It provides a consolidated evidence base for cybersecurity researchers and practitioners, though it is incremental as it synthesizes existing studies rather than introducing new methods.
This paper tackles the problem of inconsistent findings in phishing susceptibility studies by conducting a systematic review and meta-analysis of 16 years of user studies, finding that age and gender significantly affect susceptibility and that training improves detection ability.
Several previous studies have investigated user susceptibility to phishing attacks. A thorough meta-analysis or systematic review is required to gain a better understanding of these findings and to assess the strength of evidence for phishing susceptibility of a subpopulation, e.g., older users. We aim to determine whether an effect exists; another aim is to determine whether the effect is positive or negative and to obtain a single summary estimate of the effect. OBJECTIVES: We systematically review the results of previous user studies on phishing susceptibility and conduct a meta-analysis. METHOD: We searched four online databases for English studies on phishing. We included all user studies in phishing detection and prevention, whether they proposed new training techniques or analyzed users' vulnerability. FINDINGS: A careful analysis reveals some discrepancies between the findings. More than half of the studies that analyzed the effect of age reported no statistically significant relationship between age and users' performance. Some studies reported older people performed better while some reported the opposite. A similar finding holds for the gender difference. The meta-analysis shows: 1) a significant relationship between participants' age and their susceptibility 2) females are more susceptible than males 3) users training significantly improves their detection ability