Cross-study Reliability of the Open Card Sorting Method
This addresses the need for consistent user experience design methods, though it is incremental as it validates an existing method rather than introducing new techniques.
The study evaluated the cross-study reliability of the open card sorting method for information architecture, finding that participants produced highly similar groupings and navigation schemes across six sorts with 140 participants.
Information architecture forms the foundation of users' navigation experience. Open card sorting is a widely-used method to create information architectures based on users' groupings of the content. However, little is known about the method's cross-study reliability: Does it produce consistent content groupings for similar profile participants involved in different card sort studies? This paper presents an empirical evaluation of the method's cross-study reliability. Six card sorts involving 140 participants were conducted: three open sorts for a travel website, and three for an eshop. Results showed that participants provided highly similar card sorting data for the same content. A rather high agreement of the produced navigation schemes was also found. These findings provide support for the cross-study reliability of the open card sorting method.