Controlled Experimentation in Naturalistic Mobile Settings
This addresses a methodological problem for Human Factors researchers by providing a way to validate that controlled experiments on small devices proceed naturally, though it is incremental as it builds on existing observational approaches.
The researchers tackled the difficulty of conducting controlled user experiments on small devices in naturalistic mobile settings by developing protocols and validation checks based on observational data from 400 mobile users, ensuring experiments are perceived as natural.
Performing controlled user experiments on small devices in naturalistic mobile settings has always proved to be a difficult undertaking for many Human Factors researchers. Difficulties exist, not least, because mimicking natural small device usage suffers from a lack of unobtrusive data to guide experimental design, and then validate that the experiment is proceeding naturally.Here we use observational data to derive a set of protocols and a simple checklist of validations which can be built into the design of any controlled experiment focused on the user interface of a small device. These, have been used within a series of experimental designs to measure the utility and application of experimental software. The key-point is the validation checks -- based on the observed behaviour of 400 mobile users -- to ratify that a controlled experiment is being perceived as natural by the user. While the design of the experimental route which the user follows is a major factor in the experimental setup, without check validations based on unobtrusive observed data there can be no certainty that an experiment designed to be natural is actually progressing as the design implies.