Zero-Interaction Security -- Towards Sound Experimental Validation
This work addresses the problem of neglected reproducibility and data collection in computer science, particularly for zero-interaction security, but it is incremental as it focuses on validation rather than new methods.
The authors tackled the challenge of reproducibility and realistic datasets in computer science by reproducing five state-of-the-art zero-interaction security schemes and collecting a comprehensive real-world sensor dataset, resulting in the release of data, code, and documentation to facilitate future research.
Reproducibility and realistic datasets are crucial for advancing research. Unfortunately, they are often neglected as valid scientific contributions in many young disciplines, with computer science being no exception. In this article, we show the challenges encountered when reproducing the work of others, collecting realistic data in the wild, and ensuring that our own work is reproducible in turn. The presented findings are based on our study investigating the limits of zero-interaction security (ZIS) -- a novel concept, leveraging sensor data collected by Internet of Things (IoT) devices to pair or authenticate devices. In particular, we share our experiences in reproducing five state-of-the-art ZIS schemes, collecting a comprehensive dataset of sensor data from the real world, evaluating these schemes on the collected data, and releasing the data, code, and documentation to facilitate reproducibility of our results. In our discussion, we outline general considerations when conducting similar studies and give specific examples of technical and methodological issues that we experienced. We hope that our findings will raise awareness about the importance of reproducibility and realistic datasets in computer science and inform future research.