The Closer You Look, The More You Learn: A Grey-box Approach to Protocol State Machine Learning
This addresses the challenge of protocol analysis for security researchers and developers, offering a grey-box method that improves learning efficiency and vulnerability detection, though it is incremental as it builds on existing state machine learning approaches.
The authors tackled the problem of inferring state machine models from protocol implementations without source code, proposing STATEINSPECTOR, which uses program analyses to combine run-time memory and I/O observations, leading to deeper state discovery and the identification of standards deviations and a high-impact vulnerability in a Wi-Fi implementation.
In this paper, we propose a new approach to infer state machine models from protocol implementations. Our method, STATEINSPECTOR, learns protocol states by using novel program analyses to combine observations of run-time memory and I/O. It requires no access to source code and only lightweight execution monitoring of the implementation under test. We demonstrate and evaluate STATEINSPECTOR's effectiveness on numerous TLS and WPA/2 implementations. In the process, we show STATEINSPECTOR enables deeper state discovery, increased learning efficiency, and more insightful post-mortem analyses than existing approaches. Further to improved learning, our method led us to discover several concerning deviations from the standards and a high impact vulnerability in a prominent Wi-Fi implementation.