Server-side verification of client behavior in cryptographic protocols
This addresses security vulnerabilities in client-server protocols for applications like OpenSSL and Gmail, though it is incremental as it builds on existing verification techniques.
The paper tackles the problem of detecting malicious client behavior in cryptographic protocols by verifying that a client's messages are consistent with its intended program, without modifying the client or knowing all its inputs. The result is a toolchain that can detect exploits like Heartbleed within seconds and handle legitimate workloads such as Gmail.
Numerous exploits of client-server protocols and applications involve modifying clients to behave in ways that untampered clients would not, such as crafting malicious packets. In this paper, we demonstrate practical verification of a cryptographic protocol client's messaging behavior as being consistent with the client program it is believed to be running. Moreover, we accomplish this without modifying the client in any way, and without knowing all of the client-side inputs driving its behavior. Our toolchain for verifying a client's messages explores multiple candidate execution paths in the client concurrently, an innovation that we show is both specifically useful for cryptographic protocol clients and more generally useful for client applications of other types, as well. In addition, our toolchain includes a novel approach to symbolically executing the client software in multiple passes that defers expensive functions until their inputs can be inferred and concretized. We demonstrate client verification on OpenSSL to show that, e.g., Heartbleed exploits can be detected without Heartbleed-specific filtering and within seconds of the first malicious packet, and that verification of legitimate clients can keep pace with, e.g., Gmail workloads.