Jack Le

Jack Le, Anh H. N. Nguyen, Tien N. Nguyen

While code obfuscation impairs human code comprehension, it remains unclear if large language models share these failure modes. Building directly on a recent human study of program comprehension under code obfuscation, we evaluate whether large language models share the failure modes that obfuscation induces in human programmers. Evaluating several LLMs with five obfuscation tiers using the Block Model, we localize comprehension failures at the atom, block, relational, and macro levels. We find that reasoning-tuned models demonstrate significant alignment with human difficulty patterns across experience levels, whereas instruction and coder-tuned models show near-zero correlation. Chain-of-Thought trace length tracks task difficulty across tasks. Results indicate that performance under control-flow flattening degrades in proportion to state-space complexity, while adversarial identifier renaming disrupts comprehension through the interaction of semantic displacement and identifier-level interference. These findings suggest that reasoning-tuned LLMs approximate human sensitivity to code complexity more effectively than instruction-tuned variants.

Anh H. N. Nguyen, Jack Le, Ilse Lahnstein Coronado et al.

We investigate how code obfuscation influences human understanding of programs through an output-prediction task. To study this effect, we construct multiple levels of obfuscation, ranging from unobfuscated code to transformations involving identifier renaming, adversarially misleading identifiers, control-flow modifications, and combinations of these techniques. These transformations are applied to function-level programs written in Python and JavaScript. Participants were asked to predict program outputs while we recorded correctness, response time, and self-reported programming experience. Our results show that obfuscation generally increases the time required to reason about code and tends to reduce prediction accuracy. However, the relationship between obfuscation strength and performance is not strictly monotonic and varies across programming languages. JavaScript exhibits the expected pattern of increasing difficulty with stronger obfuscation, whereas Python displays a more complex trend in which certain renaming transformations can perform comparably to, or occasionally better than, the unobfuscated baseline. Response-time analyses further suggest that obfuscation shifts participants away from rapid, heuristic reasoning toward slower and more deliberate reasoning processes. Performance appears highest within a moderate range of response times, indicating that careful deliberation can improve accuracy, while extremely long response times often correspond to confusion. Finally, programming experience predicts performance primarily within a given language, with limited transfer across languages, suggesting that obfuscation challenges language-specific familiarity more than general programming ability.