Executability of Python Snippets in Stack Overflow
This addresses the reliability of online code resources for developers, though it is incremental as it applies an existing method to new data.
The researchers tackled the problem of assessing the executability of Python code snippets from Stack Overflow, finding that only 27.92% are executable with minor adjustments, and executability has remained stable over time without significantly affecting answer acceptance rates.
Online resources today contain an abundant amount of code snippets for documentation, collaboration, learning, and problem-solving purposes. Their executability in a "plug and play" manner enables us to confirm their quality and use them directly in projects. But, in practice that is often not the case due to several requirements violations or incompleteness. However, it is a difficult task to investigate the executability on a large scale due to different possible errors during the execution. We have developed a scalable framework to investigate this for SOTorrent Python snippets. We found that with minor adjustments, 27.92% of snippets are executable. The executability has not changed significantly over time. The code snippets referenced in GitHub are more likely to be directly executable. But executability does not affect the chances of the answer to be selected as the accepted answer significantly. These properties help us understand and improve the interaction of users with online resources that include code snippets.