SmartBugs: A Framework to Analyze Solidity Smart Contracts
This provides a practical solution for researchers and developers in blockchain security to standardize and enhance smart contract analysis, though it is incremental as it builds on existing tools and datasets.
The authors tackled the challenge of comparing and reproducing research on automated analysis of Ethereum smart contracts by developing SmartBugs, an extensible framework that simplifies tool execution, supporting 10 tools and datasets including 143 annotated vulnerable contracts and 47,518 unique contracts, and demonstrated its utility by improving vulnerability detection from 11% to 24% in a specific extension.
Over the last few years, there has been substantial research on automated analysis, testing, and debugging of Ethereum smart contracts. However, it is not trivial to compare and reproduce that research. To address this, we present SmartBugs, an extensible and easy-to-use execution framework that simplifies the execution of analysis tools on smart contracts written in Solidity, the primary language used in Ethereum. SmartBugs is currently distributed with support for 10 tools and two datasets of Solidity contracts. The first dataset can be used to evaluate the precision of analysis tools, as it contains 143 annotated vulnerable contracts with 208 tagged vulnerabilities. The second dataset contains 47,518 unique contracts collected through Etherscan. We discuss how SmartBugs supported the largest experimental setup to date both in the number of tools and in execution time. Moreover, we show how it enables easy integration and comparison of analysis tools by presenting a new extension to the tool SmartCheck that improves substantially the detection of vulnerabilities related to the DASP10 categories Bad Randomness, Time Manipulation, and Access Control (identified vulnerabilities increased from 11% to 24%).