What do all these Buttons do? Statically Mining Android User Interfaces at Scale
This enables large-scale empirical research on mobile UIs, addressing questions like app accessibility and feature consistency, though it is incremental as it builds on existing static analysis methods.
The authors tackled the problem of automatically analyzing Android app user interfaces at scale by introducing FRONTMATTER, a tool that statically extracts UI models and behavior from tens of thousands of real-world apps with high precision.
We introduce FRONTMATTER: a tool to automatically mine both user interface models and behavior of Android apps at a large scale with high precision. Given an app, FRONTMATTER statically extracts all declared screens, the user interface elements, their textual and graphical features, as well as Android APIs invoked by interacting with them. Executed on tens of thousands of real-world apps, FRONTMATTER opens the door for comprehensive mining of mobile user interfaces, jumpstarting empirical research at a large scale, addressing questions such as "How many travel apps require registration?", "Which apps do not follow accessibility guidelines?", "Does the user interface correspond to the description?", and many more. FRONTMATTER and the mined dataset are available under an open-source license.