Applying empirical software engineering to software architecture: challenges and lessons learned
This work addresses the need for systematic empirical validation in software architecture, which is incremental as it builds on existing methods to enhance research credibility.
The paper tackles the problem of advancing software architecture by promoting empirical evidence over anecdotal experience, describing challenges and lessons from applying methods like controlled experiments and systematic reviews to support the development of well-formed theories in the field.
In the last 15 years, software architecture has emerged as an important software engineering field for managing the development and maintenance of large, software- intensive systems. Software architecture community has developed numerous methods, techniques, and tools to support the architecture process (analysis, design, and review). Historically, most advances in software architecture have been driven by talented people and industrial experience, but there is now a growing need to systematically gather empirical evidence about the advantages or otherwise of tools and methods rather than just rely on promotional anecdotes or rhetoric. The aim of this paper is to promote and facilitate the application of the empirical paradigm to software architecture. To this end, we describe the challenges and lessons learned when assessing software architecture research that used controlled experiments, replications, expert opinion, systematic literature reviews, obser- vational studies, and surveys. Our research will support the emergence of a body of knowledge consisting of the more widely-accepted and well-formed software architecture theories.