Characterizing the Experience of Subjects in Software Engineering Studies
This work addresses a methodological gap for researchers conducting empirical studies in software engineering, though it is incremental as it builds on previous work.
The researchers tackled the lack of standardized criteria for characterizing the experience of subjects in software engineering studies by proposing a scheme with three attributes, finding that these attributes are positively but moderately correlated across 79 subjects in three skills.
Context: Empirical studies in software engineering are typically centered on human subjects, ranging from novice to experienced developers. The experience of these individuals is a key context factor that should be properly characterized for supporting the design of empirical studies and interpreting their results. However, the criteria adopted for characterizing the experience of subjects do not follow a standard and are frequently limited. Goal: Our research aims at establishing an optimized and comprehensive scheme to characterize the subjects' experience for studies in software engineering. Method: Based on previous work, we defined the first version of this scheme, composed of three experience attributes, including time, number of projects, and self-perception. In the last years, we applied the characterization scheme over four empirical studies, reaching the characterization of 79 subjects in three different skills. Results: We found that the attributes from our scheme are positively but moderately correlated. This finding suggests these attributes play a complementary role in characterizing the subjects' experience. Besides, we found that study subjects tend to enumerate the technical diversity of their background when summarizing their professional experience. Conclusion: The scheme proposed represents a feasible alternative for characterizing subjects of empirical studies in the field. However, we intend to conduct additional investigations with developers to evolve it.