Trustworthy AI Software Engineers
For researchers and practitioners in software engineering and AI, this paper provides a conceptual framework and practical direction for ensuring trustworthiness of AI coding agents, though it remains a vision without empirical validation.
This vision paper conceptualizes AI software engineers as participants in human-AI teams and identifies key dimensions of trustworthiness (technical quality, transparency, epistemic humility, societal alignment). It proposes evidence-centric inspection as a practical approach to operationalize trustworthiness, shifting focus from raw outputs to selective signals and justifications.
With the rapid rise of AI coding agents, the fundamental premise of what it means to be a software engineer is in question. In this vision paper, we examine what it means for an AI agent to be considered a software engineer and then critically think about what makes such an agent trustworthy. Grounded in established definitions of SE (SE) and informed by recent research on agentic AI systems, we conceptualise AI software engineers as participants in human-AI SE teams composed of human software engineers and AI agents, and we distinguish trustworthiness as a key property of these systems and actors rather than a subjective human attitude. Extending on historical perspectives and emerging visions, we identify key dimensions that contribute to the trustworthiness of AI software engineers, spanning technical quality, transparency and accountability, epistemic humility, and societal and ethical alignment. Beyond defining these dimensions, we address a critical but underexplored challenge: how trustworthiness can be operationalised in practice. We therefore introduce the notion of evidence-centric inspection, arguing that developers should evaluate selective signals and justifications of trustworthiness rather than raw outputs, and we outline implications for rethinking verification, validation, and code review in human-AI SE teams.