Interrogating the Black Box: Transparency through Information-Seeking Dialogues
This work addresses the problem of verifying whether opaque learning systems comply with ethical governance constraints, which is critical for responsible AI development.
This paper proposes an investigator agent that queries a learning agent in an information-seeking dialogue to determine if its behavior adheres to ethical policies. The framework models compliance checking into three modular components: an investigator agent, a suspect agent, and an acceptance protocol, which uses argumentation semantics to assess consistent property adherence.
This paper is preoccupied with the following question: given a (possibly opaque) learning system, how can we understand whether its behaviour adheres to governance constraints? The answer can be quite simple: we just need to "ask" the system about it. We propose to construct an investigator agent to query a learning agent -- the suspect agent -- to investigate its adherence to a given ethical policy in the context of an information-seeking dialogue, modeled in formal argumentation settings. This formal dialogue framework is the main contribution of this paper. Through it, we break down compliance checking mechanisms into three modular components, each of which can be tailored to various needs in a vast amount of ways: an investigator agent, a suspect agent, and an acceptance protocol determining whether the responses of the suspect agent comply with the policy. This acceptance protocol presents a fundamentally different approach to aggregation: rather than using quantitative methods to deal with the non-determinism of a learning system, we leverage the use of argumentation semantics to investigate the notion of properties holding consistently. Overall, we argue that the introduced formal dialogue framework opens many avenues both in the area of compliance checking and in the analysis of properties of opaque systems.