The Imbalanced User-AI Relationships as an Ethical Failure of Front-End Design in Healthcare AI
For healthcare AI designers and ethicists, this work highlights a neglected ethical dimension in front-end design, though it is conceptual and lacks empirical validation.
The paper identifies imbalanced user-AI relationships as a front-end ethical failure in healthcare AI, where patients are visible to systems but cannot understand or influence their representations, and proposes reciprocity as a design orientation to address this.
Ethical discourse on AI in healthcare has focused predominantly on back-end concerns such as bias, fairness and explainability, while the front-end interface, where patients and clinicians actually encounter AI outputs, remains under explored. This paper identifies imbalanced user-AI relationships as a distinct class of front-end ethical failure: patients are rendered highly visible to AI systems through data inference, yet cannot understand, question or influence how they are represented. Through the concept of asymmetric legibility and a chat-based telemedicine case, we show how design choices e.g., default recommendations, restricted inputs and suppressed uncertainty, undermine agency, clinician judgment and human oversight even where systems are technically accurate. We propose reciprocity as a design orientation and offer interventions for more balanced, participatory user-AI relationships in healthcare.