Maximal benefits and possible detrimental effects of binary decision aids
This addresses a problem for system designers in automation, highlighting potential detrimental effects of decision aids, though it is incremental in formal analysis.
The paper analyzes the performance of binary decision aids, showing that combining a user and an aid often yields minimal gains over the better detector alone and can even lower detection performance if weights are non-optimal.
Binary decision aids, such as alerts, are a simple and widely used form of automation. The formal analysis of a user's task performance with an aid sees the process as the combination of information from two detectors who both receive input about an event and evaluate it. The user's decisions are based on the output of the aid and on the information, the user obtains independently. We present a simple method for computing the maximal benefits a user can derive from a binary aid as a function of the user's and the aid's sensitivities. Combining the user and the aid often adds little to the performance the better detector could achieve alone. Also, if users assign non-optimal weights to the aid, performance may drop dramatically. Thus, the introduction of a valid aid can actually lower detection performance, compared to a more sensitive user working alone. Similarly, adding a user to a system with high sensitivity may lower its performance. System designers need to consider the potential adverse effects of introducing users or aids into systems.