Algorithmic Decision-Making Safeguarded by Human Knowledge
This addresses the challenge for analysts and managers in business intelligence to integrate human expertise with AI-driven decisions, though it is incremental as it builds on existing augmentation concepts.
The paper tackles the problem of conflicts between algorithmic decisions and human knowledge by proposing a framework where human insights set guardrails to clip algorithmic outputs when they seem unreasonable, showing that this augmentation improves performance in cases like lack of domain knowledge, model misspecification, or data contamination.
Commercial AI solutions provide analysts and managers with data-driven business intelligence for a wide range of decisions, such as demand forecasting and pricing. However, human analysts may have their own insights and experiences about the decision-making that is at odds with the algorithmic recommendation. In view of such a conflict, we provide a general analytical framework to study the augmentation of algorithmic decisions with human knowledge: the analyst uses the knowledge to set a guardrail by which the algorithmic decision is clipped if the algorithmic output is out of bound, and seems unreasonable. We study the conditions under which the augmentation is beneficial relative to the raw algorithmic decision. We show that when the algorithmic decision is asymptotically optimal with large data, the non-data-driven human guardrail usually provides no benefit. However, we point out three common pitfalls of the algorithmic decision: (1) lack of domain knowledge, such as the market competition, (2) model misspecification, and (3) data contamination. In these cases, even with sufficient data, the augmentation from human knowledge can still improve the performance of the algorithmic decision.