The Effect of Presetting Decision-Making Criteria by Humans on AI Reliance

Abstract

To address tasks where complete AI automation is unsuitable, a collaborative decision-making process that integrates human and AI expertise is needed. However, conventional approaches in which humans approve or reject AI suggestions risk overreliance on AI and reduction of accuracy. An alternative has humans make an initial prediction, then review the AI’s recommendation before finalizing decisions, yet complex real-world tasks may cause people to rely heavily on AI without sufficient initial analysis. This paper investigates the effects of presenting hypothetical decision criteria to AI prior to forming predictions, comparing changes in AI dependence and accuracy. Specifically, humans select key features they prioritize, and AI references a Rashomon set with similar predictive performance but distinct rationales, offering predictions aligned with those criteria. User study on student grade prediction tasks reveals that examining presenting decision criteria fosters more balanced AI reliance.