Optimizing human-AI collaboration: Effects of motivation and accuracy information in AI-supported decision-making [Optimierung der Mensch-KI-Zusammenarbeit: Auswirkungen von Motivation und Genauigkeitsinformationen bei KI-gestützter Entscheidungsfindung]

Eisbach, Simon; Langer, Markus; Hertel, Guido

Abstract

Abstract: Artificial intelligence (AI) systems increasingly support human decision-making in fields like medicine, management, and finance. However, such human-AI (HAI) collaboration is often less effective than AI systems alone. Moreover, efforts to make AI recommendations more transparent have failed to improve the decision quality of HAI collaborations. Based on dual process theories of cognition, we hypothesized that suboptimal HAI collaboration is partly due to heuristic information processing of humans, creating a trust imbalance towards the AI system. In an online experiment with 337 participants, we investigated motivation and accuracy information as potential factors to induce more deliberate elaboration of AI recommendations, and thus improve HAI collaboration. Participants worked on a simulated personnel selection task and received recommendations from a simulated AI system. Participants' motivation was varied through gamification, and accuracy information through additional information from the AI system. Results indicate that both motivation and accuracy information positively influenced HAI performance, but in different ways. While high motivation primarily increased humans’ use in high-quality recommendations only, accuracy information improved both the use of low- and high-quality recommendations. However, a combination of high motivation and accuracy information did not yield additional improvement of HAI performance.