The AI Overtrust Paradox in Cybersecurity: A Bayesian Threat Detection Model and Practical algorithm

Consider an AI-based chatbot that processes a natural language dialogue with a human user to identify the most vulnerable components and cyber threats in the distributed computer communication network. We introduce and analyze two characteristics of human-AI interaction: chatbot trustworthiness and search effectiveness. The chatbot performs a sequence of scans over the given nodes in the system and continues searching until the human user declares in some round that the target has been found and terminates the search; the goal is to find the optimal sequence of scans that maximizes the expected user satisfaction. We develop a fast Bayesian algorithm to find this optimal sequence. Along with this result, we provide a formal proof of the “'more-for-less” paradox, demonstrating that reducing trust in AI can, under certain conditions, lead to improved overall search, Finally, we explain the underlying probabilistic mechanism behind this paradox.