Old Fictions, New Skins: Evaluating the Manipulative Capabilities of LLMs in Healthcare

Abstract

Large language models (LLMs) are increasingly piloted in African healthcare contexts, raising concerns about their potential to manipulate users in high-stakes settings. In a randomised experiment, we examined the manipulative capabilities of two publicly available models, ChatGPT 5.2 and DeepSeek V3.2, among Kenyan participants (N = 303). Participants interacted with either a manipulative variant or a non-manipulative variant before making a treatment decision within a hypothetical clinical scenario. The manipulative variant was prompted to covertly steer participants towards an incorrect treatment option while the non-manipulative variant served as the control condition. Manipulation success rates were higher in the manipulative condition (59.5%) than in the control condition (44.0%), with the effect reaching significance (OR = 2.11, 95% CI [1.12, 4.00], p = .021). These findings highlight the need for improved safety infrastructure specifically targeting manipulation, particularly given the integration of AI into healthcare systems across Africa.