A Transdiagnostic Space of Disorder Like Phenotypes in Reinforcement Learning Agents

Abstract

Modelling psychological disorders in artificial agents offers a testbed for computational psychiatry and a lens on affective-control failure modes. Prior work induces one or two disorders by hand-tuned reward shaping, labels the behaviour post hoc, and reports single runs. We recast disorder modelling as dose-controllable manipulation of cognitive appraisal signals in an appraisal-guided PPO agent, expressing seven disorders (anxiety, mania, obsessive-compulsive checking, depression, impulsivity, addiction, and post-traumatic stress) each as a single knob grounded in a computational psychiatry account, with each symptom measured by a preregistered assay. Across more than a thousand runs (10 seeds, four controls, 95% confidence intervals) every disorder shows a graded, monotone dose-response that no control reproduces. Beyond these induced effects, three findings emerge that were not written into the reward: disorders self-organise into a two-dimensional affective space in which mania mirrors anxiety; removing a knob remits reward-distortion disorders (mania, checking, addiction) but not avoidance disorders (anxiety, PTSD), which recover under a graded exposure curriculum; and two simultaneous knobs interact nonadditively, yielding testable comorbidity predictions. The depression and addiction knobs further reproduce their double dissociation in a 3D pixel environment (MiniWorld) with a standard convolutional agent and no appraisal critic, showing the framework generalises beyond grid worlds.