Isotropy Cliffs: The Geometric Signature of Decision-Making in Large Language Models

Abstract

We investigate the geometry of decision-making in Multiple Choice Question Answering (MCQA) through the lens of isotropy. Analyzing five open-weight models across diverse datasets, we identify decision-critical transition layers characterized by a shift in isotropy, coinciding with a major representational change and the emergence of task-relevant clusters. We demonstrate that this synchronized geometric behavior is strongly correlated with downstream accuracy ($r\approx0.84$), displaying its relevance for successful decision-making. Furthermore, we show that this transition is robust to prompt variations, suggesting that it reflects a general mechanism of model behavior.