State Space

The state space is the complete set of configurations a robot and its environment can be in. In RL, the Markov state s encodes all information needed to predict future rewards and state transitions — ideally a complete description of the world. In practice, the agent only has access to partial observations (images, joint angles) that may not fully capture the state (e.g., occluded objects, unknown physics parameters). Designing an observation space that approximates the Markov state well while remaining computationally tractable is a key challenge in robot learning system design.