Latent Space

A latent space is a compressed, lower-dimensional representation of data learned by a neural network — the output of an encoder that captures the most task-relevant features of an observation. In robot learning, latent spaces are used in VAEs (variational autoencoders) for learning structured representations of visual scenes, in world models for predicting future states, and in CVAE-based policies (like ACT) for encoding multimodal action distributions. A well-structured latent space places semantically similar observations close together, enabling interpolation, planning, and data augmentation in the latent domain rather than in raw pixel space.