Observation History

Including a window of past observations (rather than only the current frame) in the policy input, giving the policy implicit memory for partially observable tasks. N-step observation stacks are simple and effective for tasks requiring velocity estimation from position-only observations, handling brief occlusions, and detecting temporal patterns.