Why Data Quality Trumps Data Quantity

A dataset of 1,000 high-quality, diverse demonstrations often outperforms 10,000 noisy ones. Quality issues — sensor desynchronization, action discontinuities, task failures mixed with successes, and inconsistent labeling — propagate into trained policies and cause mysterious deployment failures.

The 15-Point Quality Checklist

Apply these checks to every batch of collected episodes before adding them to your training set.

  • Sensor timestamps synchronized within 10ms
  • No dropped camera frames (check frame count vs duration)
  • Action values within physical joint limits
  • No action discontinuities >2σ from mean step size
  • Success/failure labels verified by second reviewer
  • Task completed within expected duration bounds
  • No operator self-corrections in final dataset
  • Camera not occluded during critical phases
  • Proprioception matches commanded actions
  • Gripper state transitions at correct moments
  • No duplicate episodes
  • Metadata fields complete (task ID, operator, date)
  • Episode stored in target format (RLDS/LeRobot)
  • Batch statistics within expected distributions
  • Random sample spot-checked visually