Robomimic: Canonical Imitation Learning Benchmark

A practitioner's guide to Robomimic — the ARISE Initiative's six-task-suite benchmark that became the reference point for BC-RNN, BC-Transformer, and Diffusion Policy evaluations.

TL;DR

Metric	Value
Task suites	6 (Lift, Can, Square, Transport, ToolHang, NutAssembly)
Robots	Franka Emika Panda, Rethink Sawyer (simulated in robosuite)
Modalities	Low-dim state, RGB (agent + wrist camera), object poses
License	MIT
Size	~1,000 demonstrations per task × 3 data qualities (PH, MH, MG)
Simulator	robosuite (MuJoCo)

What is Robomimic?

Robomimic was introduced by the ARISE Initiative at Stanford (Ajay Mandlekar, Danfei Xu, Josiah Wong, et al.) as a rigorous empirical study of what actually works in imitation learning for robot manipulation. The team collected three versions of each task's demonstrations — Proficient-Human (PH), Mixed-Human (MH), and Machine-Generated (MG) — and then ran a massive algorithm sweep (BC, BC-RNN, HBC, IRIS, BCQ, CQL, and several offline RL baselines) over the resulting 18 (task, data-quality) combinations. The paper's core contribution was showing that history-conditioned BC (BC-RNN) is a remarkably strong baseline and that the gap between offline RL and imitation learning collapses when data quality is high.

The software release that accompanied the paper — the robomimic Python package — has become even more influential than the benchmark itself. It ships a clean HDF5 data format, a unified training loop, configurable algorithm classes, and deterministic evaluation in robosuite, which together make it the fastest way to reproduce any behavioral cloning paper from the last four years. Diffusion Policy, ACT, and BeT all report Robomimic numbers as their canonical low-data benchmark.

Robomimic is also the direct intellectual ancestor of LIBERO — LIBERO keeps the robosuite scene format and the HDF5 convention, and extends them with a lifelong learning curriculum. If you understand Robomimic, you already know 80% of the LIBERO plumbing.

How to download & load

Robomimic ships a one-line download script that fetches the canonical HDF5 files from Stanford's mirror:

# Install the framework
pip install robomimic

# Download all task suites at PH, MH, and MG quality
python -m robomimic.scripts.download_datasets \
  --tasks lift can square transport tool_hang nut_assembly \
  --dataset_types ph mh mg \
  --hdf5_types image low_dim

# Train a Diffusion Policy baseline
python -m robomimic.scripts.train \
  --config configs/diffusion_policy.json \
  --dataset datasets/square/ph/low_dim_v141.hdf5

# Evaluate the trained checkpoint in robosuite
python -m robomimic.scripts.run_trained_agent \
  --agent trained_models/.../model_epoch_2000.pth \
  --n_rollouts 50 --video_path eval.mp4

The HDF5 format is self-describing — each trajectory includes observations, actions, rewards, and object poses — so you can bypass the robomimic training loop entirely and feed episodes into your own framework (LeRobot, Octo, diffusers) with ten lines of glue code.

Common use cases & model pairings

• BC baselines. BC-RNN and BC-Transformer on Robomimic are the canonical baselines that every new imitation paper must beat.
• Diffusion Policy evaluation. Cheng Chi's Diffusion Policy paper used Robomimic as its primary benchmark, and every follow-on paper keeps the same task list.
• Offline RL comparisons. The MG (machine-generated) split is one of the few real robot manipulation datasets designed specifically for offline RL research.
• Sim-to-real curriculum. Because the tasks are MuJoCo-based, teams often warm-start real-robot Franka policies on Robomimic Lift or Square before switching to real-world data.

Benchmarks & leaderboards

Robomimic tracks success rate averaged over 50 rollouts per task. Diffusion Policy reports 100% on Lift and Can, 95%+ on Square, and 80%+ on Transport and ToolHang from the PH split. See the Papers with Code Robomimic entry, the official project page, and the Mandlekar et al. CoRL 2021 paper for the reference numbers.

Technical deep dive: PH vs MH vs MG data

Robomimic's most useful design decision is that every task ships in three data-quality regimes. Proficient-Human (PH) is 200 demonstrations from a single expert teleoperator who had extensive practice on the task. Mixed-Human (MH) is 300 demonstrations split across six teleoperators of varying skill, which introduces realistic heterogeneity in trajectory shape, success rate, and hesitation. Machine-Generated (MG) is 300 rollouts from a sub-optimal SAC agent, used specifically to study offline RL under suboptimal-behaviour-policy assumptions.

The empirical punchline from the CoRL 2021 paper is that BC-RNN on PH data matches or beats every offline RL method tested on every task. On MH data the gap narrows but BC still wins. Only on MG data do offline RL methods (specifically CQL and IRIS) outperform BC, which lines up with the theoretical expectation that BC can only be as good as its data. This is why Diffusion Policy and follow-on papers almost always report the PH and MH numbers first.

Another subtle but important detail is the low_dim vs image observation split. Low-dim observations include ground-truth object poses, which makes them much easier to learn from than pixels. Always report both numbers — a method that only works with privileged state information is not ready for real deployment.

Known limitations

• Simulation only. All data is MuJoCo-generated. Sim-to-real transfer is not directly supported and requires your own randomization recipe.
• Small per-task coverage. 200-300 demos per task is tiny by modern VLA standards. MimicGen was designed specifically to scale Robomimic seeds.
• No language instructions. Tasks are identified by ID, not by natural language, so Robomimic cannot train language-conditioned policies directly.
• Franka / Sawyer only. Cross-embodiment transfer requires combining with Open X-Embodiment or similar.

FAQ

Is Robomimic deprecated now that LIBERO exists? No — Robomimic is still the cleanest single-task BC benchmark. LIBERO is the benchmark for lifelong learning; they serve different purposes and share infrastructure.

Can I use Robomimic as a training framework without using the datasets? Yes — the robomimic Python package is widely used as a general-purpose BC training loop on top of custom data.

What's the fastest way to baseline a new algorithm? Run BC-RNN and Diffusion Policy on PH data for all 6 tasks with the shipped configs. That takes ~2 GPU-days and gives you a reproducible comparison point.