Platform

SVRC Eval

The deployment-readiness platform for physical AI. Most robotics teams don't stall on model capability — they stall on the loop between a policy that works in the lab and one that survives a real workcell. SVRC Eval is the stack that runs that loop: refine your data, run your policy against real acceptance criteria, and feed every failure back into targeted collection.

Book a walkthrough See how the loop works
Hardware → Data → Eval, one stack Live refinery & CLI Operator-in-the-loop evaluation
Why this exists

The sim-to-real gap is a loop problem, not a model problem

After two years supplying hardware and data to robotics teams, the place projects stall is remarkably consistent — and it's not where people expect. It's three specific failure modes inside the sim-to-real loop.

1 · Actuator physics doesn't transfer
A policy trained in sim requests torque profiles real actuators can't sustain — thermal throttling, joint backlash, and unit-to-unit variance mean two of the same arm off the line don't share dynamics. "Embodiment-agnostic" turns out embodiment-specific.
2 · Teleop data is structurally off-policy
Bootstrap demonstrations come from a human operator, not your model. That covariate shift doesn't shrink by collecting more of the same — it shrinks through on-policy rollouts on real hardware, with failures fed back into targeted collection.
3 · Nobody defines "working" before deploy
Teams ship on lab success rates, then meet the customer's acceptance criteria in the long tail — the wrinkled shirt, the 4pm glare, the pallet 3cm off spec. Without an eval harness that encodes those criteria, readiness is a guess.
The loop we run

RL² — reinforcement learning in real life

Real hardware, real failure distributions, run as a cycle instead of post-hoc debugging. Each stage of the loop maps to something we operate today.

Deploy
Run the policy on real hardware in the target environment.
Capture failures
Log the rollouts and the failure distribution, not just the wins.
Evaluate
Score against acceptance criteria — automated metrics plus operator review.
Target collection
Failures define exactly what to collect next — not more of the same.
Redeploy
Retrain, gate on the eval, and run the cycle again.
What's in the platform

One stack, from raw capture to a readiness gate

SVRC Eval is built on the data infrastructure we already run in production. Every capability below is live today.

Data refinery
Egocentric and teleop capture, refined automatically: quality control, hand-mesh extraction, action segmentation, and LeRobot-v2 packaging. Run it from the workbench or the centeros refinery CLI.
Managed teleop collection
A real operator network and a WebSocket teleop layer with MCAP ingestion and time-synced multi-sensor alignment. This is the source of your on-policy and targeted data.
Model registry & rollouts
Register and version your policies, then run inference sessions in shadow or execute mode and record every action and outcome for evaluation.
Evaluation runs & regression gates
Define pass criteria — success rate, latency, recovery — per task and per dataset. Runs are tracked over versions, and a drop below threshold can automatically trigger a retrain with warm-start.
CLI, API & MCP
Everything is scriptable through the centeros CLI and exposed as MCP tools, so the loop runs from your own pipeline or from an agent — not just a dashboard.
Dataset management & access
Index, inspect, and share datasets with signed, time-limited access, and pull targeted slices back into the loop as failures dictate.
How readiness scoring works

Acceptance criteria, not vanity metrics

Lab success rate is not readiness. We encode customer acceptance criteria — the specific tasks, environments, and edge cases you'll be judged on in production — into an eval you can re-run on every model version.

Automated regression gates
Metric thresholds tracked across versions, so a regression is caught before it ships — the robotics equivalent of a CI gate on your policy.
Operator-in-the-loop evaluation
Physical tasks are hard to score automatically — a shirt folded "correctly" isn't a unit test. Our operator network runs real rollouts and judges outcomes against your criteria, so the score reflects the workcell, not a proxy.
Failure-driven data targeting
Every failed rollout points at the exact scene, object, or condition to collect next. The eval doesn't just grade — it writes your next data order.
Example readiness report SVRC Eval · sample output
model pi0-fast v0.3 status scored source 3 sessions · 52 decisive actions
88.5%
success rate · 46 ✓ / 6 ✗
161 ms
avg action latency
PASS
vs acceptance criteria
Top failure modes — these write the next targeted data order
operator_rejected · 4 timeout · 2

Sample output from the eval-runs pipeline scored on real rollout telemetry. Fully-automated policy scoring across arbitrary tasks is where we're investing next; today, readiness combines automated gates with our operator network so the number means something in the real world.

Work with us

Hitting one of these three walls?

We're comparing notes with teams building real deployments — and we host builders at 90 Welsh St in San Francisco. If your policy works in the lab but not the workcell, let's talk.

Book a walkthrough See data services