Sensors & Peripherals

Meta Quest 3 VR Teleop

Turn a Meta Quest 3 headset into a real-time robot arm controller — no leader arm required. Hand-tracking pose streams over UDP to Python, then to any supported arm. Latency under 120 ms on Wi-Fi 6.

Start Setup → Full Wiki Guide →

<120ms

End-to-End Latency

50 Hz

UDP Packet Rate

No ROS

Required

Compatible Arms

How It Works

Headset → UDP → Robot Arm

The system is built around a one-way UDP bridge. The Unity app on the Quest 3 reads hand-tracking data and broadcasts binary pose packets over Wi-Fi. A Python server on the control PC receives those packets and forwards commands to the robot via its native SDK. No ROS, no middleware — just sockets.

Meta Quest 3 Unity hand-tracking
VRHandPoseSender.cs
XR Hands subsystem

→

UDP Wi-Fi Port 8888 / 8889
Binary packet · 45 bytes
~50 Hz

→

Python Server UDP receiver thread
Safety validator
Bounded queue

→

Robot Controller Arm-specific SDK
Workspace clamping
Emergency stop

→

Robot Arm 6-DOF end-effector
Real-time motion

The key architectural insight: only the robot controller module changes between arm models. The Quest 3 Unity app, the UDP protocol, and the Python server infrastructure are all arm-agnostic. Swapping from one arm to another requires replacing roughly 80 lines of Python.

Getting Started

Your Setup Journey

Follow these steps to go from a bare Quest 3 to your first live teleoperation session. Takes about 1–2 hours on a familiar system.

Network & Prerequisites

Put Quest 3 and the control PC on the same LAN; verify UDP ports 8888/8889 are open

~15 min

Unity App Configuration

Set target PC IP, positionOffset, rotationOffset, and scaleFactor in the Inspector

~20 min

Python Server & Dependencies

Install piper_sdk, activate the CAN interface, run teleoperation_main.py

~20 min

piper_controller.py Walkthrough

Understand connect, set_pose, set_gripper, and emergency_stop before going live

~30 min

Safety Validation & First Session

Dry-run at 25% speed, verify workspace bounds, run live teleoperation

~30 min

Open Full Setup Guide →

Supported Hardware

Compatible Robot Arms

The modular controller pattern means only one Python file changes per arm. All arms share the same Quest 3 Unity app and UDP protocol.

AgileX Piper

6-DOF arm · CAN bus via USB adapter · piper_sdk Python library

Controller: piper_controller.py

Production-ready

OpenArm

7-DOF arm · SocketCAN / ROS 2 MoveIt2 · community open-source

Controller: openarm_controller.py

Beta

DK1 Bimanual Kit

Dual 6-DOF arms · uses port 8888 (right) and 8889 (left) simultaneously

Controller: dk1_controller.py

Beta

VLAI L1

Dual 6-DOF humanoid arms · ROS 2 bridge over TCP/IP

Controller: vlai_l1_controller.py

In development

To add a new arm, implement a controller class with five methods — connect, disconnect, set_pose, set_gripper, emergency_stop — and swap the import in teleoperation_main.py. See the adapter interface guide.

Technical Overview

System at a Glance

Headset	Meta Quest 3 (hand-tracking mode)
Transport	UDP over Wi-Fi (ports 8888 / 8889)
Packet rate	~50 Hz from Unity; robot commanded at 30 Hz
Packet size	45 bytes (fixed-length binary, little-endian)
End-to-end latency	50–120 ms on Wi-Fi 6 LAN
Host runtime	Python 3.10+ (no ROS required)
Unity version	2022.3 LTS+ with XR Hands package
Primary reference arm	AgileX Piper (CAN bus via USB)

Full Specs & Protocol →

Deep Dives

Guides & References

SVRC-curated content covering the full Quest 3 teleoperation stack.

Setup Guide — UDP Receiver & piper_controller.py → UDP Packet Schema & Latency Specs → Full Developer Wiki Guide → Adapter Interface — Port to Any Arm → DK1 Bimanual Kit → OpenArm 101 →

Get Help & Share

Community

Have a question about Quest 3 teleoperation or want to share your setup?

Ask the Forum → Discord #vr-teleop →

Ready to Teleop?

Get the Unity scripts, Python server, and piper_controller.py — everything you need to run your first VR teleoperation session.

Start Setup Guide Full Wiki → Book a Session