Teleoperation — Leader-Follower, Bilateral & VR

1. Teleoperation Overview

OpenArm supports 1:1 teleoperation from a leader arm to a follower arm in two control modes: unilateral (one-way position mirroring) and bilateral (two-way force feedback).

Both modes are implemented in the openarm_teleop library and share the same multi-threaded architecture and per-joint control parameters.

2. Teleoperation Setup Guide

This guide walks you through building the openarm_teleop library from source.

Prerequisites

Before starting, ensure you have:

• The openarm_description library installed (see Software > Robot Description)
• Ubuntu 22.04 or 24.04 with SocketCAN configured
• Two OpenArm units (leader + follower) each with their own CAN interface

1. Clone the teleop repository.

   git clone https://github.com/enactic/openarm_teleop.git
   cd openarm_teleop

2. Install dependencies.

   sudo apt install -y software-properties-common
   sudo add-apt-repository -y ppa:openarm/main
   sudo apt update
   sudo apt install -y \
     libeigen3-dev \
     libopenarm-can-dev \
     liborocos-kdl-dev \
     liburdfdom-dev \
     liburdfdom-headers-dev \
     libyaml-cpp-dev \
     openarm-can-utils

3. Build the teleop library.

   cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
   cmake --build build

4. Initialize the CAN network.

   Each arm requires a dedicated CAN interface (1 arm = 1 CAN port).

   # Single interface (one arm)
   openarm-can-configure-socketcan can0 -fd
   
   # Four interfaces (bimanual leader + follower)
   # can0 = right leader, can1 = left leader
   # can2 = right follower, can3 = left follower
   openarm-can-configure-socketcan-4-arms -fd

3. Unilateral Control

Unilateral control is the basic teleoperation setup, where the follower arm directly mimics the leader's movements through position commands. It is a one-way control scheme — the leader sends motion commands, but no force feedback is returned.

System Architecture

The system uses a multi-threaded structure with three components managed through a custom PeriodicTimerThread class:

• Leader Thread — Reads leader arm joint states at high frequency
• Follower Thread — Sends position commands to the follower arm
• Admin Thread — Coordinates communication between leader and follower, handles the Control class logic

Control Parameters

Per-joint configuration for unilateral control:

Friction compensation formula applied per joint:

tau_f = Fc * tanh(k * dq) + Fv * dq + Fo

Running Unilateral Control

# Right arm (leader: can0, follower: can2)
cd openarm_teleop
./script/launch_unilateral.sh right_arm can0 can2

# Left arm (leader: can1, follower: can3)
./script/launch_unilateral.sh left_arm can1 can3

# Default CAN interfaces (omit arguments to use defaults)
# right_arm: can0 / can2
# left_arm:  can1 / can3

The launch script:

1. Validates the arm side argument
2. Applies CAN interface defaults if not specified
3. Generates temporary URDFs via xacro
4. Executes the unilateral control binary
5. Cleans temporary files on exit

4. Bilateral Control

Bilateral control enables two-way force feedback between the leader and follower arms, allowing the operator to feel what the follower arm touches. This is the recommended mode for contact-rich manipulation tasks.

System Architecture

The bilateral control system uses the same three-thread architecture as unilateral control, with the addition of force/torque feedback transmission from follower to leader:

• Leader Thread — Reads leader states; applies reaction forces from follower
• Follower Thread — Sends position commands; reads contact forces
• Admin Thread — Manages the bidirectional Control class

The Control class handles both bilateral and unilateral logic — mode is selected at launch time.

Control Parameters

Same per-joint parameters as unilateral (Kp, Kd, Fc, k, Fv, Fo) with additional tuning critical for stability:

• Kp — Higher values give stiffer position tracking; too high causes oscillation
• Kd — Velocity damping; critical for stability in contact
• Friction compensation uses same tanh model: tau_f = Fc * tanh(k * dq) + Fv * dq + Fo

Running Bilateral Control

# Right arm (leader: can0, follower: can2)
cd openarm_teleop
./script/launch_bilateral.sh right_arm can0 can2

# Left arm (leader: can1, follower: can3)
./script/launch_bilateral.sh left_arm can1 can3

Gain Tuning Notes

• Start with low Kp (e.g., 5–10) and increase gradually while monitoring for oscillation
• Increase Kd to damp oscillations before increasing Kp further
• Test in free space before attempting contact tasks
• Different gain sets may be needed for different joints (proximal joints typically need different tuning than distal joints)

5. VR Teleoperation

OpenArm supports real-time teleoperation using VR human body estimation streaming data. The current prototype maps Meta Quest 3 hand and arm motions to the robot's end-effector.

Key Features

• 7DOF Control — Maps both position and rotation for natural and intuitive manipulation
• Gesture-based Gripper Control — Uses VR finger distance for real-time open/close actions
• Scalable Mapping — Can incorporate full arm motion (upper/lower arm) for more human-like movements
• Real-time Operation — Runs at ~20 FPS with TCP data streaming

Current Status

The current prototype operates in NVIDIA Isaac Labs simulation, mapping Meta Quest 3 hand and arm motions to the robot's end-effector for simulated operations including object picking and placement.

VR Simulation Setup

The VR teleoperation simulation uses the Isaac Lab environment. See the Simulation page for Isaac Lab setup. The VR teleoperation code will be released as part of the openarm_teleop repository.

Architecture Overview