huggingface/lerobot
63.3
Adequate · 16 November 2026
198.2k
lines of production code
Python
primary language
5
measurements over time
What this system is
Features
Add Bi SO Follower bimanual robot support
Users can now control bimanual SO Follower arms via the new \BiSOFollower\ class and \BiSOFollowerConfig\. This implementation wraps two individual \SOFollower\ instances (left and right arms), automatically prefixing observation and action keys with \left\\ or \right\\ to distinguish the sides, while keeping top-level camera keys unprefixed. The configuration allows separate setup for each arm's port, calibration, and PID coefficients, as well as shared camera definitions.
_src/lerobot/robots/bi\_so\follower · high confidence
Add Bi SO Leader bimanual teleoperator support
Users can now control dual SO Leader arms as a single bimanual device. The new BiSOLeader class manages left and right arms independently, prefixing action and feedback data (e.g., left\_shoulder\_pan.pos) to distinguish inputs. It supports smooth handover between policy and human control by enabling/disabling torque on both arms and routing feedback commands to the correct side.
_src/lerobot/teleoperators/bi\_so\leader · high confidence
Add BiOpenArmFollower robot implementation for bimanual setups
Users can now control bimanual OpenArm follower configurations via the new BiOpenArmFollower class. This implementation manages two independent OpenArmFollower instances (left and right arms) while handling camera observation keys by prefixing per-arm cameras with 'left\' or 'right\' and keeping top-level cameras unprefixed. The robot is registered under the 'bi\_openarm\_follower' name and requires a BiOpenArmFollowerConfig specifying separate configurations for each arm.
_src/lerobot/robots/bi\_openarm\follower · high confidence
Add FLUX 3 Action PEFT example with training configuration and base export script
The \examples/flux3\ directory now includes a complete example for performing Parameter-Efficient Fine-Tuning (PEFT) on the FLUX 3 Action model. This addition provides \lora.json\, a shared configuration file for training LoRA adapters on SO-101 and DROID checkpoints, and \export\_base.py\, a script to prepare a reloadable base policy and processors from a trunk model and dataset statistics. The included README documents the usage for both SO-101 and DROID tasks, specifying default training parameters such as rank/alpha 32 LoRA, EMA decay 0.999, and effective batch size 8, while noting that DROID validation is currently limited to configuration loading checks.
examples/flux3 · high confidence
Add Homunculus teleoperator support for arm and glove devices
This change introduces the Homunculus teleoperator implementation, adding the \HomunculusArm\ and \HomunculusGlove\ classes along with their configuration classes (\HomunculusArmConfig\, \HomunculusGloveConfig\) and a joint translation utility (\homunculus\_glove\_to\_hope\_jr\_hand\). Users can now control the Homunculus Arm and Glove hardware via serial ports, with built-in support for interactive calibration of joint ranges of motion and connection state management.
src/lerobot/teleoperators/homunculus · high confidence
Add Intel RealSense camera support with manual exposure, gain, and white balance controls
Users can now capture frames and depth maps from Intel RealSense cameras using the new RealSenseCamera class. This addition allows specifying the camera by serial number or name, enabling depth stream capture, and applying manual controls for exposure, gain, and white balance via the RealSenseCameraConfig.
src/lerobot/cameras/realsense · high confidence
Add Isaac Teleop → SO-101 teleoperation and dataset recording example
Users can now teleoperate an SO-101/SO-100 follower arm and record LeRobot datasets using NVIDIA Isaac Teleop. The example supports two input devices: an XR (VR) controller (e.g., Quest 3, Pico 4) that drives the end-effector through a squeeze-to-engage clutch and LeRobot's Cartesian IK pipeline, and a back-drivable SO-101 leader arm that mirrors 1:1 onto the follower. The package includes CLI scripts for teleoperation and dataset recording, shared control-loop infrastructure, and configuration for CloudXR runtime integration.
_examples/isaac\_teleop\_to\so101 · high confidence
Add Koch Follower robot support
Introduces a new robot implementation for the Koch Follower arm (v1.0 and v1.1), including the \KochFollower\ class and \KochFollowerConfig\. This addition enables users to control the arm via a serial bus, supporting configuration for motor ports, camera inputs (including depth maps), and safety limits like \max\_relative\_target\. The implementation handles connection, calibration, and observation/action loops specific to this hardware.
_src/lerobot/robots/earthrover\_mini\_plus, src/lerobot/robots/koch\_follower, src/lerobot/robots/openarm\follower · high confidence
Add LeRobot Quickstart notebook with ready-to-paste commands
A new Jupyter notebook (\examples/notebooks/quickstart.ipynb\) has been added to provide a guided workflow for calibration, teleoperation, data collection, training, and evaluation. The notebook allows users to configure settings such as robot type, ports, cameras, and dataset parameters in a single cell, then generates ready-to-paste terminal commands for each step, simplifying the initial setup and usage of the LeRobot platform.
examples/notebooks · high confidence
Add RTC dataset evaluation script
A new evaluation script (\examples/rtc/eval\_dataset.py\) is introduced to assess Real-Time Chunking (RTC) performance. This tool allows users to compare action predictions with and without RTC enabled on dataset samples, measuring consistency and ground truth alignment. It supports various policies (such as smolvla, pi0, and pi0.5) and includes options for device selection (CUDA, MPS, CPU), random seeding for reproducibility, and optional torch.compile acceleration.
examples/rtc · high confidence
Add Reachy 2 teleoperator support
Users can now control the Reachy 2 robot as a teleoperator device. This change introduces the Reachy2Teleoperator class and its configuration, allowing users to map joint positions and mobile base velocities from the robot to the system. The implementation supports configuring which robot parts to use (mobile base, left/right arms, neck, antennas) and offers a choice between using present or goal joint positions for actions.
_src/lerobot/teleoperators/reachy2\teleoperator · high confidence
Add RobStride CAN motor bus implementation
Users can now control RobStride motors via a new CAN bus interface. This change introduces the \RobstrideMotorsBus\ class, which handles communication using the MIT control protocol, supports CAN FD, and includes configuration tables for motor types, limits, and baudrates.
src/lerobot/motors/robstride · high confidence
Add Robometer reward model support
Introduces the Robometer reward model (RobometerRewardModel) and its associated configuration (RobometerConfig), processor (RobometerEncoderProcessorStep), and a CLI tool (compute\_rabc\_weights) for computing per-frame progress and success metrics on LeRobot datasets. This enables users to evaluate robotic trajectories using the Robometer-4B vision-language model, which predicts frame-level progress and binary success based on video and task instructions.
src/lerobot/rewards/robometer · high confidence
Add SARM subtask annotation tooling
Introduces a new data processing module for SARM (Stage-Aware Reward Modeling) subtask annotation. This includes a script that uses a local Qwen3-VL model to analyze robot demonstration videos and automatically generate subtask timestamps (sparse, dense, or dual modes), along with utilities to compute temporal proportions for dataset priors.
_src/lerobot/data\processing · high confidence
Add Unitree G1 exoskeleton teleoperation support
Users can now control the Unitree G1 robot using bimanual exoskeleton arms. This change introduces the \UnitreeG1Teleoperator\ class and supporting modules for serial communication, sensor calibration, and inverse kinematics mapping, enabling exoskeleton-driven teleoperation alongside the existing wireless remote controller.
_src/lerobot/teleoperators/unitree\g1 · high confidence
Add Unitree G1 robot support with teleoperation and locomotion controllers
Users can now control the Unitree G1 humanoid robot through LeRobot. This change introduces the \UnitreeG1\ robot class and its configuration, enabling teleoperation via a DDS-to-ZMQ bridge server (\run\_g1\_server.py\) that forwards robot state and commands. It also adds three selectable locomotion/whole-body controllers—GR00T, Holosoma, and SONIC—configurable via the \controller\ field in \UnitreeG1Config\, along with utilities for inverse kinematics and joint mapping.
_src/lerobot/robots/unitree\g1 · high confidence
Add ZMQ-based camera support for remote frame streaming
Users can now connect to remote cameras via ZeroMQ by using the new ZMQCamera client and ImageServer components. The client subscribes to a ZMQ publisher, decoding Base64-encoded JPEG frames sent in a JSON protocol, and supports synchronous, asynchronous, and latest-frame reading patterns. The accompanying ImageServer captures local camera feeds using OpenCVCamera, encodes them to Base64 JPEG, and publishes them over ZMQ, allowing remote clients to receive synchronized frame data with timestamps.
src/lerobot/cameras/zmq · high confidence
Add bi-manual OpenArm leader teleoperator support
Users can now control two OpenArm leader arms simultaneously through a new \BiOpenArmLeader\ teleoperator. This component wraps two individual \OpenArmLeader\ instances (left and right), exposing combined action features prefixed with \left\\ and \right\\ to allow synchronized bimanual input.
_src/lerobot/teleoperators/bi\_openarm\leader · high confidence
Add documentation infrastructure and validation utilities
This change introduces a new set of utility scripts to enforce documentation quality and manage doctest execution. The \check\_docstrings.py\ script validates that public object signatures match their \Args:\ blocks, ensuring parameters are documented in order and defaults are accurate, with a \--fix\_and\_overwrite\ mode to auto-correct mismatches. The \check\_config\_docstrings.py\ script specifically targets robot configuration classes, requiring documentation for critical fields like \port\ and calibration behavior to prevent user confusion. Additionally, \check\_doctest\_list.py\ manages an allowlist of files (\documentation\_tests.txt\) whose docstring examples are executed, ensuring the list remains sorted and contains only existing paths.
utils · high confidence
Add keyboard teleoperation support for robots and end-effectors
Users can now control robots and end-effectors using keyboard inputs via the new \lerobot.teleoperators.keyboard\ module. This adds three teleoperator classes: \KeyboardTeleop\ for generic key capture, \KeyboardEndEffectorTeleop\ for controlling end-effectors (with optional gripper support), and \KeyboardRoverTeleop\ for mobile robots (e.g., EarthRover Mini Plus) using WASD controls with configurable speed and turning parameters. The implementation relies on \pynput\ and includes configuration classes for each type, allowing users to record and replay data using keyboard commands.
src/lerobot/teleoperators/keyboard · high confidence
Add phone-based teleoperation support for iOS and Android
Introduces a new phone teleoperator module that enables controlling the robot via a smartphone. The implementation includes configuration for iOS and Android platforms, a processor step to map phone pose and button inputs to standardized robot actions (including axis inversions and gripper control), and the core teleoperation classes that connect to the device via the HEBI Mobile I/O app, handle calibration, and read 6-DoF pose data.
src/lerobot/teleoperators/phone · high confidence
Add support for Damiao motors via CAN bus
Users can now control Damiao series motors (such as DM3507, DM4310, DM8009, and others) using a CAN bus interface. This change introduces the \DamiaoMotorsBus\ class, which handles connection via \python-can\ (supporting both standard CAN and CAN FD), auto-detection of interfaces (socketcan or slcan), and management of motor states and dynamic gains. It also includes configuration tables for motor types, limits, and specific mappings for OpenArms joints.
src/lerobot/motors/damiao · high confidence
Add support for Hope Jr arm and hand robots
Users can now control the Hope Jr robotic arm and hand through the LeRobot framework. This change introduces the \HopeJrArm\ and \HopeJrHand\ classes, along with their respective configuration classes (\HopeJrArmConfig\, \HopeJrHandConfig\), enabling connection to Feetech motor buses, camera integration (including depth maps), and motor calibration via a GUI. The arm implementation includes a safety feature to cap relative target positions, while the hand implementation supports side-specific motor inversions for left or right configurations.
_src/lerobot/robots/hope\jr · high confidence
Add support for Reachy 2 camera integration
Users can now capture frames from Reachy 2 robot cameras (teleop and depth) using the new Reachy2Camera class. This addition provides a configuration class (Reachy2CameraConfig) to specify camera parameters such as IP address, port, resolution, and color mode, and implements the standard Camera interface for synchronous and asynchronous frame reading.
_src/lerobot/cameras/reachy2\camera · high confidence
Add support for Seeed Studio reBot Arm 102 leader teleoperator
Users can now control the follower robot using the Seeed Studio StarArm102 (reBot Arm 102) as a leader teleoperation device. This change introduces the \RebotArm102Leader\ class and its configuration (\RebotArm102LeaderConfig\, \RebotArm102LeaderTeleopConfig\) in the \src/lerobot/teleoperators/rebot\_102\_leader\ module. The implementation handles connection via UART to FashionStar smart servos, joint calibration, and reading raw positions to generate robot actions, requiring the \motorbridge-smart-servo\ package.
_src/lerobot/teleoperators/rebot\_102\leader · high confidence
Add support for bimanual Seeed Studio reBot B601 follower
Users can now control a pair of Seeed Studio reBot B601 arms simultaneously using the new \bi\_rebot\_b601\follower\ robot implementation. This change introduces a bimanual wrapper that composes two single-arm \RebotB601Follower\ instances, automatically namespacing observation and action keys with \left\\ and \right\_\ prefixes to distinguish between the two sides. The implementation includes a dedicated configuration class (\BiRebotB601FollowerConfig\) that allows specifying separate configurations for each arm while supporting shared top-level cameras that remain unprefixed in observations.
_src/lerobot/robots/bi\_rebot\_b601\follower · high confidence
Add support for the OMX Leader teleoperator
Users can now control the ROBOTIS Open Manipulator (OMX) expansion arm via the OMX Leader teleoperator. This change introduces the OmxLeader class and its configuration (OmxLeaderConfig), allowing connection to the arm's serial bus, motor calibration, and position-based action reading. The implementation supports six motors (shoulder, elbow, wrist, and gripper) using XL330-M288 and XL330-M077 servos, with specific handling for the gripper's current-based position mode to enable physical triggering.
(repo-wide) · high confidence
Add support for the ROBOTIS OMX Follower robot
Users can now control the ROBOTIS OMX Follower robot through LeRobot. This change introduces the OmxFollower class and its configuration, enabling connection to the robot's serial bus (XL430/XL330 motors), camera integration (including optional depth maps), and motor calibration. The implementation supports position control with current-limited gripper behavior and allows users to record and replay policies using this specific hardware.
_src/lerobot/robots/omx\follower · high confidence
Add support for the Reachy 2 robot
Introduces a new robot implementation for the Reachy 2 platform by Pollen Robotics. This change adds the \Reachy2Robot\ class and its configuration (\Reachy2RobotConfig\), enabling users to control the robot's joints (neck, arms, antennas, mobile base) and access its cameras (left/right teleop, torso) through the LeRobot framework. The implementation handles connection to the robot via IP address, manages camera streams using \read\_latest()\, and maps LeRobot action/observation spaces to the specific Reachy 2 SDK keys.
src/lerobot/robots/reachy2 · high confidence
Add support for the Seeed Studio reBot B601 follower arm
Users can now control the Seeed Studio reBot B601 follower arm via the new \RebotB601Follower\ robot class. This implementation supports two motor families (Damiao and RobStride) and offers flexible control modes, including MIT impedance for the arm and gripper, as well as position/velocity control. The module includes configuration profiles for joint limits, motor CAN IDs, and tuning gains, enabling users to record and play back policies with this specific hardware.
_src/lerobot/robots/rebot\_b601\follower · high confidence
Added remote inference configuration examples for SONIC and generic deployments
The examples/remote\_inference directory now includes configuration files for setting up remote inference services. This adds a generic server.yaml profile for manipulation tasks, a specific unitree\_g1\_sonic.yaml profile for the Unitree G1 robot using the SONIC model, and Zenoh transport configurations (router.json5, server.json5, robot.json5) that establish secure mTLS connections with fine-grained access control policies for the inference API.
_examples/remote\inference · high confidence
Introduce Diffusion and TD-MPC policy implementations
This change adds two new policy implementations to the library: DiffusionPolicy and TDMPCPolicy. DiffusionPolicy implements the 'Diffusion Policy: Visuomotor Policy Learning via Action Diffusion' algorithm, supporting configurable vision backbones (ResNet), image preprocessing (resize/crop), and noise schedulers (DDPM/DDIM). TDMPCPolicy implements the TD-MPC algorithm with model predictive control, including support for CEM-based planning and latent world models. Both policies include configuration classes, model implementations, and processor pipelines for data normalization and feature handling.
src/lerobot/policies/diffusion · high confidence
Introduce EO-1 multimodal policy with language supervision
Adds the EO-1 policy, a new robot control model built on the Qwen2.5-VL-3B backbone that integrates visual, state, and language inputs. This policy supports both action prediction via flow matching and text generation, allowing users to train models with dual supervision on actions and language responses. The implementation includes the configuration, model logic, and processor pipeline specific to this policy.
src/lerobot/policies/eo1 · high confidence
Introduce EVO1 vision-language policy
Adds the EVO1 policy, a new capability that integrates the InternVL3 vision-language model for robotic control. This change introduces the policy configuration, model architecture, and pre/post-processing pipeline, enabling users to load and run EVO1-based policies for tasks requiring visual and language understanding. Also in this area: 10 commits (3 fixes) modifying src/lerobot.
src/lerobot/policies/evo1 · medium confidence · unverified
Introduce FLUX 3 Action policy for joint video and action denoising
Users can now leverage the FLUX 3 Action policy, a 7B world action model that processes camera frames, robot state, and text instructions to predict the next 32 actions and 32 frames simultaneously. This new policy, located in \src/lerobot/policies/flux3\, integrates a frozen video VAE and a Qwen3-VL text encoder to perform joint video and action denoising. It supports multiple camera layouts (such as DROID, single, side-by-side, and grid) and allows for fine-tuning on new robots, simulators, or video games using the provided configuration and processor modules. Also in this area: 4 commits (1 fix) modifying src/lerobot; 12 commits (4 fixes) modifying src/lerobot.
src/lerobot/policies/flux3 · medium confidence · unverified
Introduce FastWAM \`wan\` package with custom MoT video DiT and device-agnostic adapters
This change adds the \src/lerobot/policies/fastwam/wan\ package, which implements the FastWAM policy's core model components. It introduces a custom MoT video DiT (\WanVideoDiT\) that replaces the upstream Wan2.2 flash-attention path with SDPA-based masked attention (\fastwam\_masked\_attention\) to support explicit boolean masks for video/action routing. The package provides device-agnostic adapters (\WanVideoVAE38\, \WanTextEncoder\) that wrap \diffusers.AutoencoderKLWan\ and \transformers.UMT5EncoderModel\, handling latent standardization and text encoding while automatically selecting the torch device. It also includes components for loading pretrained Wan2.2 DiT weights (sharded safetensors) and a \WanContinuousFlowMatchScheduler\ for Wan-compatible inference timesteps.
src/lerobot/policies/fastwam/wan · high confidence
Introduce FastWAM policy for video-conditioned action generation
Adds the FastWAM policy, a new capability that generates robot actions by conditioning on video frames and text prompts using a Wan2.2-based diffusion transformer. This policy supports cross-embodiment fine-tuning by safely loading checkpoints with shape mismatches, allows freezing the large video expert to reduce training memory, and includes compiled inference for faster action prediction. It also features a configurable action toggle processor for specific dimension semantics and integrates with LeRobot's standard training and evaluation pipelines.
src/lerobot/policies/fastwam · high confidence
Introduce GaussianActor policy for continuous control
Added the GaussianActor policy, a new capability for maximum-entropy continuous-control algorithms like SAC. This release includes the policy configuration, model implementation, and data processors, allowing users to train agents using a tanh-squashed diagonal Gaussian output distribution. The policy supports configurable shared or separate encoders for actor and critic networks, handles both continuous and discrete (e.g., gripper) actions, and provides default normalization settings for visual and state observations.
_src/lerobot/policies/gaussian\actor · high confidence
Introduce Hugging Face-compatible MolmoAct2 model implementation
This change adds the core Hugging Face model components for the MolmoAct2 policy, including the main \MolmoAct2\ model class, configuration classes for the vision transformer, text model, and adapter, and specialized processors for images, videos, and actions. It introduces a \UniversalActionProcessor\ that encodes/decodes action chunks using DCT and BPE tokenization, and provides inference utilities with CUDA graph support for optimized generation. This location specifically provides the model architecture, configuration, and data processing logic required to integrate MolmoAct2 with the Hugging Face Transformers ecosystem. Also in this area: 12 commits (4 fixes) modifying src/lerobot.
_src/lerobot/policies/molmoact2/molmoact2\_hf\model · medium confidence · unverified
Introduce LeKiwi robot support with host/client architecture
Adds the LeKiwi robot implementation, including local control via \LeKiwi\ (connecting to the Feetech motor bus and cameras) and remote teleoperation via \LeKiwiClient\ (communicating with a \LeKiwiHost\ over ZMQ). The change introduces configuration classes (\LeKiwiConfig\, \LeKiwiClientConfig\, \LeKiwiHostConfig\) defining camera settings (MJPG, specific resolutions/rotations), motor parameters (degrees vs range, retries, safety clamps), and network ports. It also provides the \LeKiwiHost\ script for running the local robot side and streaming observations to a remote client.
src/lerobot/robots/lekiwi · high confidence
Introduce LingBot-VA policy: autoregressive video-action world model
Adds the LingBot-VA policy, an autoregressive video-action world model built on the Wan2.2 video-diffusion stack. This policy interleaves prediction of future video latents and robot actions in a single dual-stream transformer, supporting streaming inference with a KV cache and per-step action queues. It integrates frozen sub-models (Wan VAE, UMT5 text encoder, and tokenizer) that are lazily loaded from a separate pretrained path to keep the main checkpoint small, and provides configuration for camera layouts (width\_concat or robotwin\_tshape), attention modes, and inference hyperparameters like guidance scales and flow-matching schedulers. Also in this area: 10 commits (3 fixes) modifying src/lerobot.
_src/lerobot/policies/lingbot\va · medium confidence · unverified
Introduce MolmoAct2 VLM-based robotics policy
Adds the MolmoAct2 policy, a vision-language model-based robotics controller that combines a Molmo backbone with per-layer flow-matching action experts. This release includes the policy configuration, the model implementation wrapping the Hugging Face model, and a dedicated pre/post-processing pipeline for multimodal inputs (images, state, task text). It also introduces a custom AdamW optimizer variant that performs independent gradient clipping per model component and uses low-memory BF16 update compensation to match the official training recipe.
src/lerobot/policies/molmoact2 · high confidence
Introduce Multi-Task Diffusion Transformer (DiT) policy
Adds a new Multi-Task Diffusion Transformer policy that supports both diffusion and flow matching objectives for multi-task robot learning with text and vision conditioning. The implementation includes the \MultiTaskDiTPolicy\ model, \MultiTaskDiTConfig\ for configuring transformer architecture, diffusion/flow matching parameters, and CLIP-based vision/text encoders, and a processor pipeline for normalizing inputs and unnormalizing actions.
_src/lerobot/policies/multi\_task\dit · high confidence
Introduce OpenCV camera backend with configurable rotation, FOURCC, and Windows compatibility
Users can now access cameras via a new OpenCV-based backend (OpenCVCamera) that supports configurable image rotation (including 180-degree correction), explicit FOURCC codec selection (e.g., MJPG, YUYV) to optimize frame rates, and a configurable OpenCV backend identifier. The implementation includes specific compatibility fixes for Windows systems by disabling MSMF hardware transforms and ensures proper resource cleanup and device handle release when connection setup fails.
src/lerobot/cameras/opencv · high confidence
Introduce PI0 policy with relative action support and configurable tokenization
The PI0 policy is now available in the \src/lerobot/policies/pi0\ module, providing a complete implementation including configuration (\PI0Config\), model architecture (\PI0Policy\), and processor pipelines. Users can now leverage PI0 for robotic control with support for relative actions (converting absolute actions to relative ones based on state), configurable text tokenization via the PaliGemma tokenizer, and flexible training options such as freezing the vision encoder or training only the action expert. The policy integrates with the existing processor pipeline system, handling normalization, device placement, and action unnormalization automatically.
src/lerobot/policies/pi0 · high confidence
Introduce PI0.5 policy with short-horizon memory and real-time chunking support
The PI0.5 policy is now available, featuring short-horizon visual and proprioceptive memory (MEM) to fuse historical observations into the model's context. It also supports Real-Time Chunking (RTC) for inference delay compensation and optional relative action conversion. The policy includes a dedicated configuration, processor pipeline, and memory implementation to handle these capabilities.
src/lerobot/policies/pi05 · high confidence
Introduce PI0Fast policy with relative action support and optimized decoding
A new PI0Fast policy is added to the library, providing a fast, autoregressive vision-language action model. This policy supports relative actions (converting absolute actions to deltas relative to the current state) via a configurable flag, allowing for more robust control in dynamic environments. The implementation includes a dedicated processor pipeline that handles state tokenization, language prompt construction, and action tokenization, while also optimizing inference by stopping decoding at an end-of-action marker and utilizing a KV cache for faster autoregressive generation.
_src/lerobot/policies/pi0\fast · high confidence
Introduce Real-Time Chunking (RTC) policy utilities
This change adds the \src/lerobot/policies/rtc\ package, providing the core infrastructure for Real-Time Chunking (RTC) inference. It introduces \RTCConfig\ for managing settings like execution horizon and guidance weights, \RTCProcessor\ to handle prefix guidance and denoising corrections, and \ActionQueue\ to manage action chunks and leftovers in real-time control scenarios. The package also includes \LatencyTracker\ for performance monitoring, debug tracking/visualization tools, and helpers for re-anchoring relative actions.
src/lerobot/policies/rtc · high confidence
Introduce Reward Classifier model for image-based reward prediction
Added a new reward model type, \reward\_classifier\, which uses a pre-trained image encoder (such as ResNet) to classify visual observations. This change introduces the \Classifier\ model, \RewardClassifierConfig\, and a dedicated processor pipeline in \src/lerobot/rewards/classifier\, enabling users to define reward models that process image inputs (e.g., \observation.image\) to produce classification logits.
src/lerobot/rewards/classifier · high confidence
Introduce SARM stage-aware reward model and RA-BC weighting
Adds the SARM (Stage-Aware Reward Modeling) package under src/lerobot/rewards/sarm, providing a new reward model that predicts both stage classification and within-stage progress for long-horizon robot manipulation. The implementation includes configuration (SARMConfig), model architecture (StageTransformer and SubtaskTransformer), and a processor that encodes inputs using CLIP and validates episode annotations. It also introduces RA-BC (Reward-Aligned Behavior Cloning) sample weighting via the RABCWeights class and a compute\_rabc\_weights script to precompute progress values, enabling training samples to be weighted by task progress deltas.
src/lerobot/rewards/sarm · high confidence
Introduce SO Leader teleoperator support for SO-100 and SO-101 arms
This change adds the SO Leader teleoperator implementation for the SO-100 and SO-101 robotic arms. Users can now control these devices via the new \SOLeader\, \SO100Leader\, and \SO101Leader\ classes, which handle motor communication through a Feetech bus. The implementation includes configuration options for port selection, angle units (degrees), and transient read retries to improve stability. It also provides full lifecycle management including connection, torque control, motor setup, and a guided calibration process with a visual homing diagram.
_src/lerobot/teleoperators/so\leader · high confidence
Introduce SO-100 and SO-101 follower robot support
This change adds support for the SO-100 and SO-101 follower robots, introducing the \SOFollower\, \SO100Follower\, and \SO101Follower\ classes along with their configuration schemas. Users can now control these arms via a Feetech motor bus, with configurable PID gains (P, I, D coefficients) and safety limits for relative target positions. The implementation includes a kinematic processor that translates relative delta commands into absolute end-effector poses, supports both latched and current-pose reference modes, and handles depth camera integration. It also features robust bus read retries to handle transient Feetech errors and a homing calibration process with an ASCII diagram guide.
_src/lerobot/robots/so\follower · high confidence
Introduce SmolVLA policy with VLM-based action expert and Real-Time Chunking support
Users can now train and infer with the SmolVLA policy, which integrates the SmolVLM2-500M-Video-Instruct vision-language model with a specialized action expert. This policy supports Real-Time Chunking (RTC) for low-latency inference and allows users to fine-tune only the action expert while keeping the VLM frozen, or train the full model. The implementation includes configuration for normalization, device handling, and processor pipelines for pre- and post-processing actions.
src/lerobot/policies/smolvla · high confidence
Introduce TOPReward zero-shot reward model
Added the TOPReward reward model, which uses an off-the-shelf vision-language model (default Qwen3-VL-8B-Instruct) to score video trajectories based on task instructions without fine-tuning. This change introduces the model configuration, the inference-only model class, and a processor step to encode frames and instructions into the required VLM format. It also includes a utility script to compute per-frame reward progress curves for LeRobot datasets, enabling users to evaluate trajectory quality using this zero-shot approach.
src/lerobot/rewards/topreward · high confidence
Introduce VQ-BeT policy for behavior generation
Adds the VQ-BeT (Vector Quantized Behavior Transformer) policy, implementing the 'Behavior Generation with Latent Actions' approach. This feature introduces a new policy architecture that combines a Residual VQ-VAE for action discretization with a GPT-based transformer for predicting action chunks from multi-step observations. The package includes the \VQBeTPolicy\ model, \VQBeTConfig\ for hyperparameter configuration (e.g., vision backbone, GPT layers, VQ-VAE settings), and processor pipelines for data normalization and device management. Users can now leverage this policy for visuomotor control tasks, with support for pretrained ResNet backbones and configurable action chunking.
src/lerobot/policies/vqbet · high confidence
Introduce Wall-X policy for cross-embodiment robotic control
Adds the Wall-X policy, a new capability for robotic control that leverages the Qwen2.5-VL foundation model with flow matching to predict actions from visual and language inputs. This change introduces the full policy implementation in \src/lerobot/policies/wall\_x\, including the \WallXConfig\ for configuration, \WallXPolicy\ for model inference, and processor pipelines for handling multimodal data (images, state, and text prompts). It also includes utility functions for preprocessing, constants for camera mappings, and a README symlink, enabling users to train and run Wall-X models for tasks requiring unified action representations across different robot embodiments.
_src/lerobot/policies/wall\x · high confidence
Introduce XVLA policy with multi-view vision and action-space support
Adds the XVLA (Extended Vision-Language-Action) policy to LeRobot, enabling robot control using a Florence-2 vision-language backbone combined with a soft-prompted transformer head. The policy supports multi-view image inputs, language conditioning, and proprioceptive state, with configurable freezing of vision/language encoders and training of the policy transformer and soft prompts. It includes a registry of action spaces (e.g., EE6D for end-effector pose with 6D rotation and gripper, and joint-space layouts) with dedicated loss computation and preprocessing/postprocessing hooks. Processor pipelines handle image normalization, tokenization, domain ID injection, and device placement, while configuration supports chunked action prediction, normalization modes, and optimizer/scheduler presets. Also in this area: 4 commits (1 fix) modifying src/lerobot; 10 commits (3 fixes) modifying src/lerobot.
src/lerobot/policies/xvla · medium confidence · unverified
Introduce asynchronous remote inference engine for robot policy execution
LeRobot now supports offloading policy inference to a remote server via the new \remote\_inference\ package, allowing robots to maintain uninterrupted local control while the server predicts actions asynchronously. This feature introduces a client-server architecture using the Zenoh transport for communication, with the server managing a single model deployment and the client handling observation capture, request dispatch, and action consumption. The implementation includes a wire protocol with versioning and capability negotiation to ensure compatibility between the robot and the policy server, supporting features like chunk alignment, optional blending of incoming predictions, and language generation. Operators configure the remote server with specific checkpoints and execution parameters, while clients discover and connect to ready instances, validating schema and mode compatibility before motion. Also in this area: 7 commits (0 fixes) modifying src/lerobot.
_src/lerobot/remote\inference · medium confidence · unverified
Introduce bimanual teleoperator support for OpenArm Mini and reBot 102 Leader
Users can now operate bimanual robot setups using the OpenArm Mini and the Seeed Studio reBot Arm 102 leader teleoperators. This change adds the \BiOpenArmMini\ and \BiRebot102Leader\ classes, which compose two single-arm instances and namespace their action and feedback keys with \left\\ and \right\\ prefixes. The \bi\_openarm\_mini\ variant supports bidirectional feedback, while the \bi\_rebot\_102\_leader\ is a leader-only device that does not implement feedback. Configuration is handled via new \BiOpenArmMiniConfig\ and \BiRebot102LeaderConfig\ classes, and the teleoperator factory in \utils.py\ has been updated to instantiate these new bimanual types.
src/lerobot/teleoperators · high confidence
Introduce disk-less episode-pool video streaming
Added a new \src/lerobot/streaming\ module that enables training-time dataset streaming without requiring local disk storage for video data. This includes an \EpisodeByteCache\ for bounded, in-memory caching of synthesized MP4 byte spans, an \ExactCoveragePool\ for deterministic, exactly-once frame sampling, and an \EpisodeParquetReader\ for efficient column-projection reads of episode metadata. The module also provides low-level primitives for MP4 indexing (\mp4.py\), byte-range fetching via fsspec or native HTTP (\range\_fetch.py\), and revision-safe sidecar caching (\sidecar.py\), allowing the system to stream video episodes directly from remote storage while managing memory and coverage deterministically. Also in this area: 4 commits (1 fix) modifying src/lerobot.
src/lerobot/streaming · medium confidence · unverified
Introduce distributed HILSerl training architecture
The \src/lerobot/rl\ module now provides a complete distributed Reinforcement Learning stack for Human-In-The-Loop (HILSerl) training. This includes an \actor\ server that runs the policy on the robot and collects experience, and a \learner\ server that receives transitions via gRPC to update the policy. The system features a thread-safe \ReplayBuffer\ with memory optimization options, an \OnlineOfflineMixer\ for combining data sources, and a configurable training pipeline supporting both threaded and multiprocessing concurrency. Additionally, a \crop\_dataset\_roi\ utility is included to allow users to manually define image regions of interest for dataset cropping.
src/lerobot/rl · high confidence
Introduce gamepad teleoperation with hidapi fallback for macOS
Added a new gamepad teleoperator module that allows users to control the robot via physical gamepads. The implementation includes a configuration option \hidapi\_fallback\ (defaulting to \False\) to switch from pygame to hidapi for controller detection, addressing reliability issues on macOS where pygame may fail to detect some controllers. The module exposes standard teleoperation actions (delta\_x, delta\_y, delta\_z, and optional gripper control) and supports keyboard-based input as a fallback for environments where gamepads are unavailable or for testing.
src/lerobot/teleoperators/gamepad · high confidence
Introduce placo-based RobotKinematics solver
The model module now exposes a new \RobotKinematics\ class that leverages the \placo\ library to perform forward and inverse kinematics for robots defined by URDF files. Users can initialize the solver with a URDF path and target frame, then compute end-effector poses from joint positions or solve for joint positions given a desired pose, with configurable iteration counts and position/orientation weights.
src/lerobot/model · high confidence
Introduce standardized robot base classes and factory
The \src/lerobot/robots\ module now provides a unified \Robot\ abstract base class and \RobotConfig\ dataclass, establishing a consistent interface for connecting, calibrating, and controlling supported hardware. A new \make\_robot\_from\_config\ factory function in \utils.py\ instantiates specific robot implementations (such as Koch, OMX, SO100, Hope Jr, Reachy 2, OpenArm, and ReBot B601) from configuration objects, while also including a safety utility to clamp relative action targets.
src/lerobot/robots · high confidence
Introduce steerable language annotation pipeline for LeRobot datasets
The \src/lerobot/annotations\ package now includes a steerable annotation pipeline that automatically generates structured language data for robot episodes. The pipeline runs in six phases: it decomposes episodes into subtasks, generates persistent plan and memory rows, creates mid-episode interjections with paired speech acknowledgements, and produces grounded VQA pairs (bbox, keypoint, count, attribute, spatial). It uses timestamped contact sheets to efficiently ground a VLM (defaulting to Qwen3-VL) in visual content, storing the results as \language\_persistent\ and \language\_events\ columns in the dataset's parquet shards. The system is designed to run on Hugging Face Jobs and includes configuration for task augmentation, seeded relabeling, and coordinate normalization.
src/lerobot/annotations · high confidence
Introduce unified reward model framework with factory and base classes
The \src/lerobot/rewards\ module now provides a structured foundation for reward models, introducing a \PreTrainedRewardModel\ base class that handles safetensors loading from local directories or the Hugging Face Hub, and a factory module (\factory.py\) that dynamically instantiates specific reward models (such as RewardClassifier, SARM, Robometer, and TOPReward) and their associated pre/post-processing pipelines based on configuration. This change centralizes reward model creation and device placement, allowing users to select and load reward models via string identifiers rather than direct class imports.
src/lerobot/rewards · high confidence
Introduces gRPC transport layer for async inference with retry support
Adds a new gRPC-based transport module to enable asynchronous inference, allowing robots to stream observations to a remote policy server and receive actions. The implementation includes a Protocol Buffers definition for services like AsyncInference and LearnerService, along with Python utilities for chunked binary serialization of large payloads (up to 4 MB) and automatic gRPC retry mechanisms for network resilience.
src/lerobot/transport · high confidence
Introduction of the ACT policy implementation
The Action Chunking Transformer (ACT) policy is now available for use. This change introduces the core policy components, including the \ACTConfig\ for configuration, the \ACTPolicy\ model implementation, and the \make\_act\_pre\_post\_processors\ function for data handling. Users can now leverage this policy for robot control tasks, with support for features like temporal ensembling and VAE-based latent representations as defined in the configuration.
src/lerobot/policies/act · high confidence
LeRobot datasets v3.0: new modular reader/writer architecture and streaming support
The dataset module has been refactored to version 3.0, introducing a modular architecture that splits the core \LeRobotDataset\ into dedicated \DatasetReader\ and \DatasetWriter\ components for cleaner separation of read and write logic. This update adds native support for streaming datasets from Hugging Face Storage Buckets via the new \StreamingLeRobotDataset\ and a pluggable \BaseDatasetReader\ interface. Users also gain the ability to access private Hub datasets by passing a token, and benefit from improved dataset aggregation that correctly handles multi-video datasets and heterogeneous encoder metadata.
src/lerobot/datasets · high confidence
New Docker images for benchmarking environments
Added dedicated Dockerfiles to build isolated benchmark images for LIBERO, LIBERO-plus, MetaWorld, RoboCasa, RoboCerebra, RoboMME, RoboTwin 2.0, and VLABench. Each image extends the nightly GPU base and pre-configures the specific simulator dependencies, assets, and environment registrations required to run integration tests for that benchmark. Also in this area: 1 commit (1 fix) fixing docker; 1 commit (0 fixes) modifying docker.
docker · low confidence · unverified
New LeKiwi example scripts for evaluation, recording, replay, and teleoperation
The LeKiwi example directory now includes five new Python scripts: evaluate.py for running a trained policy and recording evaluation episodes, record.py for teleoperated data collection using a leader arm or keyboard, replay.py for playing back recorded actions, rollout.py for autonomous policy execution without recording, and teleoperate.py for direct manual control. These scripts provide a complete workflow for interacting with the LeKiwi robot, including dataset creation, policy inference, and visualization via Rerun.
examples/lekiwi · high confidence
New SO-100 arm example with phone teleoperation and policy evaluation
Added a complete example suite in \examples/phone\_to\_so100\ for the SO-100 follower arm, enabling users to record datasets via phone teleoperation (\record.py\, \teleoperate.py\), evaluate trained policies (\evaluate.py\), replay recorded trajectories (\replay.py\), and deploy policies without recording (\rollout.py\). The scripts demonstrate the full pipeline: mapping phone actions to end-effector poses, converting to joint commands via inverse kinematics, and handling observations via forward kinematics, all integrated with the LeRobot dataset and policy processor systems.
_examples/phone\_to\so100 · high confidence
New SO100 end-effector examples for recording, teleoperation, and policy evaluation
Added \examples/so100\_to\_so100\_EE\ (and the parallel \examples/phone\_to\_so100\) with \record.py\, \teleoperate.py\, \evaluate.py\, \replay.py\, and \rollout.py\ scripts. These examples demonstrate how to record datasets, teleoperate, and run policies in end-effector (EE) space by using \RobotProcessorPipeline\ steps—\ForwardKinematicsJointsToEE\ and \InverseKinematicsEEToJoints\—to convert between joint and EE coordinates. The \record.py\ scripts create LeRobot datasets with features derived from these pipelines, while \evaluate.py\ and \rollout.py\ show how to load a pretrained ACT policy and apply the same kinematic conversions at inference time.
_examples/so100\_to\_so100\EE · high confidence
New backward compatibility replay example for SO arms
A new \replay.py\ script has been added to the \examples/backward\_compatibility\ directory, allowing users to replay recorded robot actions from a LeRobot dataset. This example specifically handles coordinate transformations for SO-series arms (adjusting shoulder and elbow positions) to ensure compatibility with follower robots, providing a concrete usage pattern for the \LeRobotDataset\ and \make\_robot\_from\_config\ APIs.
_examples/backward\compatibility · high confidence
New dataset example scripts for loading, transforming, and managing LeRobot datasets
Added four new example scripts in the \examples/dataset\ directory to demonstrate advanced LeRobot dataset capabilities. \load\_lerobot\_dataset.py\ shows how to inspect metadata, load specific episodes, and use timestamp-based frame selection with PyTorch DataLoaders. \use\_dataset\_image\_transforms.py\ demonstrates applying default, custom, or pure torchvision image augmentations at training time. \use\_dataset\_tools.py\ provides examples for dataset manipulation, including deleting episodes, splitting into train/val sets, adding/removing/modifying features, and merging datasets. \create\_progress\_videos.py\ enables creating MP4/GIF videos with per-frame progress overlays from dataset metadata. \slurm\_compute\_rabc.py\ introduces a distributed pipeline for computing SARM progress values across SLURM workers.
examples/dataset · high confidence
New dataset maintenance and annotation scripts
Added several new CLI scripts to the \src/lerobot/scripts\ directory to support dataset management and annotation workflows. \augment\_dataset\_quantile\_stats.py\ computes and adds quantile statistics (q01–q99) to existing LeRobot datasets that lack them, using running histograms and optional frame sampling to bound memory. \convert\_dataset\_v21\_to\_v30.py\ migrates datasets from codebase version 2.1 to 3.0, restructuring metadata and data files and optionally pushing the updated dataset to the Hub. \lerobot\_annotate.py\ runs a steerable annotation pipeline to populate \language\_persistent\ and \language\_events\ columns in dataset parquet files, with support for remote execution via Hugging Face Jobs. \lerobot\_calibrate.py\ provides a CLI to recalibrate connected robots or teleoperators. \lerobot\_convert\_dcp.py\ converts distributed checkpoint (DCP) shards into a single distributable safetensors model and optionally publishes it to the Hub. \lerobot\_dataset\_viz.py\ visualizes dataset episodes with support for local, remote streaming, and Foxglove display modes. \lerobot\_edit\_dataset.py\ offers dataset editing operations including episode deletion, splitting, merging, feature removal, task modification, stats recomputation, and image-to-video conversion. \lerobot\_eval.py\ evaluates policies on environments with configurable rollout recording and metric reporting. \lerobot\_find\_cameras.py\ detects and reports available OpenCV and RealSense cameras on the system.
src/lerobot/scripts · high confidence
New declarative configuration system for training, parallelism, and recording
The \src/lerobot/configs\ package introduces a structured, dataclass-based configuration system that replaces ad-hoc argument handling. Users can now explicitly control distributed training topology (FSDP2, HSDP, DDP, and context-parallel placeholders) via \ParallelismConfig\, manage execution runtime details like gradient accumulation and mixed-precision scaling through \AcceleratorConfig\, and enable model weight averaging with the opt-in \EMAConfig\. Data recording is now governed by \DatasetRecordConfig\, which supports streaming video encoding, configurable depth units, and optional Hub tagging. The system also standardizes policy loading via \PreTrainedConfig\ and provides dedicated configs for evaluation (\EvalPipelineConfig\) and dataset handling (\DatasetConfig\), ensuring that CLI arguments and YAML/JSON files round-trip consistently without importing heavy runtime dependencies during parsing.
src/lerobot/configs · high confidence
New distributed training runtime with FSDP2 checkpointing and environment guards
The \src/lerobot/distributed\ package introduces a new runtime layer for distributed training, centered on FSDP2 (Fully Sharded Data Parallel) support. It provides sharding-aware checkpointing via PyTorch Distributed Checkpoint (DCP) primitives, allowing model and optimizer states to be saved and loaded across topology changes. The \Accelerator\ factory now strictly manages configuration through \TrainPipelineConfig\, hard-erroring if legacy \accelerate\ environment variables interfere, unless explicitly overridden. Additionally, it includes utilities to strip conflicting context-parallel hooks from accelerate and finalize sharded policies for correct inference-style calls. Also in this area: 10 commits (3 fixes) modifying src/lerobot.
src/lerobot/distributed · medium confidence · unverified
New environment wrappers for MetaWorld, RoboCasa, RoboMME, and RoboTwin 2.0
The \src/lerobot/envs\ module now includes dedicated Gymnasium wrappers for several new robotics benchmarks. MetaWorld support is added via \metaworld.py\ and its configuration file, providing 50 manipulation tasks with pixel-based observations. RoboCasa365 kitchen environments are wrapped in \robocasa.py\, handling task resolution and action conversion for the PandaOmron robot. RoboMME evaluation is supported through \robomme.py\, wrapping the benchmark's episode environments. Additionally, \robotwin.py\ introduces a wrapper for the RoboTwin 2.0 simulator, supporting dual-arm control and official instruction generation. These additions expand the library's capability to train and evaluate policies on a wider range of standardized robotic tasks. Also in this area: 4 commits (2 fixes) modifying src/lerobot; 4 commits (1 fix) modifying src/lerobot.
src/lerobot/envs · medium confidence · unverified
New inference engine architecture with sync, RTC, and remote backends
The \src/lerobot/inference\ module introduces a new execution framework that decouples policy inference from rollout logic. It provides three distinct backends selectable via CLI: \sync\ for inline synchronous execution, \rtc\ for asynchronous Real-Time Chunking in a background thread, and \remote\ for exclusive remote deployment via Zenoh. This change adds the core contracts for asynchronous inference (including \InferenceEngine\, \PolicyQuery\, and \ActionChunk\), a factory for creating these engines, and a dispatch system that handles action interpolation, motor ticks, and motion permission holds. Also in this area: 10 commits (3 fixes) modifying src/lerobot.
src/lerobot/inference · medium confidence · unverified
New lerobot.common package for shared inference, training, and logging utilities
A new \lerobot.common\ package has been introduced to host cross-cutting modules that bridge multiple lerobot packages (policies, processor, configs). This location contributes three main utilities: \control\_utils\ provides the \predict\_action\ function for single-step robot inference (including mixed-precision support) and helpers for teleoperator handover; \train\_utils\ manages checkpoint persistence, resume logic, and training metadata; and \wandb\_utils\ handles Weights & Biases logging, including run resumption and artifact uploads. Also in this area: 10 commits (3 fixes) modifying src/lerobot; 4 commits (1 fix) modifying src/lerobot.
src/lerobot/common · medium confidence · unverified
New modular optimizer, scheduler, and scaler state management
The training pipeline now uses a dedicated \lerobot.optim\ module to centralize optimizer and learning-rate scheduler configuration and state persistence. Users can select from multiple optimizer configs (Adam, AdamW, SGD, and the new XVLAAdamW which applies differential learning rates to VLM parameters) and several scheduler strategies (CosineDecayWithWarmup, Diffuser, VQBeT, ConstantWithWarmup, and CosineAnnealingWithWarmup). Checkpointing now explicitly saves and restores optimizer, scheduler, and fp16 gradient-scaler states as JSON files, ensuring that resumed training continues with the correct learning-rate schedule and loss-scaling state.
src/lerobot/optim · high confidence
New modular rollout engine with interactive control and pluggable strategies
The \src/lerobot/rollout\ package introduces a new policy deployment engine that decouples policy execution from data recording. It provides a \RolloutController\ for programmatic lifecycle management (start, pause, reset, stop) and an \InteractiveSession\ that enables chat-style CLI control (e.g., \/start\, \/subtask\, \/vqa\) when \--interactive=true\ is used. The engine supports pluggable rollout strategies—such as \Base\, \Sentry\, \Highlight\, \Episodic\, and \DAgger\—registered via a CLI flag (\--strategy.type\), allowing users to select different recording and execution modes. It also includes an inference layer with both synchronous and real-time clock (RTC) modes, along with context management for hardware, policy, and dataset integration.
src/lerobot/rollout · high confidence
New modular rollout strategy framework with multiple execution modes
The rollout system has been refactored into a modular strategy architecture located in \src/lerobot/rollout/strategies\. This introduces a \RolloutStrategy\ abstract base class and a factory (\create\_strategy\) that dispatches to specific implementations based on configuration. Users can now select from five distinct execution modes: \Base\ (autonomous policy execution with no recording), \Sentry\ (continuous autonomous recording with automatic episode rotation and background Hub uploads), \Highlight\ (autonomous rollout with on-demand recording via a ring buffer), \Episodic\ (multi-episode recording mirroring \lerobot-record\ with configurable reset phases and optional teleoperator handover), and \DAgger\ (interactive human-in-the-loop data collection with autonomous/correction phases). The framework also supports 'Bring Your Own Strategy' via a naming convention for third-party extensions.
src/lerobot/rollout/strategies · high confidence
New motor control module with calibration GUI and bus abstraction
The \src/lerobot/motors\ package introduces a new motor control subsystem. It provides a \MotorsBusBase\ abstract interface for communicating with motors (supporting serial and CAN buses), along with concrete data models like \Motor\ and \MotorCalibration\. A new \calibration\_gui.py\ file adds a Pygame-based graphical interface for visually adjusting motor calibration ranges. Additionally, \encoding\_utils.py\ supplies helper functions for sign-magnitude and two's complement encoding/decoding, which are used by the bus implementations to handle motor data transmission.
src/lerobot/motors · high confidence
New port\_droid dataset conversion and upload pipeline
The examples/port\_datasets directory now includes a complete set of scripts to convert the DROID dataset into the LeRobot format and upload it to Hugging Face. This includes port\_droid.py, which defines the dataset schema and conversion logic; slurm\_port\_shards.py and slurm\_aggregate\_shards.py, which handle distributed processing of dataset shards and their subsequent aggregation; slurm\_upload.py, which manages the creation of the Hugging Face repository, dataset card, and chunked file uploads; and display\_error\_files.py, a utility for debugging failed worker processes during the conversion.
_examples/port\datasets · high confidence
New training examples for Diffusion and ACT policies
Added two new scripts in the training examples directory: \train\_policy.py\ demonstrates offline training of a Diffusion Policy on the PushT environment, while \train\_with\_streaming.py\ shows how to train an ACT Policy on large datasets (like DROID) using the new \StreamingLeRobotDataset\ to avoid loading all data into memory at once.
examples/training · high confidence
New tutorial examples for ACT, Diffusion, Pi0, SmolVLA, and HIL-SERL policies
Added tutorial scripts in the \examples/tutorial\ directory demonstrating how to train and use several robot learning policies. This includes training and inference examples for ACT and Diffusion policies, as well as inference examples for Pi0, SmolVLA, and HIL-SERL (using GaussianActor and a reward classifier). These examples show how to configure the SO100Follower robot, handle camera inputs, and run offline training or online inference loops.
examples/tutorial · high confidence
New utility modules for action interpolation, cadence pacing, and evaluation statistics
The \src/lerobot/utils\ package now includes \ActionInterpolator\ for smoothing robot control by interpolating between policy actions, \CycleTimer\ for pacing and reporting timing in real-time control loops, and \eval\_stats\ for computing Wilson score intervals and Fisher's exact tests on evaluation success rates. These additions support smoother motion, precise cadence monitoring, and statistically robust evaluation reporting.
src/lerobot/utils · high confidence
Repository initialization with comprehensive development and security tooling
The repository is initialized with a complete set of configuration files to standardize development workflows and ensure code quality. This includes a \.pre-commit-config.yaml\ that enforces formatting with Ruff, checks for typos, and runs security scans via Bandit, gitleaks, and zizmor. A \.dockerignore\ file is added to optimize Docker builds by excluding local environments, caches, and sensitive data, while \.gitattributes\ configures Git LFS for large binary assets like models and videos. Additionally, new documentation files (\AGENTS.md\, \AGENT\_GUIDE.md\, \AI\_POLICY.md\) provide structured guidance for contributors and AI agents, and a \Makefile\ defines targets for end-to-end testing of policies like ACT and Diffusion.
(repo-wide) · high confidence
Run training and annotation jobs on HuggingFace GPUs
Users can now execute \lerobot-train\ and \lerobot-annotate\ remotely on HuggingFace Jobs infrastructure by passing \--job.target\. This new capability automatically handles dataset availability by pushing local-only datasets to private Hub repositories, constructs the necessary pod setup and commands (including installing dependencies like vLLM for annotation), and streams job logs to the local terminal. The implementation ensures that client-specific arguments are filtered out and that results are pushed back to the Hub when configured. Also in this area: 10 commits (3 fixes) modifying src/lerobot; 4 commits (1 fix) modifying src/lerobot.
src/lerobot/jobs · medium confidence · unverified
Unified policy factory and base class for LeRobot policies
The \src/lerobot/policies\ module now provides a centralized entry point and factory system for all supported policies. Users can discover available models via the new \\_\init\\_.py\ exports (e.g., \PI0Config\, \DiffusionConfig\, \GrootConfig\) and instantiate them programmatically using \make\_policy\ and \get\_policy\_class\ from \factory.py\. This system supports lazy loading of heavy optional dependencies and enables third-party policy registration. A new \PreTrainedPolicy\ base class standardizes model loading, saving, and device handling, while utility functions like \prepare\_observation\_for\_inference\ and \make\_robot\_action\ simplify the conversion of raw environment data into model-ready tensors and robot commands.
src/lerobot/policies · high confidence
Architecture
Introduce unified processor pipeline and step registry
The \src/lerobot/processor\ module has been restructured to provide a unified, composable pipeline for transforming environment transitions and policy actions. This change introduces a \ProcessorStepRegistry\ and a \DataProcessorPipeline\ class, allowing users to build custom data processing chains by registering and ordering specific steps. The module now includes dedicated processor steps for common operations such as adding batch dimensions, moving data to specific devices (CPU/GPU) with dtype casting, normalizing/unnormalizing features, and converting between policy actions, robot actions, and environment transitions. Environment-specific processors for LIBERO and IsaacLab Arena are also included to handle their unique observation formats, alongside helpers for human-in-the-loop teleoperation and delta-action mapping.
src/lerobot/processor · high confidence
Behavioural changes
1 commit (0 fixes) modifying src/lerobot/robots/bi\_yam\_follower, src/lerobot/robots/yam\_follower
A change to existing behaviour in src/lerobot/robots/bi\_yam\_follower, src/lerobot/robots/yam\_follower — 1 commit, 7 files: “feat(robot): yam robot arm with motorbridge (\#4855)” (refs \#4855).
_src/lerobot/robots/bi\_yam\_follower, src/lerobot/robots/yam\follower · medium confidence · unverified
18 commits (6 fixes) modifying src/lerobot/policies/vla\_jepa
A change to existing behaviour in src/lerobot/policies/vla\_jepa — 18 commits (6 fixs), 8 files: “feat(vla\_jepa): opt-in world\_model\_rope\_legacy for checkpoints trained before \#4835 (\#5098)”; “refactor(vla-jepa): removing gpu roundtrip (\#3750)”; “feat(policies): add VLA-JEPA (\#3568)”; and 15 more (refs \#4835, \#5098, \#3750, \#3568, \#4421, …).
_src/lerobot/policies/vla\jepa · medium confidence · unverified
2 commits (0 fixes) modifying scripts
A change to existing behaviour in scripts — 2 commits, 6 files: “feat(ci): benchmark smoke tests with isolated Docker images (LIBERO + MetaWorld) (\#3319)”; “feat(sim): one simulator core for batched envs and sim robots, in-process or served (\#4888)” (refs \#3319, \#4888).
scripts · medium confidence · unverified
2 commits (0 fixes) modifying src/lerobot/policies/lawam/lam\_core
A change to existing behaviour in src/lerobot/policies/lawam/lam\_core — 2 commits, 8 files: “feat(policies): add LaWAM policy (\#3999)”; “chore(mypy): cover policies (\#4763)” (refs \#3999, \#4763).
_src/lerobot/policies/lawam/lam\core · medium confidence · unverified
2 commits (0 fixes) modifying src/lerobot/robots/sim\_robot
A change to existing behaviour in src/lerobot/robots/sim\_robot — 2 commits, 3 files: “refactor(envs): make EnvConfig options only, build envs in the factory (\#5338)”; “feat(sim): one simulator core for batched envs and sim robots, in-process or served (\#4888)” (refs \#5338, \#4888).
_src/lerobot/robots/sim\robot · medium confidence · unverified
4 commits (1 fix) modifying src/lerobot/policies/fineart\_vla
A change to existing behaviour in src/lerobot/policies/fineart\_vla — 4 commits (1 fix), 6 files: “feat: add FineART-VLA policy built on pi0.5 (\#4184)”; “Bump transformers version to 5.22 (\#5220)”; “feat(policies): accept the starting noise in predict\_action\_chunk (\#4813)”; and 1 more (refs \#4184, \#5220, \#4813, \#5224).
_src/lerobot/policies/fineart\_vla, src/lerobot/policies/longwam, src/lerobot/policies/wall\_oss\05 · medium confidence · unverified
7 commits (0 fixes) modifying src/lerobot
A change to existing behaviour in src/lerobot — 7 commits, 46 files: “feat(train): configurable TF32 for training and evaluation (\#5096)”; “feat(optim): add the warmup\_stable\_decay scheduler (\#5103)”; “perf(groot): keep N1.7 camera frames on device during preprocessing (\#5112)”; and 4 more (refs \#5096, \#5103, \#5112, \#5117, \#5156, …). Also in this area: 4 commits (1 fix) modifying src/lerobot; Add DM05 policy support.
(repo-wide) · medium confidence · unverified
Deprecate FeetechMotorsBus in favor of unified SerialMotorsBus
The \FeetechMotorsBus\ class is now deprecated and will be removed in a future release. Users should switch to \SerialMotorsBus\ from \lerobot.motors\, which automatically detects the motor family (including Feetech) based on the motor model. The legacy \FeetechMotorsBus\ now emits a deprecation warning upon initialization, and its \protocol\_version\ argument is ignored.
src/lerobot/motors/feetech · high confidence
Deprecation of DynamixelMotorsBus in favor of unified SerialMotorsBus
The \DynamixelMotorsBus\ class is now deprecated and emits a warning when instantiated; users should switch to the unified \SerialMotorsBus\ from \lerobot.motors\, which automatically detects the motor family (including Dynamixel) based on the model. The legacy \OperatingMode\ and \TorqueMode\ enums remain available for one release to ensure backward compatibility but are also deprecated.
src/lerobot/motors/dynamixel · high confidence
Image transforms now operate on batches with backward-compatible deprecation wrappers
The \lerobot.transforms\ module has been restructured so that all image augmentations (such as \BatchedColorJitter\, \BatchedGaussianNoise\, and \BatchedRandomAffine\) now process batches of frames \(B, N, C, H, W)\ directly, enabling efficient GPU execution. To maintain compatibility with existing code, the previous per-sample transform names (e.g., \GaussianNoise\, \MotionBlur\) are retained in a \deprecated\ module; they now act as thin wrappers that add a batch dimension, invoke the new batched logic, and emit a \FutureWarning\ before being removed in a future release.
src/lerobot/transforms · high confidence
Introduces structured type definitions for environment transitions and robot actions
The library now provides explicit type definitions in \lerobot\_types.py\ to standardize how environment transitions and robot interactions are represented. Users can now rely on the \EnvTransition\ TypedDict, which strictly defines the structure of a single environment step (including observation, action, reward, done, truncated, info, and complementary\_data fields), and the \TransitionKey\ class for accessing these fields. Additionally, specific type aliases like \PolicyAction\, \RobotAction\, and \EnvAction\ are introduced to clarify the expected data types for different action sources, improving type safety and IDE support for developers building policies or interacting with the simulation environment.
src/lerobot · high confidence
New modular camera subsystem with configurable backends
The \src/lerobot/cameras\ package has been restructured into a new modular subsystem that standardizes camera access across different hardware. Users can now configure cameras via a unified \CameraConfig\ interface, supporting multiple backends including OpenCV, Intel RealSense, Reachy 2, and ZMQ. The system introduces a \read\_latest()\ method for non-blocking frame retrieval (useful for UI or logging) alongside the existing synchronous and asynchronous read methods, and allows specifying OpenCV-specific settings like color mode and rotation directly in the configuration.
src/lerobot/cameras · high confidence
Refactored RL algorithm stack with new base classes and SAC implementation
The reinforcement learning algorithm module has been restructured to support a pluggable algorithm architecture. A new abstract base class (RLAlgorithm) and configuration registry (RLAlgorithmConfig) define the interface for algorithm training, state management, and serialization. The Soft Actor-Critic (SAC) algorithm is now implemented as a concrete subclass of this base, with its own dedicated configuration class (SACAlgorithmConfig) and factory functions to instantiate algorithms by name. This change introduces a standardized way to load, save, and extend RL algorithms within the Lerobot framework.
src/lerobot/rl/algorithms · high confidence
Shared flow-matching and VLA utility primitives
Extracted common flow-matching sampling primitives (timestep and noise samplers, Euler integration, and training-input construction) and VLA helper utilities (sinusoidal positional embeddings, attention mask preparation, and image resizing) into new shared modules in \src/lerobot/policies/common\. These changes replace policy-specific copy-pasted implementations with canonical, stateless functions, ensuring that existing checkpoints remain unaffected while providing a unified foundation for current and future policies.
src/lerobot/policies/common · high confidence
Wall-X model now uses native Transformers Qwen2.5-VL with custom MoE and optimized vision attention
The Wall-X policy no longer vendors a copy of the Qwen2.5-VL model; instead, it subclasses the native Hugging Face Transformers implementation to add Wall-X-specific features. This includes a hard-routed Mixture-of-Experts (MoE) for the text decoder layers, which routes tokens based on type and supports mixed-precision execution (float32 layernorms with bfloat16 projections). Additionally, the vision attention path now offers a packed \torch.nn.attention.varlen\ backend for improved performance when batching multiple camera frames, falling back to standard SDPA if the environment (PyTorch version, CUDA device, or dtype) does not support it. The configuration layer also maintains backward compatibility with legacy flat checkpoint layouts by forwarding attribute access to the nested text config.
_src/lerobot/policies/wall\_x/qwen\model · high confidence
Test coverage
1 commit adding/updating tests in tests/policies/lingbot\_vla\_v2; 2 commits adding/updating tests in tests/policies/praxis; 9 commits adding/updating tests in tests/policies/flux3; 9 commits adding/updating tests in tests/remote\_inference; Add test suite for VLA-JEPA policy; Added behavior-pinning tests for shared flow-matching and VLA utilities; Added comprehensive test coverage for dataset operations; Added comprehensive test coverage for the new data processor pipeline; Added comprehensive tests for training configuration validation and plugin loading; Added distributed training test coverage; Added integration tests for the DM05 policy; Added mock hardware and robot components for testing; Added smoke tests for the EO1 policy interface; Added test artifact generation script for policy policies; Added test artifacts and utility script for dataset backward compatibility; Added test artifacts for image transform configurations; Added test artifacts for pusht\_diffusion policy; Added test artifacts for xarm lift medium TDMPC policies; Added test coverage for Hugging Face Jobs orchestration; Added test coverage for Real-Time Chunking (RTC) components; Added test coverage for dataset editing, training, evaluation, and annotation scripts; Added test coverage for motor bus implementations; Added test coverage for multiple robot implementations; Added test suite for the language annotation pipeline; Added tests for EMA training, multi-GPU acceleration, and visual feature validation; Added tests for Multi-Task DiT policy; Added tests for PI0, PI0.5, and FineART-VLA policies; Added tests for PI0Fast policy integration; Added tests for RL actor-learner communication, data mixing, and training infrastructure; Added tests for Reachy 2, Rebot 102, and Unitree G1 teleoperators; Added tests for XVLA policy integration with LeRobot; Added tests for environment configuration, dispatch, and wrapper logic; Added tests for optimizer, scheduler, and gradient scaler state persistence; Added tests for reward model infrastructure and specific reward models; Added tests for the FastWAM policy implementation; Added tests for transport utility functions; Added unit tests for LingBot-VA policy configuration, factory, modules, and processor; Added unit tests for MolmoAct2 policy; Added unit tests for OpenCV, RealSense, and Reachy2 camera implementations; Added unit tests for the EVO1 policy implementation; Added unit tests for utility modules; Expanded test coverage for policy factory, configuration, and inference contracts; Initial test suite and infrastructure; Initialize model test package; New test fixtures for cadence, datasets, and policies.
Dependencies
LeRobot 0.6.2 dependency specification
The project now specifies its dependencies in pyproject.toml for version 0.6.2, requiring Python 3.12+ and pinning core libraries such as torch (\>=2.7,\<2.12.0), torchvision (\>=0.22.0,\<0.27.0), and transformers (\>=5.4.0,\<5.6.0). It introduces structured optional dependency groups for datasets, training, hardware, and visualization, while explicitly constraining placo (\<0.9.16) and cmeel packages to resolve ABI compatibility issues on Ubuntu 24.04.
(dependencies) · high confidence
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
How this codebase got here
Score
- CAI 67 → 63 (-3.6)
- Rubric changed (rubric-2026.11.2 → rubric-2026.11.4) — scores are not directly comparable.
Lenses
- Code Health 65 → 61 (-4.7)
- Architecture 86 → 99 (+13.1)
- Maturity 75 → 68 (-6.9)
- Readiness 62 → 57 (-4.3)
- Security 79 → 80 (+1.0)
- Performance 80 (new)
Resolved (80)
- Dependency hygiene PARTLY measured — Python dependencies read, no exact pin to grade for currency
- Duplicated block (10–11 lines × 2) (src/lerobot/policies/eo1/configuration_eo1.py)
- Duplicated block (11 lines × 2) (src/lerobot/policies/xvla/processor_xvla.py)
- Duplicated block (11 lines × 2) (src/lerobot/robots/earthrover_mini_plus/robot_earthrover_mini_plus.py)
- Duplicated block (11–12 lines × 2) (src/lerobot/envs/metaworld.py)
- Duplicated block (11–13 lines × 2) (src/lerobot/policies/lawam/vlas/flowmatching_expert.py)
- Duplicated block (14–15 lines × 2) (src/lerobot/envs/robocasa.py)
- Duplicated block (14–15 lines × 2) (src/lerobot/motors/damiao/damiao.py)
- Duplicated block (14–15 lines × 3) (src/lerobot/envs/libero.py)
- Duplicated block (15 lines × 2) (src/lerobot/policies/smolvla/modeling_smolvla.py)
- Duplicated block (15 lines × 2) (src/lerobot/teleoperators/homunculus/homunculus_arm.py)
- Duplicated block (17–18 lines × 2) (src/lerobot/envs/robocasa.py)
- Duplicated block (18 lines × 2) (src/lerobot/rl/actor.py)
- Duplicated block (19–20 lines × 2) (src/lerobot/envs/metaworld.py)
- Duplicated block (20 lines × 2) (src/lerobot/teleoperators/homunculus/homunculus_arm.py)
- Duplicated block (21–22 lines × 2) (examples/dataset/create_progress_videos.py)
- Duplicated block (5 lines × 2) (src/lerobot/policies/fastwam/wan/video_dit.py)
- Duplicated block (5 lines × 2) (src/lerobot/rewards/sarm/compute_rabc_weights.py)
- Duplicated block (5–9 lines × 2) (src/lerobot/policies/molmoact2/molmoact2_hf_model/modeling_molmoact2.py)
- Duplicated block (6 lines × 2) (src/lerobot/envs/configs.py)
- …and 60 more
New (298)
- ActionExpert.forward_trunk_rope (cognitive 23) (src/lerobot/policies/praxis/praxis_core/action_expert.py)
- ActionExpert.forward_trunk_rope (cyclomatic 16) (src/lerobot/policies/praxis/praxis_core/action_expert.py)
- BiYamFollower.connect (cognitive 19) (src/lerobot/robots/bi_yam_follower/bi_yam_follower.py)
- BimanualServeSession.prepare (cognitive 17) (src/lerobot/policies/praxis/praxis_core/bimanual.py)
- Change coupling: lerobot_calibrate.py ↔ lerobot_find_joint_limits.py (src/lerobot/scripts/lerobot_calibrate.py)
- Change coupling: lerobot_calibrate.py ↔ lerobot_replay.py (src/lerobot/scripts/lerobot_calibrate.py)
- Duplicated block (10 lines × 2) (src/lerobot/policies/praxis/praxis_core/bimanual.py)
- Duplicated block (10 lines × 2) (src/lerobot/policies/praxis/praxis_core/kernels/layers_triton.py)
- Duplicated block (11 lines × 2) (src/lerobot/policies/praxis/praxis_core/backbone.py)
- Duplicated block (12–13 lines × 2) (src/lerobot/policies/praxis/praxis_core/bimanual.py)
- Duplicated block (14 lines × 2) (src/lerobot/policies/praxis/praxis_core/bimanual.py)
- Duplicated block (14 lines × 2) (src/lerobot/policies/praxis/praxis_core/bimanual.py)
- Duplicated block (14–17 lines × 2) (src/lerobot/sim/integrations/metaworld.py)
- Duplicated block (5 lines × 2) (examples/praxis/convert_checkpoint.py)
- Duplicated block (5 lines × 2) (src/lerobot/policies/praxis/modeling_praxis.py)
- Duplicated block (5 lines × 2) (src/lerobot/policies/praxis/praxis_core/kernels/layers_triton.py)
- Duplicated block (5 lines × 2) (src/lerobot/policies/praxis/praxis_core/robot_io.py)
- Duplicated block (6 lines × 2) (src/lerobot/policies/praxis/praxis_core/bimanual.py)
- Duplicated block (6 lines × 2) (src/lerobot/policies/praxis/praxis_core/bimanual.py)
- Duplicated block (6 lines × 3) (src/lerobot/rl/actor.py)
- …and 278 more
Changes since last survey
- 27 commits — 17 feature/other, 10 fixes
By area
- src/lerobot — 13 commits
- (root) — 7 commits
- docs/source — 3 commits
- docker/Dockerfile.sim — 1 commit
- docker/sim — 1 commit
- tests/policies — 1 commit
- tests/robots — 1 commit
Notable commits
- fix: fix(datasets): escape video paths in concat manifests (#5168)
- fix: fix(dependencies): pillow bump (#5263)
- fix: fix(deps): relax lancedb upper bound to 1.0.0 (#5341)
- fix: fix(docker): resolve robosuite with RoboCasa's MuJoCo pin (#5293)
- fix: fix(processor): refuse rename_map collisions instead of silently dropping a key (#5316)
- fix: fix(tests): checkpoint resolution molmoact2 (#5322)
- fix: fix(utils): preserve small rotation vectors (#5159)
- fix: fix(xvla): apply the LIBERO env processors to LIBERO only and check the action conversion (#5084)
- fix: refactor(g1): clarify controller naming and fix stale exo joystick input (#4990)
- fix: test(yam): stamp fake CAN frames when received to fix flaky feedback tests (#5265)
- change: chore(dependencies): Bump lerobot to 0.7.0 (#5268)
- change: chore(dependencies): Bump lerobot to 0.7.1 (#5269)
- change: chore(dependencies): update uv.lock (#5238)
- change: chore(dependencies): update uv.lock (#5267)
- change: chore(dependencies): update uv.lock (#5271)
- change: chore: fix typo in pyproject.toml comment (#5275)
- change: docs(datasets): make the v3 loading example runnable (#5303)
- change: docs(praxis): refresh stale statements; no internal path in the base fixture (#5325)
- change: docs: add RoboColosseum benchmark guide (#5274)
- change: feat(dataset): support storage options for remote Lance datasets (#4794)
- …and 7 more
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
Survey your own repository
huggingface/lerobot was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.
About this page
- The score is its most recent published measurement, taken on 16 November 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
- Measured at commit fe1c54594f6ba275fccf210b9f7b11f0e2324b89 — the exact code this score is about.
- Scored under rubric-2026.11.4 — the same rubric and the same method as every other entry in this index.
- Measured by Canine Code Analyzer (cca.canine.dev) using cca measure-2026.11.73+preprod-52de90b74216.