具身开发者社区

外观

教程huggingface.co6 天前

LeRobotDataset v3：目录结构、meta/info.json、加载与转换实操 ​

本文按 LeRobotDataset v3.0 文档整理其目录结构、meta/info.json 作用、训练加载、Hub 流式读取、图像增强、v2.1 到 v3.0 转换，以及 finalize() 常见注意事项。

AI 整理
  • 数据与采集

> 本文依据以下官方文档整理：huggingface.co/docs/lerobot/lerobot-dataset-v3。命令与版本以文档当前版本为准。

前置条件 ​

  • LeRobotDataset v3.0 will be included in lerobot >= 0.4.0
  • Until that stable release, you can use the main branch by following the build from source instructions
  • 用于录制示例的 SO-101：so101_follower 与 so101_leader
  • 用于训练加载示例的 PyTorch 与 torch.utils.data.DataLoader
  • 可访问 Hugging Face Hub dataset repo 或 HF Storage Bucket

1. 理解 v3.0 的文件化存储设计 ​

LeRobotDataset v3.0 将存储与用户 API 解耦：底层使用较少的大文件，高层仍提供直观的 episode 级访问。Tabular data 存在 Apache Parquet；Visual data 将相机帧拼接并编码进 MP4；Metadata 用 JSON/Parquet 描述 schema、FPS、统计信息和 episode 分段。

2. 按目录结构定位 meta/info.json 与数据分片 ​

简化目录中，meta/info.json 是 canonical schema，包含 features、shapes/dtypes、FPS、codebase version，以及定位 data/video shards 的 path templates。meta/stats.json 暴露为 dataset.meta.stats；meta/tasks.jsonl 映射自然语言任务与整数 ID；meta/episodes/ 保存每个 episode 的长度、任务和 offsets；data/ 是 Parquet shards，videos/ 是各 camera 的 MP4 shards。

3. 录制 v3.0 数据集并推送到 Hub ​

文档示例使用 SO-101 录制数据集，并通过 --dataset.repo_id=${HF_USER}/record-test 推送到 Hub。命令中还设置了相机、任务、episode 数、streaming encoding 与 encoder threads。

bash
lerobot-record \
  --robot.type=so101_follower \
  --robot.port=/dev/tty.usbmodem585A0076841 \
  --robot.id=my_awesome_follower_arm \
  --robot.cameras="{ front: {type: opencv, index_or_path: 0, width: 1920, height: 1080, fps: 30}}" \
  --teleop.type=so101_leader \
  --teleop.port=/dev/tty.usbmodem58760431551 \
  --teleop.id=my_awesome_leader_arm \
  --display_data=true \
  --dataset.repo_id=${HF_USER}/record-test \
  --dataset.num_episodes=5 \
  --dataset.single_task="Grab the black cube" \
  --dataset.streaming_encoding=true \
  # --dataset.rgb_encoder.vcodec=auto \
  --dataset.encoder_threads=2

4. 用 LeRobotDataset 加载训练数据 ​

LeRobotDataset 返回 PyTorch tensors 的 Python dictionaries，并能接入 torch.utils.data.DataLoader。示例先从 Hub 加载 yaak-ai/L2D-v3，再按索引随机访问，并用 delta_timestamps 为指定图像 key 取时间窗口。

python
import torch
from lerobot.datasets import LeRobotDataset

repo_id = "yaak-ai/L2D-v3"

## 1) Load from the Hub (cached locally)
dataset = LeRobotDataset(repo_id)

## 2) Random access by index
sample = dataset[100]
print(sample)
## {
##   'observation.state': tensor([...]),
##   'action': tensor([...]),
##   'observation.images.front_left': tensor([C, H, W]),
##   'timestamp': tensor(1.234),
##   ...
## }

## 3) Temporal windows via delta_timestamps (seconds relative to t)
delta_timestamps = {
    "observation.images.front_left": [-0.2, -0.1, 0.0]  # 0.2s and 0.1s before current frame
}

dataset = LeRobotDataset(repo_id, delta_timestamps=delta_timestamps)

## Accessing an index now returns a stack for the specified key(s)
sample = dataset[100]
print(sample["observation.images.front_left"].shape)  # [T, C, H, W], where T=3

## 4) Wrap with a DataLoader for training
batch_size = 16
data_loader = torch.utils.data.DataLoader(dataset, batch_size=batch_size)

device = "cuda" if torch.cuda.is_available() else "cpu"
for batch in data_loader:
    observations = batch["observation.state"].to(device)
    actions = batch["action"].to(device)
    images = batch["observation.images.front_left"].to(device)
    # model.forward(batch)

5. 使用 StreamingLeRobotDataset 直接流式读取 ​

StreamingLeRobotDataset 可直接从 Hub 迭代数据，不需要下载本地副本，也不需要把数据加载到内存。HF Storage Bucket 的 hf://buckets/ 数据也可用同样方式读取，只需传入 repo_type="bucket"。

python
from lerobot.datasets import StreamingLeRobotDataset

repo_id = "yaak-ai/L2D-v3"
dataset = StreamingLeRobotDataset(repo_id)  # streams directly from the Hub
python
dataset = StreamingLeRobotDataset("my-org/my-bucket", repo_type="bucket")
bash
lerobot-train \
  --dataset.repo_id=my-org/my-bucket \
  --dataset.repo_type=bucket \
  --dataset.streaming=true \
  ...

6. 为训练加载添加图像 transforms ​

Image transforms 只在训练时应用，不在 recording 时应用；录制数据时保存 raw images。加载数据集时可通过 image_transforms 传入默认配置、自定义配置，或直接传入 torchvision.transforms.v2 transform。

python
from lerobot.datasets import LeRobotDataset
from lerobot.transforms import ImageTransforms, ImageTransformsConfig, ImageTransformConfig

## Option 1: Use default transform configuration (disabled by default)
transforms_config = ImageTransformsConfig(
    enable=True,  # Enable transforms
    max_num_transforms=3,  # Apply up to 3 transforms per frame
    random_order=False,  # Apply in standard order
)
transforms = ImageTransforms(transforms_config)

dataset = LeRobotDataset(
    repo_id="your-username/your-dataset",
    image_transforms=transforms
)

## Option 2: Create custom transform configuration
custom_transforms_config = ImageTransformsConfig(
    enable=True,
    max_num_transforms=2,
    random_order=True,
    tfs={
        "brightness": ImageTransformConfig(
            weight=1.0,
            type="ColorJitter",
            kwargs={"brightness": (0.7, 1.3)}  # Adjust brightness range
        ),
        "contrast": ImageTransformConfig(
            weight=2.0,  # Higher weight = more likely to be selected
            type="ColorJitter",
            kwargs={"contrast": (0.8, 1.2)}
        ),
        "sharpness": ImageTransformConfig(
            weight=0.5,  # Lower weight = less likely to be selected
            type="SharpnessJitter",
            kwargs={"sharpness": (0.3, 2.0)}
        ),
    }
)

dataset = LeRobotDataset(
    repo_id="your-username/your-dataset",
    image_transforms=ImageTransforms(custom_transforms_config)
)

## Option 3: Use pure torchvision transforms
from torchvision.transforms import v2

torchvision_transforms = v2.Compose([
    v2.ColorJitter(brightness=0.2, contrast=0.2, saturation=0.2, hue=0.1),
    v2.GaussianBlur(kernel_size=3, sigma=(0.1, 2.0)),
])

dataset = LeRobotDataset(
    repo_id="your-username/your-dataset",
    image_transforms=torchvision_transforms
)

7. 预览 transforms 效果 ​

调参前可用 visualization script 预览增强效果。该命令会保存 example images，展示 each transform 对数据的影响，帮助调整参数。文档建议先保守设置、小范围开始，并确认 transforms 看起来合理。

bash
lerobot-imgtransform-viz \
  --repo-id=your-username/your-dataset \
  --output-dir=./transform_examples \
  --n-examples=5

8. 将 v2.1 数据集转换为 v3.0 ​

转换器会把 per-episode files 聚合为更大的 shards，并写入 episode offsets/metadata。文档中的转换先安装带 v3 支持的 pre-release build，再对 Hub 上已有 v2.1 数据集运行转换命令。

bash
## Pre-release build with v3 support:
pip install "https://github.com/huggingface/lerobot/archive/33cad37054c2b594ceba57463e8f11ee374fa93c.zip"

## Convert an existing v2.1 dataset hosted on the Hub:
python -m lerobot.scripts.convert_dataset_v21_to_v30 --repo-id=<HF_USER/DATASET_ID>

验证是否跑通 ​

加载时可执行示例中的 sample = dataset[100] 并 print(sample)；使用 delta_timestamps 后，print(sample["observation.images.front_left"].shape) 应显示 [T, C, H, W]，其中 T=3。流式读取时能创建 StreamingLeRobotDataset(repo_id)；转换后应完成 parquet/mp4 聚合并更新 meta/episodes/*。

文档里点名的注意事项 ​

  • When creating or recording datasets, you must call dataset.finalize() before push_to_hub().
  • Without calling finalize(), parquet files will be incomplete and the dataset won’t load properly.
  • Dataset v3.0 uses incremental parquet writing with buffered metadata for efficiency; finalize() flushes buffered episode metadata, closes parquet writers, and ensures the dataset is valid for loading.
  • Transforms are applied during training time only, not during recording.
  • ImageTransformsConfig is disabled by default: enable default is False.
  • Strong augmentations can hurt performance if too aggressive; using too many transforms simultaneously can make training unstable.

相关 SDK 条目 ​

来源

本页事实对应的公开页面，以原文为准。

  1. huggingface.co/docs/lerobot/lerobot-dataset-v3访问 2026-10-22