# LeRobotDataset v3：目录结构、meta/info.json、加载与转换实操

> 原文链接：https://robotics.hellobike.com/tutorials/lerobot-dataset-v3-format
> 频道：tutorials | 更新日期：2026-10-22 | 标签：LeRobotDataset v3.0, Hugging Face Hub, meta/info.json, StreamingLeRobotDataset, ImageTransforms, v2.1 to v3.0
> 核心摘要：本文按 LeRobotDataset v3.0 文档整理其目录结构、meta/info.json 作用、训练加载、Hub 流式读取、图像增强、v2.1 到 v3.0 转换，以及 finalize() 常见注意事项。

# LeRobotDataset v3：目录结构、meta/info.json、加载与转换实操

> 本文依据以下官方文档整理：[huggingface.co/docs/lerobot/lerobot-dataset-v3](https://huggingface.co/docs/lerobot/lerobot-dataset-v3)。命令与版本以文档当前版本为准。

## 前置条件

- LeRobotDataset v3.0 will be included in lerobot >= 0.4.0
- Until that stable release, you can use the main branch by following the build from source instructions
- 用于录制示例的 SO-101：so101_follower 与 so101_leader
- 用于训练加载示例的 PyTorch 与 torch.utils.data.DataLoader
- 可访问 Hugging Face Hub dataset repo 或 HF Storage Bucket

## 1. 理解 v3.0 的文件化存储设计

LeRobotDataset v3.0 将存储与用户 API 解耦：底层使用较少的大文件，高层仍提供直观的 episode 级访问。Tabular data 存在 Apache Parquet；Visual data 将相机帧拼接并编码进 MP4；Metadata 用 JSON/Parquet 描述 schema、FPS、统计信息和 episode 分段。

## 2. 按目录结构定位 meta/info.json 与数据分片

简化目录中，`meta/info.json` 是 canonical schema，包含 features、shapes/dtypes、FPS、codebase version，以及定位 data/video shards 的 path templates。`meta/stats.json` 暴露为 `dataset.meta.stats`；`meta/tasks.jsonl` 映射自然语言任务与整数 ID；`meta/episodes/` 保存每个 episode 的长度、任务和 offsets；`data/` 是 Parquet shards，`videos/` 是各 camera 的 MP4 shards。

## 3. 录制 v3.0 数据集并推送到 Hub

文档示例使用 SO-101 录制数据集，并通过 `--dataset.repo_id=${HF_USER}/record-test` 推送到 Hub。命令中还设置了相机、任务、episode 数、streaming encoding 与 encoder threads。

```bash
lerobot-record \
  --robot.type=so101_follower \
  --robot.port=/dev/tty.usbmodem585A0076841 \
  --robot.id=my_awesome_follower_arm \
  --robot.cameras="{ front: {type: opencv, index_or_path: 0, width: 1920, height: 1080, fps: 30}}" \
  --teleop.type=so101_leader \
  --teleop.port=/dev/tty.usbmodem58760431551 \
  --teleop.id=my_awesome_leader_arm \
  --display_data=true \
  --dataset.repo_id=${HF_USER}/record-test \
  --dataset.num_episodes=5 \
  --dataset.single_task="Grab the black cube" \
  --dataset.streaming_encoding=true \
  # --dataset.rgb_encoder.vcodec=auto \
  --dataset.encoder_threads=2
```

## 4. 用 LeRobotDataset 加载训练数据

`LeRobotDataset` 返回 PyTorch tensors 的 Python dictionaries，并能接入 `torch.utils.data.DataLoader`。示例先从 Hub 加载 `yaak-ai/L2D-v3`，再按索引随机访问，并用 `delta_timestamps` 为指定图像 key 取时间窗口。

```python
import torch
from lerobot.datasets import LeRobotDataset

repo_id = "yaak-ai/L2D-v3"

## 1) Load from the Hub (cached locally)
dataset = LeRobotDataset(repo_id)

## 2) Random access by index
sample = dataset[100]
print(sample)
## {
##   'observation.state': tensor([...]),
##   'action': tensor([...]),
##   'observation.images.front_left': tensor([C, H, W]),
##   'timestamp': tensor(1.234),
##   ...
## }

## 3) Temporal windows via delta_timestamps (seconds relative to t)
delta_timestamps = {
    "observation.images.front_left": [-0.2, -0.1, 0.0]  # 0.2s and 0.1s before current frame
}

dataset = LeRobotDataset(repo_id, delta_timestamps=delta_timestamps)

## Accessing an index now returns a stack for the specified key(s)
sample = dataset[100]
print(sample["observation.images.front_left"].shape)  # [T, C, H, W], where T=3

## 4) Wrap with a DataLoader for training
batch_size = 16
data_loader = torch.utils.data.DataLoader(dataset, batch_size=batch_size)

device = "cuda" if torch.cuda.is_available() else "cpu"
for batch in data_loader:
    observations = batch["observation.state"].to(device)
    actions = batch["action"].to(device)
    images = batch["observation.images.front_left"].to(device)
    # model.forward(batch)
```

## 5. 使用 StreamingLeRobotDataset 直接流式读取

`StreamingLeRobotDataset` 可直接从 Hub 迭代数据，不需要下载本地副本，也不需要把数据加载到内存。HF Storage Bucket 的 `hf://buckets/` 数据也可用同样方式读取，只需传入 `repo_type="bucket"`。

```python
from lerobot.datasets import StreamingLeRobotDataset

repo_id = "yaak-ai/L2D-v3"
dataset = StreamingLeRobotDataset(repo_id)  # streams directly from the Hub
```

```python
dataset = StreamingLeRobotDataset("my-org/my-bucket", repo_type="bucket")
```

```bash
lerobot-train \
  --dataset.repo_id=my-org/my-bucket \
  --dataset.repo_type=bucket \
  --dataset.streaming=true \
  ...
```

## 6. 为训练加载添加图像 transforms

Image transforms 只在训练时应用，不在 recording 时应用；录制数据时保存 raw images。加载数据集时可通过 `image_transforms` 传入默认配置、自定义配置，或直接传入 `torchvision.transforms.v2` transform。

```python
from lerobot.datasets import LeRobotDataset
from lerobot.transforms import ImageTransforms, ImageTransformsConfig, ImageTransformConfig

## Option 1: Use default transform configuration (disabled by default)
transforms_config = ImageTransformsConfig(
    enable=True,  # Enable transforms
    max_num_transforms=3,  # Apply up to 3 transforms per frame
    random_order=False,  # Apply in standard order
)
transforms = ImageTransforms(transforms_config)

dataset = LeRobotDataset(
    repo_id="your-username/your-dataset",
    image_transforms=transforms
)

## Option 2: Create custom transform configuration
custom_transforms_config = ImageTransformsConfig(
    enable=True,
    max_num_transforms=2,
    random_order=True,
    tfs={
        "brightness": ImageTransformConfig(
            weight=1.0,
            type="ColorJitter",
            kwargs={"brightness": (0.7, 1.3)}  # Adjust brightness range
        ),
        "contrast": ImageTransformConfig(
            weight=2.0,  # Higher weight = more likely to be selected
            type="ColorJitter",
            kwargs={"contrast": (0.8, 1.2)}
        ),
        "sharpness": ImageTransformConfig(
            weight=0.5,  # Lower weight = less likely to be selected
            type="SharpnessJitter",
            kwargs={"sharpness": (0.3, 2.0)}
        ),
    }
)

dataset = LeRobotDataset(
    repo_id="your-username/your-dataset",
    image_transforms=ImageTransforms(custom_transforms_config)
)

## Option 3: Use pure torchvision transforms
from torchvision.transforms import v2

torchvision_transforms = v2.Compose([
    v2.ColorJitter(brightness=0.2, contrast=0.2, saturation=0.2, hue=0.1),
    v2.GaussianBlur(kernel_size=3, sigma=(0.1, 2.0)),
])

dataset = LeRobotDataset(
    repo_id="your-username/your-dataset",
    image_transforms=torchvision_transforms
)
```

## 7. 预览 transforms 效果

调参前可用 visualization script 预览增强效果。该命令会保存 example images，展示 each transform 对数据的影响，帮助调整参数。文档建议先保守设置、小范围开始，并确认 transforms 看起来合理。

```bash
lerobot-imgtransform-viz \
  --repo-id=your-username/your-dataset \
  --output-dir=./transform_examples \
  --n-examples=5
```

## 8. 将 v2.1 数据集转换为 v3.0

转换器会把 per-episode files 聚合为更大的 shards，并写入 episode offsets/metadata。文档中的转换先安装带 v3 支持的 pre-release build，再对 Hub 上已有 v2.1 数据集运行转换命令。

```bash
## Pre-release build with v3 support:
pip install "https://github.com/huggingface/lerobot/archive/33cad37054c2b594ceba57463e8f11ee374fa93c.zip"

## Convert an existing v2.1 dataset hosted on the Hub:
python -m lerobot.scripts.convert_dataset_v21_to_v30 --repo-id=<HF_USER/DATASET_ID>
```

## 验证是否跑通

加载时可执行示例中的 `sample = dataset[100]` 并 `print(sample)`；使用 `delta_timestamps` 后，`print(sample["observation.images.front_left"].shape)` 应显示 `[T, C, H, W]`，其中 `T=3`。流式读取时能创建 `StreamingLeRobotDataset(repo_id)`；转换后应完成 parquet/mp4 聚合并更新 `meta/episodes/*`。

## 文档里点名的注意事项

- When creating or recording datasets, you must call `dataset.finalize()` before `push_to_hub()`.
- Without calling `finalize()`, parquet files will be incomplete and the dataset won’t load properly.
- Dataset v3.0 uses incremental parquet writing with buffered metadata for efficiency; `finalize()` flushes buffered episode metadata, closes parquet writers, and ensures the dataset is valid for loading.
- Transforms are applied during training time only, not during recording.
- ImageTransformsConfig is disabled by default: `enable` default is `False`.
- Strong augmentations can hurt performance if too aggressive; using too many transforms simultaneously can make training unstable.

## 相关 SDK 条目

- [LeRobot](/sdks/lerobot)

## 原始参考来源 (Sources)

- [huggingface.co/docs/lerobot/lerobot-dataset-v3](https://huggingface.co/docs/lerobot/lerobot-dataset-v3)