Robot Dataset Analysis and VLA Inputs/Outputs · Study
English translation
Execution conditions
Small-scale data analysis requires requests, pandas, pyarrow, and scikit-learn. Read only one public Parquet file; an internet connection is required.
VLA model inference and training use separate hardware and model-specific environments.
If the required package is missing, run %pip install pyarrow==23.0.1 in a Colab or Jupyter code cell.
Topics covered
Read public demonstration data and establish evaluation criteria, then check the meanings of inputs and outputs connected to vision, language, and action models.
Demonstration data and evaluation
Consecutive frames are strongly correlated, so random frame splits can differ from episode-level splits. Also distinguish recorded action prediction errors from actual task success rates.
The VLA interface
Camera input, current robot state, and language instructions lead to an action chunk. Using a pretrained policy requires knowing the robot-specific action representation, normalization, and timing.
VLM, VLA, and serving
A VLM interprets scenes, while a VLA outputs actions. vLLM can support inference and serving for supported models; whether its output is a valid robot action must be verified separately.
Worked example
Read the first Parquet shard of LeRobot svla_so100_pickplace. Do not download videos or model weights.
"""Read one public LeRobot data shard, without downloading videos or model weights."""
import io,json
import numpy as np
import pandas as pd
import requests
repo='lerobot/svla_so100_pickplace'
def read_json(url):
response=requests.get(url,timeout=30);response.raise_for_status();return response.json()
manifest=read_json('https://huggingface.co/api/datasets/'+repo)
revision=manifest['sha']
base='https://huggingface.co/datasets/'+repo+'/resolve/'+revision+'/'
metadata=read_json(base+'meta/info.json')
shards=sorted(x['rfilename'] for x in manifest['siblings'] if x['rfilename'].startswith('data/') and x['rfilename'].endswith('.parquet'))
assert shards,'Dataset layout changed; inspect the repository data tree.'
response=requests.get(base+shards[0],timeout=60);response.raise_for_status()
frame=pd.read_parquet(io.BytesIO(response.content))
print('Repository / revision / shard:',repo,revision,shards[0])
print('Format / FPS / episodes:',metadata.get('codebase_version'),metadata['fps'],metadata['total_episodes'])
print('Features:',metadata['features'])
print('Loaded rows / episodes:',len(frame),frame['episode_index'].nunique())
state=np.stack(frame['observation.state'].to_numpy())
action=np.stack(frame['action'].to_numpy())
print('State / action shape:',state.shape,action.shape)
# Task text and video paths are described in the repository metadata.
# No robot commands, model execution, remote uploads, or video downloads occur here.
Repository / revision / shard: lerobot/svla_so100_pickplace 728583b5eaf9e739a7f119e2def466fa1d552402 data/chunk-000/file-000.parquet
Format / FPS / episodes: v3.0 30 50
Features: {'action': {'dtype': 'float32', 'shape': [6], 'names': ['main_shoulder_pan', 'main_shoulder_lift', 'main_elbow_flex', 'main_wrist_flex', 'main_wrist_roll', 'main_gripper'], 'fps': 30.0}, 'observation.state': {'dtype': 'float32', 'shape': [6], 'names': ['main_shoulder_pan', 'main_shoulder_lift', 'main_elbow_flex', 'main_wrist_flex', 'main_wrist_roll', 'main_gripper'], 'fps': 30.0}, 'observation.images.top': {'dtype': 'video', 'shape': [480, 640, 3], 'names': ['height', 'width', 'channels'], 'info': {'video.fps': 30.0, 'video.height': 480, 'video.width': 640, 'video.channels': 3, 'video.codec': 'av1', 'video.pix_fmt': 'yuv420p', 'video.is_depth_map': False, 'has_audio': False}}, 'observation.images.wrist': {'dtype': 'video', 'shape': [480, 640, 3], 'names': ['height', 'width', 'channels'], 'info': {'video.fps': 30.0, 'video.height': 480, 'video.width': 640, 'video.channels': 3, 'video.codec': 'av1', 'video.pix_fmt': 'yuv420p', 'video.is_depth_map': False, 'has_audio': False}}, 'timestamp': {'dtype': 'float32', 'shape': [1], 'names': None, 'fps': 30.0}, 'frame_index': {'dtype': 'int64', 'shape': [1], 'names': None, 'fps': 30.0}, 'episode_index': {'dtype': 'int64', 'shape': [1], 'names': None, 'fps': 30.0}, 'index': {'dtype': 'int64', 'shape': [1], 'names': None, 'fps': 30.0}, 'task_index': {'dtype': 'int64', 'shape': [1], 'names': None, 'fps': 30.0}}
Loaded rows / episodes: 19631 50
State / action shape: (19631, 6) (19631, 6)