Dataset Viewer
Auto-converted to Parquet Duplicate
Search is not available for this dataset
video
video
2
129
End of preview. Expand in Data Studio

Embodied First Exam: media

This repository holds the pictures and videos behind the Embodied First Exam, a benchmark for code agents in robotics. Each question in the exam is a frozen simulation scene with a goal written for a person, and the answer is a program, which is marked by replaying it in a fresh simulator.

Leaderboard data (every counted attempt of the ranked agents, with a link to its full run): leaderboard/.

This repository is for media only. The agent runs (every step, tool call and picture an agent saw, the verifier's replay and the full trajectory) are in MLL-Lab/embodied-exam-runs, the only place they are kept; the website's run viewer reads them from there. The raw trials the runs are built from are archived, encrypted, in MLL-Lab/efe-raw-trials. Until 2026-11-15 an older copy of some runs was kept here under runs/; it is in this repository's history at revision c9b2ce8ff963647d8a820539b37f73dedb602172.

Layout

masters/<suite>/<task>/video.mp4     master video of the task being solved (2560x1440 or larger, ray or path traced)
masters/<suite>/<task>/poster.png    a full-resolution frame
masters/<suite>/<task>/meta.json     how it was made: source run, renderer, size, speed, success check
masters/<suite>/teaser.mp4           the suite's teaser, and highlights.json
site/<suite>/...                     the web copies the website's pages load
web/<suite>/...                      older compressed copies, still linked by the blog posts
blog/<case>/                         the blog's re-rendered case videos (video.mp4, web.mp4, poster.jpg, meta.json)
leaderboard/                         the leaderboard's data: every counted attempt, with a link to its run

How the masters are made

Since 16 October 2026 every master is rendered natively above 1080p (2560x1440 is the target) with ray tracing or path tracing; rasterized renders are not accepted. Each video replays a solved run of the graded task itself (the reference solution, a successful agent run, or an official demonstration), with physics untouched; only the camera, lights and materials change. Tasks without a solved run show a slow orbit of their own starting scene. The full standard is masters/MEDIA_STANDARD.md.

Contributing

Suite authors send their media through a review repository; after review it is mirrored here. Please do not open pull requests on this repository.

Licence

The videos and pictures are renders of the suites' simulation scenes, made by the suite authors and the exam's maintainers. The scenes and assets keep the licences of their upstream benchmarks; see each suite's page on the exam's website.

Downloads last month
46,433