THE BENCHMEND VIDEO GALLERY
Agent as Policy
Benchmark Rollouts
GPT-6-Astra high rollout
From the evaluation recordings
LIBERO
490 rollouts
400-episode comparison
| Metric | Original | Revision |
|---|
| Success rate | 80.50% | 89.25% |
|---|
| IAS | 88.25% | 100.00% |
|---|
Explore LIBERO ↗
From the evaluation recordings
RoboTwin
631 rollouts
500-episode comparison
| Metric | Original | Revision |
|---|
| Success rate | 67.20% | 86.00% |
|---|
| IAS | 73.80% | 93.00% |
|---|
Explore RoboTwin ↗
From the evaluation recordings
RoboCasa
365 rollouts
365 original episodes
| Metric | Original |
|---|
| Success rate | 45.21% |
|---|
| IAS | 80.82% |
|---|
Explore RoboCasa ↗
From the evaluation recordings
RoboDojo
42 rollouts
42 original episodes
| Metric | Original |
|---|
| Success rate | 14.29% |
|---|
| IAS | 78.57% |
|---|
Explore RoboDojo ↗