Abstract
Current humanoid dexterous manipulation remains largely confined to end-effectors and simple grasps, falling short of humans’ ability to engage the entire body by bracing and sliding bulky objects against the arms and torso. We introduce Whole-Body Dexterity (WBD), a general framework that uses the robot's body and limbs as contact surfaces to grasp and manipulate objects. Our sim-to-real pipeline generates a cache of feasible whole-body grasps, trains reinforcement-learning experts on an any-to-any grasp objective, and distills them into a single deployable proprioception-only policy. The distilled policy transfers zero-shot to a real Unitree G1, repositioning a basketball and cardboard box, completing 90% of trials, compared with 18% for direct retargeting and 4% for a generalist whole-body controller.
45/50 · distilled controller
9/50 · direct motion retargeting
2/50 · generalist whole-body controller
Teleoperation setup
All trials are run by a single skilled teleoperator wearing an Xsens motion-capture suit, which streams SMPL body motion that serves as a motion command. The operator commands the upper body, while a separate controller handles the lower body.
Long-horizon shelf-to-table task
The shelf task combines several dexterous skills into one sequence: pick a box from a shelf, stow it under one arm, carry it to a table, and place it.
| Controller | Pick ↑ | Stow ↑ | Carry ↑ | Un-stow ↑ | Place ↑ |
|---|---|---|---|---|---|
| SONIC | 10/10 | 0/10 | 0/10 | 0/10 | 0/10 |
| GMR | 10/10 | 3/10 | 2/10 | 1/10 | 1/10 |
| WBD (ours) | 10/10 | 9/10 | 9/10 | 9/10 | 7/10 |
On-body repositioning
The object moves from a stable two-arm front grasp to a target region on the body and back to a stable two-arm front grasp.
| Task | Controller | Reached ↑ | Round trip ↑ | Region (s) ↓ | Total (s) ↓ |
|---|---|---|---|---|---|
| Shoulder tap | SONIC | 2/10 | 0/10 | 15.2 | — |
| GMR | 3/10 | 1/10 | 26.0 | 30.1 | |
| WBD (ours) | 10/10 | 9/10 | 7.7 | 17.3 | |
| Underarm stow | SONIC | 1/10 | 0/10 | 28.0 | — |
| GMR | 4/10 | 2/10 | 31.5 | 49.8 | |
| WBD (ours) | 10/10 | 9/10 | 10.6 | 26.2 |
| Task | Controller | Reached ↑ | Round trip ↑ | Region (s) ↓ | Total (s) ↓ |
|---|---|---|---|---|---|
| Shoulder tap | SONIC | 2/10 | 2/10 | 15.9 | 24.9 |
| GMR | 6/10 | 4/10 | 8.0 | 8.5 | |
| WBD (ours) | 10/10 | 10/10 | 4.1 | 6.3 | |
| Underarm stow | SONIC | 2/10 | 0/10 | 26.1 | — |
| GMR | 2/10 | 1/10 | 9.3 | 17.8 | |
| WBD (ours) | 10/10 | 10/10 | 8.6 | 13.0 |