Whole-body grasp generation
Energy-based optimization and simulation filtering produce the arm-and-torso grasp cache used for training.
WBD-EX Object Repositioning Experts
Try out interactive whole-body dexterous experts!
Any-to-any experts (WBD-EX)
Reinforcement-learning experts learn to reposition an object and settle it in a left-arm, both-arm or right-arm hold, using goals sampled from the grasp cache.
Distilled sim-to-real controller
The experts are distilled into one deployable policy that transfers zero-shot to a real Unitree G1 and is steered interactively by a teleoperator.
How does Whole-Body Dexterity (WBD) work?
Generate whole-body grasps, train experts to move objects between them, then distill the experts into one controller.
Whole-Body Grasp Generation
Grasps are optimized from chosen contact links (blue), then kept only if they survive perturbations in simulation.
WBD-EX Training
Experts start from cached grasps and learn to move objects toward sampled positions in left-arm, both-arm or right-arm holds. Intermediate goals connect the current position to a farther goal, allowing sliding and rolling across the torso.
Simulation Experiments
We add noise to expert commands and compare direct execution with WBD refinement. The guidance weight controls how strongly motion generation follows the coarse command.
Real-World Experiments
90%of real-world trials completed
45/50 across repositioning and the shelf task
45/50 across repositioning and the shelf task
5×as many completions as direct retargeting
GMR: 9/50, SONIC: 2/50
GMR: 9/50, SONIC: 2/50
7/10full shelf task completions
generalist controller (SONIC): 0/10
generalist controller (SONIC): 0/10
Experimental setup: teleoperation with WBD controller
An Xsens motion capture suit provides coarse motion references. WBD refines the arm targets; a separate controller handles lower-body motion and balance. GMR (General Motion Retargeting) directly retargets the operator's motion; SONIC is a generalist whole-body controller.
On-body repositioning: basketball (2× speed)
SONIC
GMR
WBD (ours)
Shoulder tap
Underarm stow
| Task | Object | Controller | Region reached ↑ | Full round trip ↑ | Time to region (s) ↓ | Total time (s) ↓ |
|---|---|---|---|---|---|---|
| Shoulder tap | Ball | SONIC | 2/10 | 0/10 | 15.2 | — |
| GMR | 3/10 | 1/10 | 26.0 | 30.1 | ||
| WBD | 10/10 | 9/10 | 7.7 | 17.3 | ||
| Box | SONIC | 2/10 | 2/10 | 15.9 | 24.9 | |
| GMR | 6/10 | 4/10 | 8.0 | 8.5 | ||
| WBD | 10/10 | 10/10 | 4.1 | 6.3 | ||
| Underarm stow | Ball | SONIC | 1/10 | 0/10 | 28.0 | — |
| GMR | 4/10 | 2/10 | 31.5 | 49.8 | ||
| WBD | 10/10 | 9/10 | 10.6 | 26.2 | ||
| Box | SONIC | 2/10 | 0/10 | 26.1 | — | |
| GMR | 2/10 | 1/10 | 9.3 | 17.8 | ||
| WBD | 10/10 | 10/10 | 8.6 | 13.0 |
10 trials per controller, task and object. Times are averaged over successful trials.
Shelf-to-table task: pick, stow, carry, un-stow and place
| Controller (10 trials each) | Pick ↑ | Stow ↑ | Carry ↑ | Un-stow ↑ | Place ↑ |
|---|---|---|---|---|---|
| SONIC | 10/10 | 0/10 | 0/10 | 0/10 | 0/10 |
| GMR | 10/10 | 3/10 | 2/10 | 1/10 | 1/10 |
| WBD | 10/10 | 9/10 | 9/10 | 9/10 | 7/10 |