DR-LfD
Decompose and ReorganizePlanning with Primitives and Visuomotor Policies Learned from Demonstrations
IEEE Transactions on Robotics (T-RO)Yizhou Chen et al. · show all 10 authors and affiliations
DR-LfD decomposes demonstrations into reusable skills and composes them with task and motion planning. Under unseen object placements in three real-world tasks, it succeeds in 83% of trials, compared with 18% for π₀.₅.

Video Introduction
Skill Reuse and Composition
Cup-Sponge-Screwdriver. DR-LfD composes three separately learned skills into a 19-step plan with three distinct bimanual operations. Trained on the same 60 demonstrations, DP and π₀.₅ succeed in 0% and 10% of 20 trials under OOD placements, while DR-LfD succeeds in 50%.
Skill composition and reuse
TAMP composes three separately learned skills into one long-horizon task: screwdriver packing, cup handoff, and cup wiping.
Rollouts under different initial object placements (6× speed).
Skill improvement under clutter. With four distractors on the table, the success rate of DR-LfD drops from 50% to 0%. Because skills are learned separately, retraining each skill with five cluttered-background demonstrations restores the success rate to 55% (4+R in the figure), without recollecting any long-horizon demonstration.

Spatial Generalization
Diffusion Policy
OOD setting · Failure · 4× speed
DR-LfD
Leaky setting · Success · 4× speed
SE(3)-equivariant diffusion primitives trained on ≈20 demonstrations adapt to new object positions and orientations without explicit 6-DoF object-pose estimation.
Object Handoff — OOD placements (4× speed).
Screwdriver Packing — OOD placements (4× speed).
Screwdriver Packing on Franka — single-arm platform, placements randomized by tossing (4× speed).
Randomization via Tossing
Tossing leaves the screwdriver at a random position and orientation, and the equivariant primitive adapts to each new placement.
One-shot video, the robot adapts to any displacements
Constraint Handling
Reachability and obstacle avoidance
DR-LfD exposes the reachability and collision constraints of each visuomotor policy to TAMP, which inserts corrective actions when a constraint is violated.

Unreachable Setup
When an object lies beyond one arm’s reach, DR-LfD uses the other arm to move it into their shared workspace before insertion.
Obstructed Setup
When a pole blocks the workspace, DR-LfD relocates it to a safe pose before bimanual insertion.
Reactive Recovery
After each learned skill, a wrist-camera contact detector checks whether the expected contact is established. If it is not, DR-LfD re-perceives the scene and replans locally.

When an operator moves the target object away before the grasp, DR-LfD recovers and succeeds in 9/10 Object Handoff, 5/10 Screwdriver Packing, and 10/10 Cup Wiping trials.
Object handoff — cup pulled away mid-reach (2× speed).
Screwdriver packing — screwdriver taken away (2× speed).
Cup wiping — cup relocated before wiping.
Cup-sponge-screwdriver — recovery during the full long-horizon task (4× speed).
Key Contributions
Core idea: DR-LfD learns individual manipulation skills from demonstrations and composes them with TAMP to achieve new task goals.
- TAMP-gated system — Contact changes define atomic skills that TAMP composes into task plans.
- Object-centric primitives — Equivariant diffusion models learn object-centric primitives that adapt to new object positions and orientations.
- TAMP-compatible visuomotor policies — Initiation conditions, termination conditions, and physical constraints connect learned policies to sampling-based TAMP.
- Evaluation — Real-world and simulation experiments test skill composition, spatial generalization, and physical constraint handling against imitation learning baselines.
How It Works
Method pipeline and implementation details
by contact
repertoire
TAMP
verify
- Demonstrate — Record about 20 teleoperated demonstrations per short skill.
2. Decompose by contact
A VLM identifies robot–object and object–object contact changes. These changes define atomic skills and their start and end conditions.

3. Learn each skill
Train object-centric primitives for pick-and-place actions and visuomotor policies for contact-rich actions. Use standard TAMP motion skills for free-space moves.

- Plan a new task — TAMP selects and orders the learned skills, adding regrasping or obstacle removal to satisfy safety and reachability constraints.
- Execute and verify — The robot perceives, plans, executes, and verifies each step. Failed steps trigger local replanning.
Real-world Results
Real-world evaluation / Unseen placements
Higher success across all three OOD tasks.
Success rates under OOD position and rotation shifts. ALOHA · 20 trials per task and method.
Object handoff
95%
+75 points over π₀.₅
Screwdriver packing
80%
+70 points over π₀.₅
Cup-sleeve insertion
75%
+50 points over π₀.₅
Full results, evaluation protocol, and limitations
Spatial Generalization — Real World (n = 20 per setup)
Each task uses 20 demonstrations. DP and DR-LfD are trained from scratch on these demonstrations, and π₀.₅ is fine-tuned on the same data. ID, OOD, and Leaky placements are defined in Real-world Task Settings.
| Task | Setup | DP | π₀.₅ | DR-LfD |
|---|---|---|---|---|
| Object Handoff | ID | 90% | 100% | 100% |
| OOD | 30% | 20% | 95% | |
| Leaky | 60% | 90% | 100% | |
| Screwdriver Packing | ID | 55% | 60% | 80% |
| OOD | 10% | 10% | 80% | |
| Leaky | 0% | 75% | 65% | |
| Cup-sleeve Insertion | ID | 75% | 85% | 80% |
| OOD | 35% | 25% | 75% | |
| Leaky | 35% | 35% | 40% |
In the Leaky setting, π₀.₅ leads on Screwdriver Packing. Cup-sleeve Insertion remains difficult for all methods; fine insertion and noisy single-view depth sensing contribute to failures.
Real-world Task Settings
Tasks and initial placements
All real-world tasks run on a bimanual ALOHA robot with one overhead RGB-D camera.
Comparative tasks
Object Handoff. The right arm hands a tape roll to the left arm.
Screwdriver Packing. The robot packs a screwdriver into its box and closes the lid.
Cup-sleeve Insertion. The two arms insert a paper cup fully into a cup sleeve.
Initial Object Placements
Colored regions mark where each object starts. ID tests use the same regions as the demonstrations, and OOD tests enlarge every region. For Screwdriver Packing, OOD also replaces the three demonstrated headings (−10°, 0°, 10°) with a continuous range. Leaky tests tilt the target objects against static supports.

Cup-Sponge-Screwdriver
This long-horizon task composes three skills: screwdriver packing, cup handoff, and cup wiping. Each skill is learned from 20 demonstrations.
Training and Deployment
Each skill is demonstrated separately, with its objects in a small region at a fixed orientation (a–c). At deployment, all three skills run in one scene, and the objects start anywhere in much larger regions (d). The screwdriver heading varies within (−90°, 90°), and the cup and sponge headings vary within (−45°, 45°).

Simulation Benchmarks (LIBERO & DexMimicGen)
LIBERO. DR-LfD reaches 98% on both the Spatial and Object suites, versus 84% and 78% for Diffusion Policy trained with 50 demos per task. On Long8, adding online perception raises the success rate of DR-LfD from 30% to 90% over 10 rollouts.
Data efficiency on DexMimicGen. On the contact-rich Two-Arm Threading and Two-Arm Assembly tasks, we train DR-LfD, Diffusion Policy (DP), and SE(3)-equivariant SDP with 100 to 1000 demonstrations per task.

DR-LfD trained on just 100 demonstrations outperforms DP and SDP trained with 1000 demonstrations.
Citation
@article{chen2026decompose,
title = {Decompose and Reorganize: Planning with Primitives and Visuomotor Policies Learned from Demonstrations},
author = {Chen, Yizhou and Xu, Hang and Yu, Dongjie and Lu, Yupu and Xu, Tengye and Zhang, Zeqing and Zhang, Wei and Ren, Yi and Chen, Ben M. and Pan, Jia},
journal = {arXiv preprint arXiv:2607.25397},
year = {2026}
}