§18.2 Long-horizon and dexterous tasks

Two problems that get bundled and should not

Long-horizon and dexterous are different failures, and treating them as one word obscures where the field is stuck. Long-horizon difficulty is a problem of time and error accumulation. Dexterity is a problem of contact and control bandwidth. A robot can be good at one and hopeless at the other, and most current systems are exactly that.

The long-horizon problem is the compounding-error problem from §6.3, scaled up until it dominates. A behavior-cloned policy makes a small error, the small error moves the state slightly off the training distribution, the next action is drawn from a worse-conditioned part of the model, and the errors feed on themselves. Over a five-second grasp this is a nuisance you can often absorb. Over a two-minute, twelve-step task it is fatal, because the probability of executing all twelve steps correctly is the product of twelve per-step success rates, and even 95% per step compounds to roughly 54% over twelve. The arithmetic is unforgiving and it explains why demos are short. A one-minute autonomous task is a genuine research result; a ten-minute one is rare enough to make headlines.

The dexterity problem is different in kind. A parallel-jaw gripper has one degree of freedom and forgives a lot: close it near an object and the two plates find the grasp. A five-fingered hand has twenty or more degrees of freedom, and using them means controlling contact, when a fingertip touches, how hard, in what sequence, with feedback fast enough to catch a slip before the object falls. This is where the model-free imitation approach strains. The training signal, teleoperated demonstrations, is thin for hands because teleoperating twenty joints is hard for the human operator too, and the control bandwidth needed to react to a slip is higher than the 5-to-20 Hz many VLAs run at from §14.4. Contact dynamics are stiff and near-discontinuous, exactly the regime the smooth flow-matching heads from §10.4 were designed to handle but have not fully conquered.

Where the horizon problem is being attacked

The clearest line of work accepts that a single flat policy will not hold a long task together and reintroduces structure. Long-VLA (arXiv:2508.19958) is a representative attempt: rather than asking one policy to emit a two-minute trajectory in one undifferentiated stream, it gives the model a way to represent and sequence sub-tasks, so the long task is decomposed into segments the policy can actually execute and a higher level tracks which segment it is in. LiLo-VLA (arXiv:2602.21531) works the same territory. The intuition should feel familiar, because it is the §4.1 symbolic planner’s intuition wearing new clothes: hard long tasks are easier when you decompose them into sub-goals and solve the sub-goals. The classical planners did this with hand-written operators; the new work tries to learn the decomposition and keep it differentiable, which is the part that is genuinely open.

A second attack comes from a direction Chapter 5 would recognize. Physical Intelligence’s π0.6, documented in their RECAP note (pistar06.pdf), takes a π-family policy and improves it with reinforcement learning from the robot’s own experience rather than from more human demonstrations alone. This matters for long horizons specifically, because RL optimizes the whole return, the eventual completion of the task, rather than imitating one step at a time, and a policy trained against the full-task reward has pressure to recover from the off-distribution states that compounding error drives it into. That is exactly the failure §6.3 said imitation cannot fix on its own and §6.5 said RL was the tool for. The π0.6 result is the field circling back to reward-based learning to buy the robustness that pure cloning cannot, now on top of a foundation model rather than from scratch.

Where the dexterity problem is being attacked

Dexterity has attracted dedicated foundation models, which is itself the news. For most of this book the story was one generalist policy doing many tasks; dexterity is hard enough that two well-funded efforts have decided it deserves a purpose-built model. Genesis AI’s GENE and RLWRLD’s RLDX-1, both surfacing through 2026, are foundation models aimed squarely at dexterous, contact-rich manipulation rather than at the pick-and-place tasks that fill Open X-Embodiment. The bet behind them is that dexterity is not a task you get for free by scaling a generalist, that the data, the action representation, and the control rate a five-fingered hand needs are different enough to warrant specializing the whole stack.

Whether that bet is right is open, and it is a good example of a live disagreement a reader can track. The generalist camp holds that dexterity emerges from scale, that a large enough policy trained on enough hand data will pick up contact control the way RT-2 picked up semantics (§12.5). The specialist camp, GENE and RLDX-1 among them, holds that contact-rich control has structure, tactile feedback loops, force modulation, finger gaiting, that you should build in rather than wait for, and that a generalist spread across a hundred embodiments will never allocate enough capacity to any one hand. The honest status: nobody has shown human-level dexterity from either approach, and the gap to a human hand is still large. A person re-grasps a slipping object without thinking; the best robot hands do it slowly, in the lab, on objects they have seen.

The measurement problem hiding underneath

There is a reason progress here is hard to read, and it connects back to Chapter 15. Long-horizon and dexterous tasks are exactly the ones the standard benchmarks measure worst. A benchmark of short table-top tasks (§15.4) says nothing about a two-minute task, and a success-rate number on a grasp says nothing about the quality of the manipulation, whether the object was handled gently, whether the grip would have survived a perturbation. The field is partly stuck because it is optimizing what it can measure, and the measurements reward short and forgiving over long and dexterous. §15.5’s insistence on time-to-completion and on variance, not just binary success, is a partial fix, and it is no accident that the labs pushing hardest on dexterity are also the ones building their own evaluations rather than trusting LIBERO.

Consider one concrete illustration of how badly the two problems interact. A folding-laundry task, a favorite in demo reels, is both long and dexterous: it runs for minutes, and each fold asks the hand to manage a deformable object whose state the robot cannot fully observe. A rigid box has a pose you can estimate; a shirt has a configuration space that is effectively infinite, and the gripper has to feel its way through contact it cannot predict from vision alone. When a laundry demo works, it usually works on one shirt, folded one way, from one starting configuration, and the moment you hand it a different garment the success rate falls off a cliff. That cliff is the honest measure of where the field is: not the polished clip, but how fast performance degrades when the object or the sequence changes even slightly. A reader who wants to feel the frontier directly can reproduce the cliff on a tabletop with two objects of different stiffness and a three-step task.

This is where a reader with one robot can actually contribute, which §18.5 develops. You do not need a fleet to study long-horizon failure; you need one arm, one genuinely multi-step task, and the discipline to log where in the sequence the policy breaks and why. The compounding-error curve is reproducible on a tabletop. So is the observation that a policy’s dexterity collapses the moment you swap the training object for a slightly slipperier one, which is the generalization-to-contact question the specialist models are betting their existence on.

Long horizons and dexterity are the capabilities that separate a compelling demo from a robot that earns its keep, and they remain the clearest statement of how far there is to go. The horizon problem is being met with learned decomposition and with reinforcement learning bolted onto foundation policies; the dexterity problem is being met with purpose-built models and a still-unsettled argument about whether it needed them. Neither is close to a human hand executing a two-minute plan without a thought. The next section turns to a resource both problems might eventually draw on: the enormous quantity of video showing humans doing exactly these tasks, and the models beginning to learn from it.