§18.1 Generalization across embodiments

The bet, and where it stands

§12.4 introduced the wager that a lot of the field placed on this problem: Open X-Embodiment (arXiv:2310.08864). Pool demonstrations from many robots into one dataset, train a single policy on all of it, and hope the shared structure of manipulation, reach toward the thing, close on the thing, move it, transfers across the differences in hardware. The RT-X models trained this way did show positive transfer. A policy trained on the pooled data outperformed policies trained on any single robot’s data alone, which is the result that made the whole approach credible. Octo (arXiv:2405.12213) built on the same dataset with a design meant to absorb heterogeneity directly: a transformer with swappable input and output heads, so a new robot with a different observation and action layout could be attached without retraining the trunk.

So the bet paid off, partially, and the partiality is the interesting part. Transfer works well inside a rough equivalence class. Table-top arms doing pick-and-place share enough that pooling their data helps, and a generalist trained across a dozen of them beats a specialist on the ones it has seen and reaches a usable starting point on ones it has not. Push outside that class and the story gets worse fast. A bimanual system, a mobile manipulator, a quadruped, a humanoid: these do not share an action space with a table-top arm in any way a naive concatenation of joint vectors can express, and simply adding their data to the pool can hurt the arm policies rather than help them. §15.2 made the same point from the data side, that Open X-Embodiment has no single branded successor and that growth now comes from large teleop fleets feeding individual foundation models rather than from one dataset serving everything. The dream of a universal pool ran into the fact that a humanoid and a UR5 are not doing the same math.

What action space are we even generalizing over?

The technical core of the problem is a representation question, and it is worth stating plainly because most of the current research bets are really bets on the answer. If you want one policy to drive many robots, the policy has to emit something that means the same thing to all of them. What is that something?

Make the mismatch concrete. Suppose your policy outputs a 7-vector: six joint velocities plus a gripper command. On the Franka it was trained on, element four drives the elbow, and a value of 0.3 there produces a specific, learned amount of elbow flexion. Feed that identical vector to a Kinova with a different joint ordering and different gear ratios, and element four now drives a wrist rotation at a different scale, so the arm does not do a slightly wrong version of the task, it does something unrelated to it. The numbers transfer; the meaning does not. That single example is the whole problem in miniature, and it is why the choice of what the policy emits matters more than almost any other design decision when many robots are the target.

The candidates each fail somewhere. End-effector pose in Cartesian space, “move the gripper to this position and orientation,” is appealingly robot-agnostic; a pose is a pose regardless of the arm underneath it. But it pushes the whole problem down into the inverse kinematics and controller, which is exactly the §4.2–§4.3 machinery, and it says nothing at all for a robot whose “end effector” is five fingers or two arms. Joint-space targets are precise and directly executable, and they are the least transferable thing imaginable, since joint 3 on one robot is not joint 3 on another. A learned universal action space is the ambitious middle: UniAct (arXiv:2501.10105) proposes a shared vocabulary of “universal actions,” a codebook of behaviors that is common across embodiments, with a small per-robot decoder translating a universal action into that robot’s specific commands. This is the cleanest statement of the idea that there is a level of abstraction, above joints and below language, where all manipulators are doing the same thing. Whether that level really exists, or whether it is an artifact that works on the benchmarks and frays on hardware, is genuinely open. It is a good problem for a student to pick up in §18.5, because you can probe it with one robot and a released checkpoint.

The boundary that is dissolving: whole-body control

There is a shift happening right now that scrambles a distinction this book drew carefully back in Chapter 4, and it belongs here because it is the most concrete form the embodiment question is taking. In Chapter 4, locomotion and manipulation lived on opposite sides of a line. Getting a legged robot to walk was a model-based control problem, a matter of dynamics, contact schedules, and a hand-tuned controller you could reason about; manipulation was where the learned policies went. A humanoid, under that division of labor, ran a classical whole-body controller for its legs and torso and handed off to a VLA for its arms and hands. Two systems, two design philosophies, a clean interface between them.

That interface is being deleted. Figure’s Helix-02, introduced in §14.2, ships what the company calls “System 0,” a neural whole-body controller that replaces the hand-coded locomotion stack, folding balance and stepping into the same learned system that drives the hands. Google DeepMind’s Gemini Robotics 2, announced at the end of July 2026 and covered in §14.6, went further and demonstrated the first VLA to drive an entire humanoid, legs, torso, arms, and multi-fingered hands, under one unified policy, on Apptronik’s Apollo 2. Loco-manipulation, walking-while-carrying, reaching-while-balancing, stops being a handoff between a classical controller and a learned one and becomes a single network’s output. WholebodyVLA (OpenDriveLab, ICLR 2026) is the open-research version of the same direction.

Read this against §4.4, which argued that classical methods remain load-bearing in modern robots precisely in locomotion and low-level control. That argument was correct when it was written and it is now the exact frontier being contested. The whole-body policies are betting that the model-based locomotion controller, one of the last places where hand-derived dynamics clearly beat learning, can be absorbed into the same foundation model that does everything else. If that bet holds, the embodiment problem and the locomotion problem collapse into each other: generalizing across embodiments stops meaning “across different arms” and starts meaning “across different bodies,” legged and wheeled and bimanual alike, with the policy responsible for figuring out how each one moves.

One policy, twenty robots

The most striking recent data point on cross-embodiment scale is LingBot-VLA 2.0, released in July 2026: a 6-billion-parameter open-source VLA reported to run one policy, unmodified, across twenty distinct robot morphologies from seventeen manufacturers, from Unitree and AgiBot to Franka and Fourier. Take the claim with the usual caution owed to a fresh release, and it still marks how far the target has moved. Open X-Embodiment pooled data from twenty-two embodiments to train a policy you then specialized. The newer systems aim for one checkpoint that a person unpacks and points at whatever robot is in the room, which is a categorically harder ask and the one the field has decided to make.

None of this is solved. Transfer across near-identical arms is routine; transfer across genuinely different bodies is a research result, not a product guarantee, and the right action representation to make it reliable is unsettled. What has changed since the early Open X-Embodiment days is the ambition and the evidence that the ambition is not hopeless: whole-body neural control is real hardware now, universal action spaces have released checkpoints you can poke at, and the classical locomotion boundary from Chapter 4 is under active demolition. The next open problem, §18.2, holds even the best of these single-embodiment-generalizing policies to a harder standard: not whether one model can drive many robots, but whether any of them can yet do the long-horizon, dexterous tasks that a human hand makes look trivial.