§14.1 Why a single forward pass is not always enough

Two jobs, two clocks

The deeper issue is that a general-purpose robot is really doing two different jobs, and they want to run at rates that differ by more than an order of magnitude.

The first job is figuring out what to do. Parse “clear the table and put the mugs in the sink,” notice that the blue mug is a kind you have never seen, decide to grab it by the rim because it is full, sequence the whole thing so you do not knock over the glass on the way. This is semantic work, and it is exactly what a large vision-language model is good at after the web pretraining we spent Part 4 on. It is also slow, both because a 7B-plus model is expensive to run and because it does not need to be fast. The decision “pick up the blue mug next” stays valid for a second or two. Re-deriving it two hundred times a second would waste enormous compute to arrive at the same answer over and over.

The second job is running the body. Track the commanded trajectory, hold balance, feel the mug start to slip and tighten the grip before it drops, absorb the small shove when a coworker brushes past. This is sensorimotor work, and its defining property is that it has to be fast. A bipedal robot maintaining balance is an unstable system in the control-theory sense; leave it alone for a tenth of a second and it topples. Contact events are worse, because the useful reaction window when a grasp slips is on the order of tens of milliseconds. None of this work needs to understand the instruction. It needs fresh sensor readings and a tight loop.

Section 4.3 already showed you a version of this division, though we did not frame it this way then. Classical robots have always run a fast inner control loop, computed-torque or impedance control at hundreds of hertz, wrapped inside a slow outer planner that reasons about goals at a leisurely pace. The inner loop does not know what the task is; it just tracks a reference and rejects disturbances. The dual-system VLAs in this chapter are, in a real sense, a rediscovery of that layered structure, with a learned VLM standing in for the planner and a learned sensorimotor policy standing in for the tracking controller. What changed is that both layers are now neural networks trained end to end on demonstrations rather than hand-derived from a dynamics model.

What a single forward pass forces you to give up

Fold both jobs into one network that produces an action from an observation, and you are forced to pick a single rate for the whole thing, because the two jobs share one forward pass. Every choice of that rate is bad in a different way.

Run the whole model fast enough for control, say two hundred hertz, and you cannot afford a 7B backbone in the loop; a model that big does not finish a forward pass in five milliseconds on hardware that fits on a robot. So you shrink the model, and now your reasoning is worse: the shrunken network handles familiar tabletop tasks but loses the open-world semantic grounding that made the big VLM worth using. Run the whole model slow enough for the big backbone, say five to ten hertz, and the control loop inherits that rate. Now the humanoid falls over, because balance cannot wait a hundred milliseconds between corrections. You can try to split the difference and land somewhere unhappy in the middle, too slow to balance and too dumb to reason, which is the worst of it.

π0’s chunking is the most sophisticated single-system dodge, and it is worth being honest that it only bends this constraint rather than breaking it. By generating a chunk, π0 lets one slow backbone pass cover a second of fast motion, so the effective control rate rises without re-running the backbone. But the chunk is open-loop within itself. The fast corrections inside that second are replayed from what the policy decided at the top of the chunk, not computed against sensor readings arriving mid-chunk. For quasi-static manipulation that gap is tolerable. For anything dynamic it is the whole problem.

The dual-system answer

The fix is to stop sharing the forward pass. Run two networks, at two rates, at the same time.

A slow system, call it System 2, is a full vision-language model. It looks at the cameras and the instruction a few times a second, does the semantic reasoning, and emits not an action but an intent: a compact latent vector that means something like “reach for the blue mug’s rim.” A fast system, call it System 1, is a small sensorimotor network running at a couple hundred hertz. It takes the latest latent from System 2 as a standing goal and closes the actual control loop against fresh proprioception and vision, turning “reach for the rim” into torques or joint targets, correcting for slip and shove on every cycle. The two share only the thin latent channel. System 2 sets direction; System 1 does the driving.

The naming is a deliberate nod to Kahneman’s fast-and-slow split in human cognition, and the analogy is decent as long as you do not lean on it: a deliberate, effortful, general reasoner supervising a quick, automatic, specialized reactor. The engineering payoff is concrete. The expensive model runs at the low rate its job actually tolerates, so you get to keep it big and smart. The cheap model runs at the high rate control demands, so balance and contact stay stable. Neither compromises to accommodate the other, because they are no longer chained to one clock.

Figure’s Helix (figure.ai/news/helix) is the cleanest deployed instance. Its System 2 is a 7B open-source VLM that runs on the order of 7 to 9 Hz; its System 1 is an 80M-parameter visuomotor transformer that runs at 200 Hz, driving a humanoid’s full upper body including individual fingers. NVIDIA’s GR00T N1 (arXiv:2503.14734) takes the same two-rate shape for humanoids, pairing a VLM reasoner with a diffusion-transformer action module, and we will pull both apart in the next two sections. The interesting design questions live in the seam between the systems: how wide the latent channel should be, how the fast system stays stable when the slow system’s intent updates underneath it, and whether the two get trained together or separately.

Trace one task through both clocks to see why the split matters. Suppose a Figure humanoid on a logistics line has to pick a plastic tote off a moving conveyor and set it on a rack. System 2 looks at the scene a handful of times a second: it reads the instruction, locates the tote, notices it is sliding left on the belt, and settles on the intent “grasp the near handle, lift, pivot right.” That intent holds steady for a second or more; the reasoning does not need refreshing while the hand is still closing. System 1, meanwhile, is doing something System 2 never sees. As the fingers wrap the handle the tote’s weight turns out higher than expected and the grip starts to slip, so the fast loop, sampling force and joint state at 200 Hz, tightens the fingers and adjusts the wrist inside maybe thirty milliseconds. A coworker bumps the robot’s shoulder; the same loop rejects the push and holds the trajectory. At no point did System 2 re-plan for the slip or the bump, because it could not have reacted in time and did not need to. The two failures that would sink a single-system policy, a stale plan and a missed contact correction, land on different networks running at the rates that can actually handle them.

None of this comes free. You have doubled the number of networks, invented a communication protocol between them, and taken on the job of keeping a 200 Hz loop stable while its goal shifts every hundred milliseconds or so. A single-system model like OpenVLA has none of that machinery, and for a slow tabletop arm it does not need it. The dual-system design earns its complexity only when the task is fast, dynamic, or dexterous enough that one clock genuinely cannot serve both jobs, which is why §14.7 comes back to the decision of whether your robot needs this at all rather than assuming it does.

We now have the argument in hand: reasoning and reacting want different rates, one forward pass can only offer one rate, and splitting the model into a slow reasoner and a fast controller lets each run at the rate its job requires. The next section makes this concrete by walking through Helix end to end, from the 7B System 2 down to the 200 Hz System 1 that moves the fingers.