§12.3 Octo: a generalist policy with a diffusion head

Smaller than you expect

The first surprise is the parameter count. OpenVLA is 7 billion parameters riding on a Llama backbone. Octo ships in two sizes, Octo-Small at 27 million parameters and Octo-Base at 93 million, and neither of those has a comma in the wrong place. It is roughly two orders of magnitude smaller. Octo does not inherit a language model that read the internet; it is a transformer trained from scratch on robot data, with language handled by a frozen off-the-shelf text encoder that turns the instruction into a few embeddings. There is no PaLM, no Llama, no web-scale semantic prior baked into the weights.

That is a real limitation and worth stating up front, because it sets expectations for everything that follows. Octo will not pick “the extinct animal” out of a lineup the way RT-2 could, since it never learned what “extinct” means. What it offers instead is a policy that trains fast, runs fast, and fine-tunes on a single GPU in a couple of hours, and an architecture whose flexibility is the actual research contribution. The team’s bet was that for a generalist manipulation policy, breadth of robot data and a well-designed interface matter more than a giant frozen language prior. Whether that bet holds depends entirely on the task, and §12.5 comes back to it when we ask what “generalist” is really buying.

The block-wise transformer that makes rewiring possible

Octo’s central idea is architectural, so it helps to picture the token stream. Everything the model consumes becomes a token: the language instruction (or a goal image) becomes a small block of task tokens, each observation timestep becomes a block of observation tokens (image patches plus any proprioceptive state), and the model inserts its own learned readout tokens at each timestep, which function a little like the [CLS] token in a vision transformer. Attention runs block-wise. Observation tokens can attend to task tokens and to earlier observations; readout tokens attend to everything before them but nothing attends back to the readouts. The action head hangs off the readout tokens.

Why arrange it this way? Because a block that nothing attends back to can be added or removed without disturbing the rest of the network. This is the mechanism behind the whole “reconfigure after pretraining” claim from the paper’s abstract. Suppose you pretrained on third-person camera images and now your robot has a wrist camera. You add a new observation-token block for the wrist stream, initialize its lightweight encoder, and fine-tune, and the pretrained attention over the existing blocks keeps working because the new tokens slot in without rewiring what was already there. Same story for a new proprioceptive input, or a new action space: you swap the small head, keep the transformer trunk. Contrast this with OpenVLA, where the input was fixed at one image and a sentence and changing it meant surgery. Octo pushed the flexibility into the tokenization scheme itself, and the block-wise attention (§8.1) is what lets that flexibility exist without retraining from scratch.

One consequence is that Octo natively supports a short observation history. Because each timestep is its own block, feeding the last two frames instead of one costs you another block and nothing else. OpenVLA’s single-frame input was a deliberate simplification; Octo’s design makes history nearly free, which matters for tasks where velocity or a partially occluded object needs a couple of frames to disambiguate.

The diffusion head, concretely

Now the part that gives the section its title. Instead of predicting an action token, Octo’s readout tokens feed a diffusion head that generates a continuous action chunk, a short sequence of future actions, by iterative denoising. If §10.2 is fresh, this is Diffusion Policy’s action head grafted onto a generalist transformer trunk. If it isn’t, here is the shape of it. The head starts from a vector of pure Gaussian noise the same size as the action chunk. A small denoising network, conditioned on the readout token’s embedding, is called repeatedly, and each call nudges the noisy vector a step closer to a clean action; after a fixed number of steps you read off the result and send it to the robot. Training is the reverse: add noise to a demonstrated action at a random level, ask the network to predict the noise, minimize the error. No bins, no discretization, no percentile edges to tune.

What does this buy over OpenVLA’s 256 bins? Two things the discrete head structurally cannot do well. First, resolution: a diffusion head outputs a real-valued vector, so it never rounds a delicate motion to the nearest of 256 levels, and the smoothness that §10.4 flagged as diffusion’s advantage carries over. Second, multimodality. When a demonstration set contains two valid ways to do a task, say, reaching around the left or the right side of an object, a classifier trained with cross-entropy tends to smear probability across both and can emit an averaged, invalid action straight down the middle. A diffusion head samples from the distribution instead of averaging it, so it can commit to one mode. This is exactly the behavior Chapter 10 built up, now doing real work inside a foundation-scale generalist policy rather than a single-task demo.

The cost shows up at inference, and Octo is honest about it. A discrete head produces its action in one forward pass. A diffusion head runs its denoising loop every single control step, so the same action costs several network calls instead of one. Octo keeps this cheap in absolute terms because the trunk is only 93M parameters and the denoising network is tiny, so it still runs comfortably faster than a 7B model, but the per-step multiplier is real and it is the same pressure that §13 addresses head-on when flow matching cuts the number of denoising steps toward one. Diffusion buys expressiveness and charges you iterations.

What the pretraining actually used

Octo was pretrained on 800,000 robot trajectories drawn from Open X-Embodiment (arXiv:2310.08864), which the team describes as the largest robot manipulation pretraining mix assembled at the time. That corpus spans many robot embodiments, camera setups, and action spaces, and Octo’s flexible tokenizer is partly a response to that heterogeneity: when your training data comes from dozens of labs with incompatible sensor rigs, a model that can absorb variable inputs is not a luxury, it is a precondition for using the data at all. The data-curation story here is a preview of §12.4 and §15.2, both of which take Open X-Embodiment apart in detail, so I will not reproduce the weighting and normalization mechanics now beyond noting that they exist and that they mattered.

The headline evaluation claim is that Octo serves as an effective initialization for downstream fine-tuning across nine robot setups from four institutions, reaching new observation and action spaces with a couple of hours of training on one GPU. That framing is deliberate and it is different from OpenVLA’s. OpenVLA led with zero-shot generalist success rates against RT-2-X; Octo leads with “how good a starting point is this for your robot.” The two papers are optimizing for adjacent but distinct notions of what a released checkpoint is for, and it is worth noticing which one matches your situation before you pick a base model in Chapter 16.

Reading Octo against OpenVLA

Put the two next to each other and the trade space of mid-2024 open VLAs comes into focus. OpenVLA: 7B parameters, a web-pretrained Llama backbone, a discrete token head, one fixed image input, strong zero-shot semantic generalization, six hertz on a 4090. Octo: 93M parameters, a from-scratch transformer, a diffusion head, a reconfigurable set of inputs and outputs, weak semantic priors, fast and cheap to fine-tune. Neither dominates. If your task leans on recognizing objects and concepts the robot never manipulated, OpenVLA’s backbone is doing something Octo cannot fake. If your task needs smooth, multimodal control and you expect to adapt the model to a nonstandard sensor rig on a modest budget, Octo’s head and interface were built for exactly that.

The deeper lesson is that “foundation action model” was not yet one thing in 2024. It named a cluster of designs that disagreed about the two questions this chapter keeps circling: what should the backbone know before it sees a robot, and how should the action leave the network. RT-2 answered “know the web, leave as tokens.” OpenVLA kept that and opened it. Octo answered “know robots broadly, leave as denoised continuous chunks, and let me rewire the rest.” π0 in Chapter 13 will offer a third answer that takes the diffusion-style continuous head seriously enough to rebuild it as flow matching, which is where the latency cost we just flagged finally gets paid down.

Octo also quietly makes the case that the dataset, not the architecture, is the load-bearing part. A 93M model with no language prior held its own as a generalist initialization because it was trained on 800k diverse trajectories, and that is a claim about data doing the heavy lifting. Both models in this chapter so far point at the same corpus, and it has now come up in every section, so the next one stops treating Open X-Embodiment as a footnote and looks directly at the dataset that made all of this possible.