Action Models for Robot Learning

Table of contents

All 18 chapters, drafted openly. Pending sections are visibly pending, never hidden behind a fake “coming soon.”

Download
▾
DRAFTED 137 / 137 sections
CHAPTERS COMPLETE 18 / 18
PARTS 5

PART ONE

Foundations and a first taste of VLAs

1

The robot learning problem

  1. 1.1 Why "action" is the hard part of robotics DRAFTED
  2. 1.2 Anatomy of an action model: inputs, outputs, training signal DRAFTED
  3. 1.3 A short history, from STRIPS to π0 DRAFTED
  4. 1.4 The four families of action models DRAFTED
  5. 1.5 What you will and will not find in this book DRAFTED
  6. 1.6 Summary DRAFTED
  7. 1.x Hands-on exercise + chapter references DRAFTED
2

Your first VLA, end-to-end

  1. 2.1 What we are going to build, and what is hidden inside DRAFTED
  2. 2.2 Setting up the environment (OpenVLA weights, LIBERO simulator) DRAFTED
  3. 2.3 Walking through the inference loop one line at a time DRAFTED
  4. 2.4 When it works and when it does not DRAFTED
  5. 2.5 What is left for the rest of the book DRAFTED
  6. 2.6 Summary DRAFTED
  7. 2.x Hands-on exercise + chapter references DRAFTED
3

Math and ML prerequisites in 30 minutes

  1. 3.1 Vectors, matrices, gradients, and why the chain rule rules robotics DRAFTED
  2. 3.2 Random variables, expectations, KL divergence DRAFTED
  3. 3.3 A 50-line PyTorch training loop, annotated DRAFTED
  4. 3.4 Three loss families: supervised, RL, self-supervised DRAFTED
  5. 3.5 Debugging a model that will not train DRAFTED
  6. 3.6 Summary DRAFTED
  7. 3.x Hands-on exercise + chapter references DRAFTED

PART TWO

The lineage that produced VLAs

4

Classical action models: planning and inverse dynamics

  1. 4.1 Symbolic actions: STRIPS, PDDL, and action schemas DRAFTED
  2. 4.2 Geometric actions: inverse kinematics and motion planning DRAFTED
  3. 4.3 Inverse dynamics and computed-torque control DRAFTED
  4. 4.4 Where classical methods are still load-bearing in modern robots DRAFTED
  5. 4.5 Summary DRAFTED
  6. 4.x Hands-on exercise + chapter references DRAFTED
5

Learning from rewards: MDPs and reinforcement learning

  1. 5.1 States, actions, rewards, and policies DRAFTED
  2. 5.2 Value iteration and policy iteration DRAFTED
  3. 5.3 Q-learning and the role of exploration DRAFTED
  4. 5.4 Why reward design is the hardest part DRAFTED
  5. 5.5 The MDP-to-robot translation problem DRAFTED
  6. 5.6 Summary DRAFTED
  7. 5.x Hands-on exercise + chapter references DRAFTED
6

Learning from demonstrations: behavior cloning and imitation learning

  1. 6.1 Why imitation is the dominant signal in modern robotics DRAFTED
  2. 6.2 Behavior cloning, step by step DRAFTED
  3. 6.3 Compounding error and DAgger DRAFTED
  4. 6.4 A glance at IRL and adversarial imitation DRAFTED
  5. 6.5 Choosing between BC, IRL, and RL DRAFTED
  6. 6.6 Summary DRAFTED
  7. 6.x Hands-on exercise + chapter references DRAFTED
7

Deep RL for control: DQN to SAC and PPO

  1. 7.1 Function approximation: from Q-tables to Q-networks DRAFTED
  2. 7.2 Policy gradients and the variance problem DRAFTED
  3. 7.3 PPO in 100 lines DRAFTED
  4. 7.4 Off-policy actor-critic: DDPG, TD3, SAC DRAFTED
  5. 7.5 Sim-to-real: domain randomization in one slide DRAFTED
  6. 7.6 Summary DRAFTED
  7. 7.x Hands-on exercise + chapter references DRAFTED

PART THREE

Modern building blocks

8

Sequence models meet control

  1. 8.1 The transformer in two pages, for control DRAFTED
  2. 8.2 Decision Transformer: control as conditional sequence modeling DRAFTED
  3. 8.3 Trajectory Transformer and beam-search planning DRAFTED
  4. 8.4 What gets tokenized: states, actions, returns, language DRAFTED
  5. 8.5 Bridge to foundation action models — and to the SSM alternative (RoboMamba) DRAFTED
  6. 8.6 Summary DRAFTED
  7. 8.x Hands-on exercise + chapter references DRAFTED
9

World models and model-based learning

  1. 9.1 What is a world model, really DRAFTED
  2. 9.2 Latent dynamics: RSSM and Dreamer DRAFTED
  3. 9.3 Planning in latent space DRAFTED
  4. 9.4 Video-prediction world models (Genie, V-JEPA) DRAFTED
  5. 9.5 World models vs. VLAs: the architecture debate DRAFTED
  6. 9.6 Summary DRAFTED
  7. 9.x Hands-on exercise + chapter references DRAFTED
  8. 9.7 World Action Models: the world model as the policy backbone — jointly predicting future video and actions (DreamZero / Cosmos 3), why a physics prior generalizes where a semantic prior does not, and how this reframes the §9.5 debate. (Added 2026-08-15; placed after 9.x to avoid renumbering the existing Summary/exercise bullets.) DRAFTED
10

Diffusion and flow models for action generation

  1. 10.1 A 10-minute introduction to diffusion models DRAFTED
  2. 10.2 Diffusion Policy and ACT DRAFTED
  3. 10.3 Flow matching and rectified flow for action DRAFTED
  4. 10.4 Trade-offs: latency, multimodality, smoothness DRAFTED
  5. 10.5 Action-head choices in modern VLAs DRAFTED
  6. 10.6 Summary DRAFTED
  7. 10.x Hands-on exercise + chapter references DRAFTED

PART FOUR

Foundation action models in depth

11

The VLA recipe: from CLIP to RT-1

  1. 11.1 CLIP and the multimodal pretraining moment DRAFTED
  2. 11.2 Language-conditioned imitation: BC-Z, RT-1 DRAFTED
  3. 11.3 Action tokenization: a small idea with large consequences DRAFTED
  4. 11.4 What RT-1 changed and what it did not DRAFTED
  5. 11.5 The data side: when does scale start to pay off DRAFTED
  6. 11.6 Summary DRAFTED
  7. 11.x Hands-on exercise + chapter references DRAFTED
12

Scaling up: RT-2, OpenVLA, and Octo

  1. 12.1 RT-2: a VLM that also outputs actions DRAFTED
  2. 12.2 OpenVLA: an open-source 7B-parameter VLA DRAFTED
  3. 12.3 Octo: a generalist policy with a diffusion head DRAFTED
  4. 12.4 Open X-Embodiment: the dataset that made all of this possible DRAFTED
  5. 12.5 What "emergent" really means in this context DRAFTED
  6. 12.6 Summary DRAFTED
  7. 12.x Hands-on exercise + chapter references DRAFTED
13

Smooth control: π0 and flow-matching action heads

  1. 13.1 The trouble with discrete action tokens DRAFTED
  2. 13.2 π0's architecture, end to end DRAFTED
  3. 13.3 Flow matching as a control objective DRAFTED
  4. 13.4 What π0 can do that earlier VLAs cannot — and the π0.5 → π0.6 → π0.7 lineage (open-world generalization, RL-from-experience, steerable generalist behavior) DRAFTED
  5. 13.5 Open questions in continuous-action foundation models DRAFTED
  6. 13.6 Summary DRAFTED
  7. 13.x Hands-on exercise + chapter references DRAFTED
14

Dual-system architectures: Helix and GR00T N1

  1. 14.1 Why a single forward pass is not always enough DRAFTED
  2. 14.2 Helix: a high-level VLM and a low-level sensorimotor model — through Helix-02's "System 0" whole-body controller (replacing hand-coded locomotion) and leaderless multi-robot coordination DRAFTED
  3. 14.3 GR00T N1 and its N1.5–N1.7 successors: humanoid-flavored dual systems — plus Gemini Robotics-ER as a third family (embodied reasoning + VLA, "Embodied Thinking," Motion Transfer) DRAFTED
  4. 14.4 Latency budgets and real-time control DRAFTED
  5. 14.5 Deployment case studies (Figure Helix-02, GR00T-enabled humanoids, dexterity-focused entrants such as Genesis AI's GENE and RLWRLD's RLDX-1) DRAFTED
  6. 14.6 Gemini Robotics 2: unified whole-body loco-manipulation (a fourth dual-system family; first VLA to run legs, torso, arms, and multi-finger hands under one learned policy, demoed on Apptronik Apollo 2) DRAFTED
  7. 14.7 Summary DRAFTED
  8. 14.x Hands-on exercise + chapter references DRAFTED
15

Datasets, benchmarks, and evaluation

  1. 15.1 What a robot dataset looks like, by example DRAFTED
  2. 15.2 Open X-Embodiment in detail — and what has (and has not) succeeded it: large teleop fleets (AGIBot Genie-1, bimanual YAM, Unitree G1) feeding newer foundation models rather than a single branded dataset DRAFTED
  3. 15.3 LeRobot: Hugging Face's dataset format, hub, and community-contributed robot data DRAFTED
  4. 15.4 Sim benchmarks (LIBERO, CALVIN, RoboCasa, SimplerEnv) and red-team-style suites such as RoboArena DRAFTED
  5. 15.5 Real-robot evaluation: variance, success rate, time-to-completion DRAFTED
  6. 15.6 Building your own evaluation DRAFTED
  7. 15.7 Summary DRAFTED
  8. 15.x Hands-on exercise + chapter references DRAFTED

PART FIVE

Building with action models

16

Fine-tuning a VLA for your robot

  1. 16.1 Picking a base model DRAFTED
  2. 16.2 Building a teleop dataset that does not waste your time DRAFTED
  3. 16.3 LoRA vs. full fine-tuning vs. action-head-only DRAFTED
  4. 16.4 Sim-to-real fine-tuning loops DRAFTED
  5. 16.5 A recipe card for new embodiments DRAFTED
  6. 16.6 Summary DRAFTED
  7. 16.x Hands-on exercise + chapter references DRAFTED
17

Evaluation, safety, and deployment

  1. 17.1 Safety as a layer, not a property DRAFTED
  2. 17.2 Runtime monitors and shielding — VLM-as-monitor patterns (Code-as-Monitor), runtime action authorization, and step-level failure-signal detection (arXiv 2605.30834) DRAFTED
  3. 17.3 A/B evaluation on hardware DRAFTED
  4. 17.4 Logging, alerting, and rollback DRAFTED
  5. 17.5 What we still cannot certify — plus emerging attack surfaces (adversarial patches that hijack VLA chain-of-thought reasoning, sensor-perturbation fragility, and the CVPR 2026 AdvML adversarial-VLA challenge) DRAFTED
  6. 17.6 Summary DRAFTED
  7. 17.x Hands-on exercise + chapter references DRAFTED
  8. 17.7 Training-time attacks: poisoned data, backdoors, and a defense literature that has not caught up — supply-chain exposure in aggregated datasets and public checkpoints, AttackVLA/BackdoorVLA targeted long-horizon triggers, ATAAT's 5%-poisoning result, and why clean-task evaluation is blind to all of it. (Added 2026-08-15; placed after 17.x to avoid renumbering existing bullets.) DRAFTED
18

Open problems and what comes next

  1. 18.1 Generalization across embodiments — including the shift toward whole-body neural control (Helix-02's System 0, and now Gemini Robotics 2's unified loco-manipulation policy) blurring the classical/learned locomotion boundary from Chapter 4 DRAFTED
  2. 18.2 Long-horizon and dexterous tasks — dexterity remains below human level even as dedicated foundation models (GENE, RLDX-1) target it directly DRAFTED
  3. 18.3 Video-pretrained action models — now with a concrete result: V-JEPA 2 → V-JEPA 2-AC, zero-shot pick-and-place from under 62 hours of unlabeled robot video DRAFTED
  4. 18.4 Reasoning + action: LLM chains of thought meet control — no longer speculative; an active subfield (embodied CoT, latent-reasoning VLAs, ERVLA's CoT-dropout at inference, Gemini's Embodied Thinking) DRAFTED
  5. 18.5 What to read next, and how to contribute DRAFTED
  6. 18.6 Summary DRAFTED
  7. 18.x Hands-on exercise + chapter references DRAFTED
  8. 18.7 Does embodied reasoning actually reason? — the BeTTER stress tests (lexical-kinematic shortcuts, composition failure, behavioral inertia), functional vs. faithful reasoning, and the Pinocchio faithfulness critic; answers the question §18.4 left open. (Added 2026-08-15; placed after 18.x to avoid renumbering existing bullets.) DRAFTED
  9. 18.8 Beyond vision: tactile sensing as a foundation-model modality — what a camera cannot see about contact, TouchWorld's predicted tactile subgoals plus high-frequency residual refinement, and why touch has no free corpus. (Added 2026-08-15; placed after 18.x to avoid renumbering existing bullets.) DRAFTED

APPENDICES

A

Linear algebra refresher DRAFTED

B

Probability and information theory DRAFTED

C

PyTorch and JAX primer DRAFTED

D

Setting up a robotics simulator DRAFTED

E

Canonical references (full bibliography, by chapter) DRAFTED

F

VLA model zoo DRAFTED