$π_0$: A Vision-Language-Action Flow Model for General Robot Control

🎯 Pitch

A robot policy that folds laundry from crumples and assembles boxes after watching 10,000 hours of internet video and robot data. It merges a pre-trained vision-language model with a new 'flow matching' action expert, allowing a single model to control seven wildly different robots—single arms to mobile manipulators—at high frequency without per-task engineering.

1. Executive Summary

This paper introduces π₀, a generalist robot policy—a vision-language-action (VLA) flow matching model built on a pre-trained vision-language model (VLM) backbone—that tackles cross-embodiment, dexterous manipulation at scale through a pre-training/post-training recipe. The model combines Internet-scale semantic knowledge from PaliGemma with a novel action expert that generates continuous action distributions via flow matching (producing 50-action chunks at up to 50 Hz), trained on over 10,000 hours of robot data spanning 7 distinct platforms and 68 tasks alongside open-source datasets. In out-of-box evaluation, π₀ outperforms prior VLA baselines such as OpenVLA and Octo by large margins—achieving near-perfect scores on shirt folding and bussing-easy tasks—while a compute-parity version trained for only 160k steps still exceeds all baselines, establishing that the architecture's combination of VLM pre-training and flow-matched action chunking is essential for high-frequency dexterous control. When fine-tuned to complex multi-stage tasks like laundry folding from arbitrary crumpled configurations or building a cardboard box, the full pre-training and post-training recipe attains more than 50% of the maximum score across all tasks, with pre-training providing especially large gains on the hardest tasks, establishing that broad pre-training on diverse, lower-quality data endows the model with recovery behaviors that purely high-quality post-training data cannot provide.

2. Context and Motivation

The Core Problem: Scaling Dexterous Robot Learning to General-Purpose Systems

The fundamental challenge this paper confronts is deceptively simple to state but enormously difficult to solve: how do we build a single robot policy that can control multiple different robot embodiments—single arms, dual arms, mobile manipulators—across dozens of dexterous manipulation tasks, from folding laundry to bussing tables to assembling boxes, while responding to natural language commands? This is the robot learning field's equivalent of building a foundation model: a general-purpose system that absorbs diverse experience and transfers useful capabilities across tasks and embodiments, rather than requiring a separate, carefully engineered solution for every new behavior.

The core tension the paper identifies is between generality and dexterity. Prior work has demonstrated one or the other—broad but crude manipulation across many tasks, or highly dexterous but narrowly specialized policies for individual behaviors—but bringing both together at the same time has remained elusive. The paper argues this requires solving three tightly coupled sub-problems simultaneously:

First, the architecture must handle fundamentally different action spaces and control frequencies. A single-arm UR5e with 7 degrees of freedom running at 20 Hz places very different demands on a model than a dual-arm mobile manipulator with 16 action dimensions running at 50 Hz. Prior VLA models (RT-2, OpenVLA) discretize actions into autoregressive token sequences, following the standard language modeling paradigm. This limits them to low-frequency control and forces all robots into the same discrete action representation, discarding the precision needed for dexterous tasks. The paper explicitly notes that such models "do not support action chunking or high-frequency control" and struggle dramatically when trained on the paper's data mixture (Section VI-A). As the results in Figure 7 show, OpenVLA—a 7B parameter VLA—achieves near-zero performance on tasks like shirt folding and grocery bagging when trained on this diverse mixture.

Second, the training data must span enough diversity to teach broadly applicable physical capabilities while still containing the quality needed for fluent task execution. This is not simply a matter of collecting more data. The paper argues for a qualitative distinction between two types of data, each serving a distinct purpose (Section III, Section V-A):

Pre-training data should be maximally diverse, covering many tasks, embodiments, and object types. Crucially, this data will contain mistakes, recoveries, suboptimal strategies, and varied approaches—and this is a feature, not a bug. The model needs to see what failure looks like and how to recover from it, because purely high-quality demonstrations never show these recovery trajectories. As the paper states: "Training on only high-quality data does not teach the model how to recover from mistakes, since mistakes are rarely seen in such data."
Post-training data should be carefully curated, high-quality demonstrations that teach the model a consistent, efficient strategy for a specific downstream task. This data is narrower and more controlled, showing the desired behavior rather than the full distribution of possible behaviors.

Prior work in robot learning has overwhelmingly used one paradigm or the other—either training on broad datasets for generalization (Octo, RT-2, OpenVLA trained on OXE) or training on small, high-quality datasets for specific dexterous behaviors (ACT, Diffusion Policy on 10-100 demonstrations). The paper's central methodological claim is that you need both in sequence—a pre-training phase that builds broad physical capabilities and recovery behaviors, followed by a post-training phase that teaches fluent task-specific strategies—and that the absence of either in prior work explains their respective limitations.

Third, and most practically challenging: the entire endeavor must be done at scale, with sufficient data, compute, and model capacity that the benefits of pre-training actually materialize. The paper invokes the "emergent abilities" observation from language modeling (Wei et al., 2022), noting that "the full benefits of large-scale pre-training are often not present at smaller scales." This means the research program cannot be meaningfully tested with toy datasets or small models—it requires the full-scale investment to evaluate whether the approach works at all.

Why This Problem Matters

The paper grounds its motivation in two complementary arguments, one practical and one intellectual.

Practical significance: enabling flexible, general-purpose robot systems. Current approaches to deploying robots in real-world settings—factories, warehouses, homes—are overwhelmingly specialized. A robot that stacks bowls typically cannot fold laundry; a robot that buses tables typically cannot assemble boxes. Each new task requires a new engineering effort: data collection, model training, system integration, and validation. The paper draws an explicit analogy to the pre-LLM era of NLP, where specialized systems were built for each language task (translation, summarization, question answering) until general-purpose pre-trained transformers made it possible to handle all of them with a single model, often outperforming the specialized systems. The vision is the same for robotics: if a single policy can be pre-trained once on diverse data and then adapted to individual downstream tasks via relatively lightweight fine-tuning, the marginal cost of adding a new robot capability drops dramatically. The paper demonstrates this concretely in Section VI-C, where π₀ fine-tuned on just 1 hour of data outperforms specialized methods (ACT, Diffusion Policy) trained from scratch on similar amounts of data.

Intellectual significance: testing whether cross-embodiment pre-training actually transfers. Beyond the engineering motivation, the paper asks a fundamental research question: does training on data from diverse robot platforms genuinely help performance on any individual platform, or does the heterogeneity simply dilute the signal? This is the "positive transfer" question, and prior evidence is mixed. Cross-embodiment training has been studied at smaller scales (OXE, Octo), but the paper notes these prior efforts typically involved simpler tasks—"object relocation and rudimentary furniture manipulation (e.g., drawer opening)"—rather than the dexterous, high-frequency behaviors studied here. The paper's experiments provide evidence for positive transfer: in the fine-tuning experiments (Section VI-C, Figure 11), the pre-trained π₀ consistently outperforms π₀ trained from scratch on the same fine-tuning data, with the gap being especially large on harder tasks (items in drawer, paper towel replacement) and at smaller fine-tuning dataset sizes. This suggests the pre-training is learning broadly useful physical primitives—grasping, reaching, applying forces, coordinating two arms—that transfer across embodiments and tasks.

Where Prior Approaches Fall Short

The paper identifies specific, structural limitations in each major category of prior work:

Autoregressive VLA models (RT-2, OpenVLA, TinyVLA). These models fine-tune pre-trained VLMs to output discretized actions as text tokens, inheriting the VLM's language understanding and reasoning. However, the autoregressive discretization paradigm creates two critical bottlenecks:

Frequency limitation: Because each action dimension must be predicted sequentially, and action chunks are not used, these models cannot produce actions at the 20-50 Hz rates needed for dynamic, contact-rich manipulation. The paper notes that OpenVLA, when trained on their data mixture which requires high-frequency control, "struggles on these tasks because its autoregressive discretization architecture does not support action chunks" (Section VI-A).
Action precision limitation: Discretizing continuous joint angles or end-effector poses into token bins necessarily quantizes the action space, losing the fine-grained precision needed for tasks like aligning a towel fold or fitting an egg into a carton slot. This is acceptable for coarse pick-and-place but fails for dexterous manipulation.

A subtler point: the autoregressive VLA design treats actions as a sequence of independent tokens to be predicted, but a 50-step action chunk for a 7-DoF arm represents a 350-dimensional continuous trajectory. The correlations across time steps and across joint axes are lost in a token-by-token prediction scheme, making it difficult to produce physically coherent motion.

Diffusion-based action policies (Diffusion Policy, Octo). These models use diffusion (or flow matching) to model continuous action distributions and support action chunking, addressing the precision and frequency limitations of autoregressive models. Octo, in particular, combines diffusion action generation with a transformer backbone trained on the OXE dataset, making it the closest prior approach to π₀ in spirit. However, the paper identifies two limitations:

Lack of VLM pre-training: Octo and Diffusion Policy train their visual encoders from scratch (or with ImageNet pre-training), missing the rich semantic knowledge, language understanding, and visual reasoning that VLMs acquire from Internet-scale data. The paper's comparison to π₀-small (which also lacks VLM pre-training) versus full π₀ (Figure 9) quantifies this gap: the VLM-initialized model shows dramatically better language following, which translates into better downstream task performance when guided by language commands.
Limited representational capacity at scale: Octo has 93M parameters, designed for efficiency. The paper shows that a "compute parity" version of π₀ trained for only 160k steps—matching or falling below the training budget of OpenVLA and Octo—already outperforms both across all out-of-box tasks (Figure 7), suggesting that capacity matters when the data mixture is sufficiently large and diverse. The π₀-small model (470M parameters), which is still much smaller than full π₀ (3.3B), also outperforms Octo, further supporting the capacity argument.

Specialized dexterous manipulation methods (ACT, Diffusion Policy trained from scratch). These methods train on small datasets (typically 10-100 demonstrations) for individual tasks and can achieve impressive results on narrow domains. However, the paper highlights a fundamental limitation: "Training on only this high-quality data results in a brittle model that does not reliably recover from mistakes" (Section VII). Because the demonstrations in such small datasets are almost entirely successful, the model never sees failure states and never learns corrective behaviors. When deployed in the real world, where unexpected perturbations, object variations, and stochastic dynamics create off-distribution states, these models fail.

The paper's pre-training/post-training recipe is designed specifically to address this: the pre-training data, by virtue of its diversity, includes recovery trajectories and varied strategies. The post-training data teaches the desired fluent behavior. A model that has seen both can, when it deviates from the post-training strategy, fall back on recovery behaviors learned during pre-training. This is directly analogous to how instruction-tuned LLMs can still draw on their broad pre-training knowledge when the fine-tuning distribution doesn't cover a situation.

Scale of prior dexterous manipulation work. The paper makes an explicit quantitative claim about the scale gap: "More dexterous tasks have been studied at a smaller scale, typically with 10s or 100s of training trajectories, equivalent to 10 or less hours" (Section II). In contrast, π₀ is pre-trained on approximately 10,000 hours of data—a 1,000× increase. This is not simply a matter of having more resources; the paper argues that certain phenomena (cross-embodiment transfer, recovery behavior learning, language grounding) may only become visible at this scale, analogous to how certain language model capabilities only emerge beyond a threshold model and data size.

How This Paper Positions Itself

The paper positions π₀ as a prototype and its training framework as an integrative contribution rather than a single algorithmic innovation. The individual components—VLM pre-training, flow matching, action chunking, mixture of experts, pre-training/post-training phases—each have precedents in prior work, and the paper is explicit about this. What is claimed as novel is the specific combination of these ingredients into a single system, operated at a scale that prior robot learning work has not attempted, and evaluated on tasks whose complexity (laundry folding from arbitrary crumpled configurations, box assembly requiring bimanual coordination and retries, table bussing requiring semantic understanding of novel objects) exceeds what has been demonstrated in end-to-end learned policies.

The paper draws an explicit parallel to the standard LLM training pipeline: pre-training on vast, diverse, Web-scale data to build broad capabilities, followed by post-training ("alignment") on curated, high-quality data to induce desirable behavior patterns. This framing serves two purposes. First, it provides a conceptual anchor—readers familiar with LLM training will immediately grasp the two-phase structure. Second, it makes an implicit argument about methodology: if the pre-training/post-training separation was crucial for unlocking LLM capabilities, perhaps it will be equally important for robot foundation models. The paper's experiments provide evidence for this claim: in the most complex tasks (Figure 13), the full pre-training and post-training recipe consistently outperforms both pre-training-only and post-training-only (scratch) ablations, with the gap being especially large on the hardest tasks.

Finally, the paper's choice to call π₀ a "prototype" and to explicitly list its limitations (Section VII)—no comprehensive understanding of data composition, unreliable performance on some tasks, unclear positive transfer boundaries—positions the work as establishing a new experimental paradigm and baseline rather than claiming to have solved the problem. The contribution is the demonstration that scale, architecture, and training recipe can work together to produce a single policy that handles this range of dexterous tasks, opening the door for systematic investigation of the many open questions the paper identifies.

3. Technical Approach

3.1 Reader Orientation

π₀ is a single neural network that takes in camera images, a text command, and the robot's current joint positions, and produces a chunk of 50 future joint commands at up to 50 Hz, enabling it to control multiple different robot bodies across dozens of dexterous tasks. The system solves the problem of unifying broad semantic understanding (from Internet-scale vision-language pre-training) with precise, high-frequency physical control (from a novel flow-matching action generation head) by separating the model into a frozen-ish VLM backbone that processes observations and language, and a smaller, trainable "action expert" that generates continuous actions, connected through shared self-attention layers.

3.2 Big-Picture Architecture (Diagram in Words)

The system has five major components, organized as a single transformer with two weight-sharing regimes:

Observation Encoders — a set of image encoders (from PaliGemma) that convert each camera view into a sequence of visual tokens, plus a linear projection that converts the proprioceptive state vector (joint angles, gripper positions) into embedding tokens.
VLM Backbone — a pre-trained 3B parameter PaliGemma transformer that processes image tokens and language tokens through 18 self-attention layers. This component inherits Internet-scale visual and semantic knowledge and is initialized from PaliGemma's weights.
Action Expert — a separate 300M parameter set of transformer weights (smaller width, smaller MLP) that processes the robot's proprioceptive state token and the noisy action tokens. It uses full bidirectional attention over the action chunk and is initialized from scratch.
Flow Matching Head — an MLP that ingests each noisy action vector with a sinusoidal timestep encoding, projects it to the action expert's embedding dimension, and after the transformer processes it, a linear projection decodes the output into the denoising vector field prediction.
Inference-Time Integrator — a forward Euler ODE solver that starts from pure noise and runs 10 steps of the learned vector field to produce the clean action chunk, reusing cached keys and values from the observation tokens across integration steps.

Information flows as follows: a language command and up to 3 camera images enter the observation encoders → image tokens and language tokens pass through the VLM backbone → the final hidden states become prefix key-value caches → the proprioceptive state is projected and added → 50 noisy action tokens (initialized as Gaussian noise) are projected with timestep encoding and passed through the action expert, which attends to the cached observation prefix and to each other bidirectionally → the action expert outputs the vector field estimate at the current noise level → the ODE solver updates the noisy actions → repeat 9 more times → the final denoised 50-step action chunk is sent to the robot for open-loop execution.

3.3 Roadmap for the Deep Dive

First, the flow matching action generation mechanism — because this is the core departure from prior VLA models and determines what the model actually outputs. Understanding why flow matching is chosen over discretization or standard diffusion establishes the architectural motivation.
Second, the action expert architecture — the mixture-of-experts design that separates VLM processing from action processing, including the attention mask, the timestep conditioning, and the weight sharing decisions. This is where the VLM and robotics components interface.
Third, the training objective and timestep sampling — how the model is supervised, why the timestep distribution is deliberately skewed toward noisier regimes, and what this implies about the difference between action generation and image generation.
Fourth, the observation encoding and input structure — how images, language, and proprioception are combined into a single sequence, including zero-padding and masking for cross-embodiment compatibility.
Fifth, inference-time mechanics — the caching strategy, the integration procedure, the temporal execution pattern, and the latency profile that makes real-time 50 Hz control feasible.
Sixth, the pre-training and post-training recipe — how the two phases differ in data composition, weighting, and purpose, since the architecture serves both phases identically; the training procedure is what determines behavior.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems and architecture paper whose core idea is that a VLM backbone can be extended with a flow-matching action head to produce a single model that controls multiple robot embodiments at high frequency, and that training this model with a pre-training/post-training recipe on large-scale diverse data yields broadly capable policies that can be fine-tuned to complex dexterous tasks.

Flow Matching Action Generation: What the Model Outputs and Why

The fundamental representational problem in robot policy learning is that actions are continuous, multi-dimensional, and temporally correlated. A single-arm robot with 7 degrees of freedom producing 50 time steps of commands outputs a 350-dimensional continuous vector. Prior VLA models (RT-2, OpenVLA) discretize each action dimension into bins and predict them autoregressively as tokens—an approach that inherits the standard language modeling machinery unchanged but destroys both temporal coherence (each step is predicted independently conditioned on previous step tokens) and precision (the continuous action is quantized). The paper's first major design decision is to instead use conditional flow matching, a continuous generative modeling framework that produces the entire action chunk jointly.

The generative modeling task. The model must learn the conditional distribution $p(\mathbf{A}_t \mid \mathbf{o}_t)$ , where:

$\mathbf{A}_t = [\mathbf{a}_t, \mathbf{a}_{t+1}, \ldots, \mathbf{a}_{t+H-1}]$ is an action chunk of $H$ future actions (the paper uses $H = 50$ );
$\mathbf{o}_t = [\mathbf{I}^1_t, \ldots, \mathbf{I}^n_t, \ell_t, \mathbf{q}_t]$ is the observation, consisting of $n$ RGB images (2 or 3, depending on the robot), a language command $\ell_t$ , and the robot's current proprioceptive state $\mathbf{q}_t$ (joint angles);
$\mathbf{a}_{t'} \in \mathbb{R}^d$ is a single action vector with dimensionality $d$ that varies by robot (7 for UR5e, 14 for bimanual systems, up to 18 for the largest mobile manipulator).

The model does not output the action directly. Instead, it outputs a vector field $\mathbf{v}_\theta(\mathbf{A}^\tau_t, \mathbf{o}_t)$ that, when integrated from $\tau = 0$ to $\tau = 1$ , transforms a sample from a simple noise distribution into a sample from the target action distribution.

The flow matching objective. The training loss is:

$\mathcal{L}_\tau(\theta) = \mathbb{E}_{p(\mathbf{A}_t \mid \mathbf{o}_t), \, q(\mathbf{A}^\tau_t \mid \mathbf{A}_t)} \left\| \mathbf{v}_\theta(\mathbf{A}^\tau_t, \mathbf{o}_t) - \mathbf{u}(\mathbf{A}^\tau_t \mid \mathbf{A}_t) \right\|^2$

where:

$\tau \in [0, 1]$ is the flow matching timestep, with $\tau = 0$ corresponding to pure noise and $\tau = 1$ corresponding to clean data;
$p(\mathbf{A}_t \mid \mathbf{o}_t)$ is the true (unknown) distribution of actions given the observation—the distribution the model aims to learn;
$q(\mathbf{A}^\tau_t \mid \mathbf{A}_t)$ is the conditional probability path, a known distribution that interpolates between noise and data, conditioned on a specific clean action chunk $\mathbf{A}_t$ ;
$\mathbf{A}^\tau_t$ is a noisy action chunk sampled from this conditional path—a mixture of the clean action and noise determined by $\tau$ ;
$\mathbf{v}_\theta(\mathbf{A}^\tau_t, \mathbf{o}_t)$ is the neural network's predicted vector field—the direction and magnitude of change it thinks should be applied to the noisy actions to move them toward the clean data;
$\mathbf{u}(\mathbf{A}^\tau_t \mid \mathbf{A}_t)$ is the target vector field derived from the conditional probability path—the ground-truth denoising direction that the model should learn to predict.

What this equation computes in operational terms: During training, for each observation-action pair in the dataset, we sample a timestep $\tau$ , construct a noisy action chunk by blending the clean action with Gaussian noise according to a schedule determined by $\tau$ , feed both the observation and the noisy actions through the network to get a predicted denoising direction, and compute the squared Euclidean distance between this prediction and the true denoising direction. The expectation is estimated via minibatch sampling.

Why flow matching over alternatives:

Versus autoregressive discretization: Flow matching models the entire action chunk jointly in a single continuous space rather than as a sequence of discrete tokens. This preserves the spatial and temporal correlations—every joint at every timestep can influence every other joint at every timestep through the bidirectional attention in the action expert (discussed below). This is critical for producing physically coherent motion, because a joint trajectory that is independently plausible at each timestep may be kinematically impossible or jerky when stitched together.
Versus standard diffusion (DDPM): Flow matching (specifically, the rectified flow / optimal transport variant used here) defines a straight probability path from noise to data. In standard diffusion, the path is curved (the forward process adds noise gradually, and the reverse process must undo this gradual corruption). Flow matching's straight paths mean fewer integration steps are needed at inference time—the paper uses only 10 steps, compared to the 50–1000 steps typical for image diffusion models. This is essential for real-time robot control, where inference must complete in tens of milliseconds.
Versus energy-based models or GANs: Flow matching provides stable training (no adversarial min-max optimization) and explicit likelihood-free density estimation (the vector field directly defines a transport map).

The specific probability path. The paper uses a simple linear-Gaussian (also called optimal transport) conditional probability path:

$q(\mathbf{A}^\tau_t \mid \mathbf{A}_t) = \mathcal{N}(\tau \mathbf{A}_t, (1 - \tau) \mathbf{I})$

where:

$\mathcal{N}(\tau \mathbf{A}_t, (1 - \tau)\mathbf{I})$ is a multivariate Gaussian distribution with mean $\tau \mathbf{A}_t$ and covariance $(1 - \tau)\mathbf{I}$ ;
When $\tau = 0$ , the distribution is $\mathcal{N}(0, \mathbf{I})$ —pure standard Gaussian noise;
When $\tau = 1$ , the distribution is $\mathcal{N}(\mathbf{A}_t, 0)$ —a delta distribution at the clean action;
For intermediate $\tau$ , the mean is a linear interpolation between noise and data, and the variance shrinks linearly.

What this path means physically: A noisy action $\mathbf{A}^\tau_t$ can be constructed by taking the clean action $\mathbf{A}_t$ , scaling it by $\tau$ (so at small $\tau$ the data signal is weak), and adding independent Gaussian noise scaled by $(1 - \tau)$ (so at small $\tau$ the noise dominates). The transition from $\tau = 0$ to $\tau = 1$ represents a straight-line transport in the space of probability distributions, moving mass from the standard Gaussian to the data distribution.

The denoising target. With this path, the target vector field simplifies to:

$\mathbf{u}(\mathbf{A}^\tau_t \mid \mathbf{A}_t) = \mathbf{A}_t - \boldsymbol{\epsilon}$

where $\boldsymbol{\epsilon} \sim \mathcal{N}(0, \mathbf{I})$ is the noise that was added to create the noisy sample. In practice, the network is trained by:

Sampling clean actions $\mathbf{A}_t$ from the dataset and random noise $\boldsymbol{\epsilon} \sim \mathcal{N}(0, \mathbf{I})$ ;
Constructing noisy actions as $\mathbf{A}^\tau_t = \tau \mathbf{A}_t + (1 - \tau)\boldsymbol{\epsilon}$ ;
Training the network to predict $\mathbf{v}_\theta(\mathbf{A}^\tau_t, \mathbf{o}_t) \approx \mathbf{A}_t - \boldsymbol{\epsilon}$ .

The network is therefore learning to isolate the noise component from the noisy signal—it sees a corrupted action and must output the direction pointing from noise toward clean data.

Why this particular path: The linear-Gaussian path has the property that the target vector field is constant (it does not depend on $\tau$ except through the construction of $\mathbf{A}^\tau_t$ ). This makes the learning problem simpler than paths where the target direction changes with the noise level. Additionally, the linear interpolation of means means that at $\tau = 0$ , the optimal prediction is simply $\mathbb{E}[\mathbf{A}_t \mid \mathbf{o}_t] - \boldsymbol{\epsilon}$ , which pushes the noisy sample toward the conditional mean of the action distribution—a natural initial direction. At $\tau$ close to 1, the noise is small, and the model refines toward the specific clean action.

Inference-time integration. At deployment, the model starts with pure noise $\mathbf{A}^0_t \sim \mathcal{N}(0, \mathbf{I})$ and iteratively applies the forward Euler method:

$\mathbf{A}^{\tau + \delta}_t = \mathbf{A}^\tau_t + \delta \, \mathbf{v}_\theta(\mathbf{A}^\tau_t, \mathbf{o}_t)$

where:

$\delta$ is the integration step size, set to $\delta = 0.1$ (requiring 10 steps to go from $\tau = 0$ to $\tau = 1$ );
At each step, the current noisy actions and the (fixed) observation are fed through the network to get the vector field at the current noise level;
The noisy actions are updated by taking a step of size $\delta$ in the direction of the predicted vector field;
After 10 steps, $\mathbf{A}^1_t$ is the predicted clean action chunk.

What the integration computes: Starting from a random guess (pure noise), the model repeatedly asks: "Given what I see and the current guess at the actions, which direction should I move this guess to make it more likely under the true action distribution?" and takes a small step in that direction. The 10-step process is not just refining a single action—it is jointly refining all 50 future actions and all joint dimensions simultaneously, because the bidirectional attention in the action expert allows every action token to influence every other action token's vector field prediction at each step.

The Action Expert: Separating VLM Processing from Action Generation

The second major architectural innovation is the action expert, a separate set of transformer weights dedicated to processing the robot's proprioceptive state and action tokens. This design is motivated by two practical constraints that pull in opposite directions.

Constraint 1: The VLM backbone should be minimally disrupted. The PaliGemma VLM has been pre-trained on billions of image-text pairs to develop rich visual representations and semantic understanding. If the action generation task were trained by simply adding action tokens to the VLM and fine-tuning all weights with a flow matching loss, the VLM's pre-trained representations could be catastrophically disrupted by the very different optimization signal. The action prediction task operates in a continuous, high-dimensional space with a different loss function, and gradients from this loss flowing through the visual encoder could overwrite the general-purpose features that make the VLM valuable in the first place.

Constraint 2: Action generation needs specialized processing. The flow matching procedure requires the model to process noisy versions of the same actions across multiple timesteps, attending bidirectionally across the entire action chunk to ensure global temporal coherence. The VLM backbone, trained with causal (autoregressive) attention and processing discrete text tokens, is not optimized for this pattern of computation. Simply forcing the VLM to handle action tokens in its existing architecture would be inefficient and might not achieve the needed precision.

The mixture-of-experts solution. The paper introduces a second set of transformer weights—the action expert—that is smaller (width 1024 vs. 2048, MLP dimension 4096 vs. 16,384) and processes only the tokens that were not present during VLM pre-training: the proprioceptive state $\mathbf{q}_t$ and the noisy action chunk $\mathbf{A}^\tau_t$ . The two experts interact only through the self-attention layers: when the action expert processes an action token, it computes attention over all tokens in the sequence (image tokens, language tokens, state token, and other action tokens), but uses its own query, key, and value projection weights for its own tokens.

Formally, for each transformer layer, the model has two sets of parameters:

$\theta_{\text{VLM}} = \{\mathbf{W}_Q^{\text{VLM}}, \mathbf{W}_K^{\text{VLM}}, \mathbf{W}_V^{\text{VLM}}, \mathbf{W}_O^{\text{VLM}}, \text{MLP}^{\text{VLM}}\}$ for image and language tokens;
$\theta_{\text{action}} = \{\mathbf{W}_Q^{\text{action}}, \mathbf{W}_K^{\text{action}}, \mathbf{W}_V^{\text{action}}, \mathbf{W}_O^{\text{action}}, \text{MLP}^{\text{action}}\}$ for state and action tokens.

When the transformer layer processes a VLM-routed token (an image patch or language token), it uses $\theta_{\text{VLM}}$ for the self-attention projection and the MLP. When it processes an action-routed token, it uses $\theta_{\text{action}}$ . However, regardless of which expert processes a given token, the attention computation gathers keys and values from all tokens using the key/value projections associated with each source token's expert. This means action tokens can attend to image tokens (using the VLM's key/value projections for those image tokens) and image tokens can attend to action tokens (using the action expert's key/value projections for those action tokens), but each token's own processing path is determined by which expert it belongs to.

Why mixture-of-experts and not a separate network: An alternative would be to encode observations with a frozen VLM and feed the resulting embeddings into a completely separate action generation network (an encoder-decoder design). The paper found that the mixture-of-experts approach—where the VLM and action expert share attention layers but not MLP weights—performs better. The likely reason is that the attention mechanism allows the action expert to dynamically query different parts of the visual and linguistic context at each layer, rather than receiving a single fixed embedding vector. For dexterous tasks, the model may need to attend to the precise location of a shirt corner (fine-grained spatial information present in intermediate VLM features) when deciding how to fold, and this information might be lost in a single bottleneck embedding.

Attention mask design. The paper uses a blockwise causal attention mask with three blocks:

Block 1: $[\mathbf{I}^1_t, \ldots, \mathbf{I}^n_t, \ell_t]$ — the image and language tokens. Within this block, attention is fully bidirectional (every token attends to every other token in the block). However, these tokens cannot attend to tokens in later blocks (the state token or action tokens). This restriction is explicitly designed "to minimize distribution shift from said pre-training" (Appendix B): during VLM pre-training, these tokens never saw proprioceptive states or actions, so preventing them from attending to these new modalities preserves the pre-trained feature distributions.
Block 2: $[\mathbf{q}_t]$ — the proprioceptive state token. This block can attend to Block 1 (so the model can condition its understanding of the robot's configuration on what it sees), but Block 1 cannot attend to it. It cannot attend to Block 3 (the action tokens). This has a specific inference-time motivation: because the model runs multiple flow matching integration steps with the same observation, preventing Block 2 from attending to Block 3 means the state token's keys and values can be cached and reused across integration steps, since they don't depend on the noisy actions that change at each step.
Block 3: $[\mathbf{a}^\tau_t, \ldots, \mathbf{a}^\tau_{t+H-1}]$ — the $H = 50$ noisy action tokens. This block can attend to the full input sequence (Blocks 1, 2, and itself). Within this block, attention is fully bidirectional—every action token attends to every other action token across all timesteps. This bidirectional attention over the temporal dimension is what allows the flow matching process to produce globally coherent action chunks, where the action at timestep $t + 30$ is influenced by what the model predicts for timestep $t + 10$ .

Why blockwise causal and not fully bidirectional everywhere: Fully bidirectional attention across all tokens would be the simplest design, but it creates two problems. First, it maximizes distribution shift for the VLM backbone, since the pre-trained model only saw causal (or blockwise-causal) attention patterns. Second, it prevents the caching optimization that makes 10-step integration feasible in real-time: if the observation tokens depended on the action tokens (which change at each integration step), the entire prefix would need to be recomputed 10 times per inference cycle, multiplying latency by roughly 10. With the blockwise causal mask, only the action tokens (Block 3) need to be reprocessed, while Blocks 1 and 2 are computed once and cached.

Timestep conditioning. The flow matching process requires the model to know the current noise level $\tau$ so it can adjust its denoising prediction accordingly (at high noise, the model should make large, coarse corrections; at low noise, it should make small, precise refinements). The paper incorporates $\tau$ into the action token embeddings through a dedicated MLP:

$\text{embed}(\mathbf{a}^\tau_{t'}, \tau) = \mathbf{W}_3 \cdot \text{swish}\left(\mathbf{W}_2 \cdot \text{concat}\left(\mathbf{W}_1 \cdot \mathbf{a}^\tau_{t'}, \, \phi(\tau)\right)\right)$

where:

$\mathbf{a}^\tau_{t'} \in \mathbb{R}^d$ is a single noisy action vector at future timestep $t'$ (with $d$ up to 18, the maximum action dimension across all robots);
$\phi: \mathbb{R} \to \mathbb{R}^w$ is a sinusoidal positional encoding function (the same type used for position embeddings in the original Transformer), which maps the scalar $\tau$ to a high-dimensional vector that oscillates at different frequencies, making it easy for the network to distinguish different noise levels;
$\mathbf{W}_1 \in \mathbb{R}^{w \times d}$ projects the action to the embedding width $w$ (which is 1024 for the action expert);
$\text{concat}(\cdot, \cdot)$ concatenates the projected action and the timestep encoding into a single vector of dimension $2w$ ;
$\mathbf{W}_2 \in \mathbb{R}^{w \times 2w}$ and $\mathbf{W}_3 \in \mathbb{R}^{w \times w}$ are learned linear projections with a Swish (SiLU) nonlinearity between them;
The output is a vector in $\mathbb{R}^w$ that serves as the input embedding for this action token in the transformer.

What this embedding does: It fuses the noisy action value with information about the current noise level before the token enters the transformer stack. This allows the action expert's self-attention and MLP layers to condition their processing on how noisy the input is. For example, at $\tau$ near 1, the model knows the actions are almost clean and should only make subtle refinements; at $\tau$ near 0, the model knows it's looking at essentially pure noise and needs to hallucinate a plausible action trajectory from scratch based on the observation context.

Why a separate MLP for timestep conditioning instead of simply prepending $\tau$ as an additional token: The token-based approach would require the model to learn to route the timestep information to the action tokens through attention, which adds an unnecessary indirection. By embedding $\tau$ directly into each action token's input representation, the model has immediate, per-token access to the noise level, which is important because the optimal denoising strategy may differ across joints and timesteps.

Action expert parameter count and scaling. The full π₀ model has 3.3 billion parameters: 3 billion from the PaliGemma VLM backbone (initialized from pre-training) and approximately 300 million from the action expert (initialized from scratch). The action expert is downsized relative to the VLM backbone—width 1024 vs. 2048, MLP dimension 4096 vs. 16,384—to reduce the computational cost of the 10 forward passes needed during inference. Since the VLM backbone processes the observation once and caches its keys and values, its larger size does not dominate inference latency; the action expert, which must be re-executed 10 times per inference cycle, is the latency bottleneck, so reducing its size is critical for achieving real-time control.

Training Objective and Timestep Sampling

The full training procedure. The model is trained end-to-end—both the VLM backbone and action expert weights are updated—on the pre-training mixture. The loss is the flow matching objective described above, applied only to the action tokens. The VLM backbone also receives gradients from this loss (the flow matching signal backpropagates through the self-attention layers into the VLM's representations), which is how the VLM adapts its pre-trained visual features to be useful for action prediction.

Timestep sampling distribution. A critical detail is how $\tau$ is sampled during training. The original flow matching papers sample $\tau$ uniformly from $[0, 1]$ . Esser et al. (2024) propose sampling from a logit-normal distribution that emphasizes middle timesteps, arguing that at high $\tau$ the model only needs to learn the identity function (the denoising target is nearly zero) and at low $\tau$ the model only needs to learn the mean of the data distribution (all paths point toward the conditional mean).

The paper argues that action prediction is fundamentally different from image generation in this regard, and designs a distribution that emphasizes low timesteps (high noise levels). The reasoning:

"While it may be relatively easy to predict the mean image conditioned on a text label, predicting the mean action conditioned on a robot observation (i.e., learning $\mathbb{E}[\mathbf{A}_t \mid \mathbf{o}_t]$ ) is a much harder problem; this is because the observation $\mathbf{o}_t$ is very informative in that it should constrain the distribution of possible actions much more than a text label constrains the distribution of possible images."

In other words, a text prompt like "a cat sitting on a couch" is compatible with an enormous variety of images (different cat breeds, couch styles, lighting, angles, compositions), so the conditional mean of the image distribution is not very informative—it's a blurry average. But a robot observation—showing the exact position of objects, the robot's arms, and a specific language command—strongly constrains what the robot should do next. The conditional mean of the action distribution is actually a very useful prediction (it's roughly "what a reasonable robot would do in this situation"), and learning it well requires the model to see many examples of the high-noise regime where the denoising target is dominated by pushing toward this conditional mean.

The specific distribution. The paper samples $\tau$ from a shifted Beta distribution:

$p(\tau) = \text{Beta}\left(\frac{s - \tau}{s}; \, \alpha = 1.5, \, \beta = 1\right)$

with $s = 0.999$ . This distribution has the following properties:

$\tau$ is bounded above by $s = 0.999$ —the model never sees perfectly clean actions during training. This is important because it means the model always has some denoising to do; at $\tau$ exactly 1, the target vector field is zero, which provides no learning signal. By stopping at 0.999, every training sample has a non-zero target, and the model can still handle $\tau = 1$ at inference because the integration step $\delta = 0.1$ means the model goes from $\tau = 0.9$ directly to $\tau = 1.0$ without needing to be accurate at exactly $\tau = 1.0$ .
The Beta distribution with $\alpha = 1.5$ and $\beta = 1$ has its mode at $\tau = s$ (well, at $\tau$ close to $s$ given the shift), but the $\beta = 1$ parameter means the density decays relatively slowly toward lower $\tau$ , putting significant probability mass on noisier regimes.
The distribution is shifted and flipped: $(s - \tau)/s$ maps $\tau = s$ to 0 and $\tau = 0$ to 1. The Beta distribution is evaluated at this transformed value, so $\tau$ near $s$ (low noise) gets probability mass proportional to $\text{Beta}(0; 1.5, 1) = 0$ while $\tau$ near 0 (high noise) gets probability mass proportional to $\text{Beta}(1; 1.5, 1) = 1.5$ . The actual density is shifted and compressed, but the qualitative effect is that lower timesteps (noisier actions) are sampled more frequently than under a uniform distribution.

Why this choice matters: If the model were trained with a uniform distribution over $\tau$ , it would spend most of its capacity on the low-noise regime where the denoising task is easy (small corrections to nearly-correct actions). By upweighting the high-noise regime, the model is forced to learn the harder problem of predicting plausible actions from essentially pure noise, which requires extracting maximum information from the observation. The paper argues this leads to better performance, especially for dexterous tasks where the action distribution is complex and the model needs strong conditioning on the visual input.

Relationship to the action horizon $H$ and control frequency. The model predicts $H = 50$ future actions. For robots running at 50 Hz, this corresponds to 1 second of control; for robots at 20 Hz, this corresponds to 2.5 seconds. The action chunk is executed open-loop: the model runs inference, produces 50 actions, the robot executes them sequentially, and then the model runs inference again (whether or not all 50 actions have been executed).

Observation Encoding and Cross-Embodiment Input Handling

The model must accept observations from 7 different robot configurations with varying numbers of cameras, different action dimensionalities, and different kinematic structures. The paper addresses this through a combination of zero-padding, masking, and linear projection.

Image encoding. Each camera image $\mathbf{I}^i_t \in \mathbb{R}^{H \times W \times 3}$ is passed through a Vision Transformer (ViT) encoder from PaliGemma, which divides the image into fixed-size patches, linearly embeds each patch, and processes them through a series of self-attention layers. The output is a sequence of patch tokens (plus a CLS token, depending on the ViT variant) that serve as the image's contribution to the transformer sequence. The paper uses PaliGemma's pre-trained ViT weights and continues to update them during training (they are not frozen).

Proprioceptive state encoding. The robot's current configuration $\mathbf{q}_t$ is a vector containing joint angles, gripper positions, and for mobile manipulators, base pose and torso height. The dimensionality of this vector varies by robot (7 for UR5e, 14 for bimanual systems, up to 18 for the mobile Fibocom with holonomic base and vertical torso). To handle this heterogeneity, the paper zero-pads all state vectors to the maximum dimensionality across all robots (18 dimensions in their setup). Similarly, action vectors $\mathbf{a}_{t'}$ are zero-padded to the same maximum dimensionality.

The padded state vector is projected to the transformer's embedding dimension using a single learned linear projection $\mathbf{W}_{\text{state}} \in \mathbb{R}^{w \times 18}$ , where $w = 1024$ for the action expert (since the state token is routed to the action expert). The zero-padding ensures that for robots with fewer DoFs, the extra dimensions are always zero, and the linear projection learns to ignore these dimensions (the corresponding columns of $\mathbf{W}_{\text{state}}$ are multiplied by zero and their gradients are zero, so they remain at their initialization values or drift slowly).

Image masking for robots with fewer cameras. Robots in the dataset have either 2 or 3 cameras. The model always expects 3 image slots, but for robots with only 2 cameras, the third image slot is "masked out"—meaning a learned mask token (or simply zero embedding) is inserted, and the attention mask is configured so that other tokens can still attend to it but the model learns that it contains no information. The paper states: "For robots with fewer than three images, we also mask out the missing image slots."

Language encoding. The language command $\ell_t$ is tokenized using the PaliGemma tokenizer and embedded using the VLM's token embedding matrix. This is exactly the same pathway as during VLM pre-training, preserving compatibility with the pre-trained weights.

Sequence construction. The full input sequence to the transformer is (using ; to denote concatenation along the sequence dimension):

$[\mathbf{I}^1_t, \ldots, \mathbf{I}^n_t, \ell_t, \mathbf{q}_t, \mathbf{a}^\tau_t, \ldots, \mathbf{a}^\tau_{t+H-1}]$

where $\mathbf{I}^i_t$ is actually a sequence of patch tokens, $\ell_t$ is a sequence of language tokens, and $\mathbf{q}_t$ and each $\mathbf{a}^\tau_{t'}$ are single tokens (each is a vector projected to the embedding dimension, producing one token per time step). The total sequence length varies depending on the image resolution (number of patches per image) and the length of the language command, but the action chunk always contributes exactly $H = 50$ tokens.

Why zero-padding and masking, not separate model heads or architectures for each robot: The alternative would be to have robot-specific input encoders or even separate models for each embodiment. The paper explicitly avoids this because the goal is a single model that can transfer knowledge across embodiments. If each robot had its own encoder, there would be no mechanism for the model to learn that "two 6-DoF arms" is a similar concept across the bimanual ARX, bimanual Trossen, and bimanual UR5e platforms. By forcing all robots through the same input projections (with zero-padding to a common dimensionality), the model can discover shared structure—for example, that the first 7 dimensions of a 14-DoF bimanual system behave similarly to the 7 dimensions of a single-arm UR5e, and that the remaining 7 dimensions represent the second arm.

Inference-Time Mechanics

The inference procedure must be fast enough to support real-time control at up to 50 Hz while generating high-quality action chunks. The paper achieves this through careful attention caching and a specific execution pattern.

Two-phase inference. Each time the model runs inference to produce a new action chunk, the computation is divided into two phases:

Phase 1: Observation encoding and prefix caching. The camera images are encoded (14 ms for 3 images on an RTX 4090) and the observation prefix (image tokens, language tokens, state token) is passed through the transformer (32 ms). This produces key-value caches for all layers of both the VLM backbone and action expert for every token in Blocks 1 and 2 of the attention mask. Since Blocks 1 and 2 cannot attend to Block 3 (the action tokens), their keys and values do not depend on the noisy actions and can be saved.

Phase 2: Iterative action denoising. Starting from $\tau = 0$ with $\mathbf{A}^0_t \sim \mathcal{N}(0, \mathbf{I})$ , the model runs 10 integration steps. At each step:

The 50 noisy action tokens are embedded with the current $\tau$ timestep conditioning;
These tokens are passed through the action expert only (the VLM backbone does not need to be recomputed because its outputs are cached and don't depend on the actions);
The action expert's attention layers compute queries from the action tokens and attend to the cached keys and values from the observation prefix, as well as to the keys and values of other action tokens;
The output of the transformer is decoded into the vector field prediction $\mathbf{v}_\theta(\mathbf{A}^\tau_t, \mathbf{o}_t)$ ;
The noisy actions are updated using forward Euler: $\mathbf{A}^{\tau + 0.1}_t = \mathbf{A}^\tau_t + 0.1 \cdot \mathbf{v}_\theta(\mathbf{A}^\tau_t, \mathbf{o}_t)$ ;
$\tau$ is incremented by 0.1.

This phase takes 27 ms total (2.7 ms per integration step).

Total latency. The numbers from Table I (Appendix D), measured on an NVIDIA GeForce RTX 4090:

Image encoders: 14 ms
Observation forward pass: 32 ms
10 × action forward pass (flow): 27 ms
Total on-board inference: 73 ms
Network latency (if off-board): 13 ms
Total off-board inference: 86 ms

Why caching is critical: Without the blockwise causal mask and caching strategy, each integration step would require re-computing the entire transformer forward pass (both VLM backbone and action expert) over the full sequence length. At 73 ms for a single full forward pass, 10 integration steps would take 730 ms, which is far too slow for real-time control (the robot would be stationary for nearly a second between action chunks). With caching, the expensive observation processing is amortized: it's done once per inference cycle, and the integration steps only need to process the 50 action tokens through the smaller action expert.

Temporal execution pattern. The model generates $H = 50$ future actions at each inference cycle, but the paper does not always wait for all 50 actions to execute before running inference again. Instead:

For UR5e and Franka robots (20 Hz control): inference runs every 0.8 seconds, after executing 16 actions (out of the 50 in the chunk). This means there is significant overlap between consecutive action chunks—the last 34 actions of the previous chunk are never executed because a new chunk is generated before they are reached.
For all other robots (50 Hz control): inference runs every 0.5 seconds, after executing 25 actions. Similarly, only the first half of each action chunk is used before the next chunk is generated.

Why not execute the full chunk? Executing actions open-loop for the full 1-second horizon (50 steps at 50 Hz) without replanning would mean the robot is blind to new observations during that second. For dexterous tasks involving contact, deformable objects, and precise alignment, 1 second of open-loop control is a long time—the actual state of the world can diverge significantly from what was predicted. By replanning every 0.5 seconds, the model can incorporate new visual feedback and adjust its actions mid-execution, improving robustness to perturbations and modeling errors.

Why no temporal ensembling? Action chunking methods like ACT typically use temporal ensembling, where actions at a given timestep are predicted by multiple overlapping chunks and averaged to produce smoother trajectories. The paper explicitly tested this: "We tried temporal ensembling early on and found that it hurt policy performance, so we opted not to aggregate actions and instead execute action chunks open-loop." The likely reason is that the flow matching model already produces temporally coherent chunks (due to bidirectional attention over the full horizon), so ensembling across chunks introduces averaging artifacts that reduce the precision needed for dexterous tasks.

Pre-Training and Post-Training Recipe

While the architecture is the same for both phases, the data, training duration, and purpose differ fundamentally between pre-training and post-training.

Pre-training data composition. The pre-training mixture consists of two major sources:

The π dataset: 903 million timesteps (approximately 10,000 hours) of dexterous manipulation data collected across 7 robot configurations and 68 tasks. This data includes:
- Single-arm tasks: UR5e (106M steps), Franka (included in the 903M total)
- Dual-arm tasks: bimanual UR5e, bimanual Trossen, bimanual ARX/AgileX (797M steps total for all dual-arm platforms)
- Mobile manipulation: Mobile Trossen, Mobile ARX, Mobile Fibocom (included in the dual-arm total)
Open-source datasets: OXE Magic Soup (a curated subset of the Open X-Embodiment dataset), Bridge v2, and DROID. These contribute 9.1% of the training mixture and include data from 22 robots with 1–2 cameras at 2–10 Hz control frequencies.

Each training example is a single timestep tuple $(\mathbf{o}_t, \mathbf{A}_t)$ , so the 903M timesteps represent 903 million individual training examples.

Data weighting. The datasets are imbalanced—for example, laundry folding tasks are overrepresented relative to other tasks. To prevent the model from overfitting to the most common task-robot combinations, the paper applies per-combination weighting:

$w_i \propto n_i^{0.43}$

where $n_i$ is the number of timesteps for a given task-robot combination $i$ , and $w_i$ is the sampling weight. The exponent 0.43 is chosen so that overrepresented combinations are down-weighted but not completely flattened: if combination A has 100× more data than combination B, it will be sampled $(100)^{0.43} \approx 7.2$ times more often, not 100 times more often. This preserves the benefits of having more data for certain tasks while preventing the model from ignoring rare task-robot combinations entirely.

Why 0.43? The paper does not justify this specific exponent, but values in the range 0.4–0.5 are common in large-scale language model training for dataset rebalancing (e.g., in The Pile and other web-scale datasets). The principle is that the marginal value of additional examples from the same distribution decreases, so the sampling probability should increase sublinearly with dataset size.

Pre-training purpose. The pre-training phase trains the full model (both VLM backbone and action expert) on this diverse mixture for 700,000 steps. The goal is to build a base model that:

Has seen a wide variety of objects, scenes, and physical interactions;
Can follow language commands for tasks present in the pre-training data;
Has acquired recovery behaviors and corrective strategies from the lower-quality, more varied demonstrations;
Can serve as a starting point for fine-tuning to new tasks.

Post-training data and purpose. The post-training phase fine-tunes the pre-trained model on a smaller, task-specific dataset of high-quality demonstrations. The amount of data varies dramatically:

"Easier" tasks (stack bowls, towel folding): 1–10 hours of data;
"Harder" tasks (items in drawer, paper towel replacement): 5–10 hours;
Complex multi-stage tasks (laundry folding, table bussing, box building): up to 100+ hours of curated, high-quality data.

The purpose of post-training is to teach the model a consistent, efficient, and fluent strategy for the specific task. The paper argues that training only on this high-quality data (without pre-training) produces brittle models that fail when they deviate from the demonstrated trajectory because they've never seen mistakes or recoveries. Conversely, running the pre-trained model zero-shot on complex tasks often doesn't achieve the fluent, human-like strategies that the high-quality data captures. The combination—broad pre-training followed by focused post-training—is what produces the best results.

How post-training differs from simply training on all data together. A natural question is why not simply include the high-quality data in the pre-training mixture and train a single model? The paper's implicit argument (drawn from the LLM training literature) is that the optimization dynamics differ: during pre-training on diverse data, the model is forced to learn broadly applicable features because no single task dominates. If high-quality data were included from the start, the model might overfit to those specific strategies and fail to develop the general recovery capabilities that come from seeing diverse, imperfect demonstrations. The two-phase approach is analogous to instruction tuning in LLMs, where the base model is pre-trained on broad data and then fine-tuned on a smaller, curated instruction dataset.

The role of language in both phases. The pre-training data includes diverse language labels—both task names and fine-grained segment annotations (approximately 2-second sub-trajectories with language descriptions). These segment annotations are important because they provide the model with grounded language for sub-behaviors (e.g., "grasp the shirt collar," "fold the sleeve inward"), which the model can later recombine when following new language commands. The post-training data includes language commands appropriate to the target task, allowing the model to be prompted with language during evaluation.

4. Key Insights and Innovations

Innovation 1: The Two-Phase Pre-Training/Post-Training Recipe as a Resolution to the Robustness-Dexterity Tension

The paper's most conceptually distinctive contribution is not any single architectural choice but the recipe itself: the explicit separation of training into a pre-training phase on diverse, lower-quality data and a post-training phase on narrow, high-quality data, with each phase serving a qualitatively different purpose. This framing resolves a long-standing tension in robot learning that the paper diagnoses: training on broad but crude data produces policies that can handle variety but lack fluent execution, while training on narrow but expert-quality data produces fluent policies that are brittle to perturbations because they've never seen mistakes or recoveries.

Prior to this work, the field largely treated data quality as a monotonic variable—higher quality is better, and the challenge is simply collecting enough of it. Methods like ACT and Diffusion Policy demonstrated impressive dexterity from as few as 10–100 expert demonstrations, implicitly suggesting that data quality could substitute for quantity and diversity. Conversely, models like Octo and RT-2/OpenVLA demonstrated broad generalization by training on large multi-task datasets (OXE), but struggled with the precise, dynamic behaviors that dexterous tasks require—a tradeoff the paper's baselines confirm.

What makes the paper's framing distinctive is the argument that low-quality data is not merely a budget compromise but a positive requirement for robustness. The intuition, stated in Section V-A, is that "the diverse (but lower quality) pre-training data allows the model to recover from mistakes and handle highly varied situations, which might not otherwise occur in the high-quality post-training data, while the post-training data teaches the model to perform the task well." This is not a claim that more data is better—it's a claim that the distribution of behaviors matters, and that a dataset containing only successful trajectories creates a model that has no concept of what failure looks like or how to escape it.

This insight draws a direct parallel to LLM training that goes deeper than the surface-level analogy. In language models, pre-training on Web-scale data endows the model with broad knowledge but not instruction-following ability; post-training (instruction tuning, RLHF) teaches the model to channel that knowledge into desirable behaviors. The paper argues robot learning follows the same pattern, with the crucial addition that robot pre-training data contains physical recoveries that constitute a repertoire of corrective behaviors—analogous to an LLM seeing diverse reasoning patterns including erroneous ones during pre-training, which later enables it to recognize and correct mistakes.

The evidence for this claim is strongest in Figure 13, where the full pre-training + post-training recipe consistently outperforms both pre-training-only (out-of-box) and post-training-only (scratch) ablations across all complex tasks, with the gap being especially dramatic on the hardest tasks (box building, where the model needs to retry folds, and table bussing, where it must recover from misgrasps on novel objects). The scratch model, trained only on high-quality data for these specific tasks, achieves systematically lower scores—consistent with the claim that it lacks the recovery behaviors that pre-training provides.

This is a fundamental reframing, not an incremental improvement. It changes the research question from "how do we collect better data for task X" to "what mixture of pre-training data creates the right base of physical capabilities, and how do we then channel those capabilities with post-training." The paper explicitly positions this as the "most important ingredient" in the introduction: "much of the recent progress with large models in NLP and computer vision has relied heavily on delicate strategies for curating pre-training and post-training data." The contribution is importing this strategic perspective into robot learning and providing the first large-scale empirical validation that the separation matters for physical tasks.

Innovation 2: Flow Matching as the "Right" Action Representation for Unifying VLM Semantics with Dexterous Control

The paper's architectural thesis is that flow matching over continuous action chunks is the enabling representation that allows a VLM backbone to control high-frequency dexterous manipulation without destroying the VLM's pre-trained representations. This is a diagnostic contribution—identifying why prior VLA models fail on dexterous tasks, and what specific property of the action representation determines success or failure.

Prior VLA models (RT-2, OpenVLA, TinyVLA) inherited the standard language modeling paradigm whole: discretize continuous actions into bins, predict them autoregressively as a sequence of "action tokens," and train with cross-entropy loss. This approach has a compelling simplicity—it reuses the entire VLM machinery unchanged—but the paper identifies a structural failure mode: the discretization and autoregressive factorization make it impossible to produce the high-frequency, temporally coherent action chunks that dexterous tasks demand. OpenVLA, a 7B parameter model, achieves near-zero performance on shirt folding and grocery bagging when trained on the paper's diverse mixture (Figure 7), confirming that this is not a capacity issue but a representation problem.

The flow matching alternative addresses this at the conceptual level by treating the action chunk as a joint, continuous entity. Rather than predicting one action dimension at a time conditioned on previous predictions, flow matching models the entire 50-step trajectory as a single sample from a continuous distribution, with bidirectional attention enabling every joint at every timestep to influence every other joint at every timestep. The paper's contribution is not flow matching itself (which originates in Lipman et al., 2022, and Liu, 2022, and has been used for image generation in Stable Diffusion 3 and others), but rather the argument that flow matching is the natural interface between VLM-scale semantic understanding and dexterous physical control, because it preserves continuous precision while being compatible with a VLM backbone that was trained for a different modality.

The significance of this framing extends beyond performance numbers. It establishes a design principle: the action representation is not a cosmetic choice but determines whether pre-trained semantic knowledge can transfer to physical control at all. An autoregressive discretization forces the model to break actions into semantically meaningless tokens that bear no relationship to the visual and linguistic concepts the VLM learned during pre-training. A continuous generative model, by contrast, allows the VLM's visual features—which encode spatial relationships, object boundaries, and semantic categories in a continuous embedding space—to directly condition a continuous action distribution without an intermediate quantization step that would discard the very precision that makes dexterous control possible.

The evidence for this framing comes from the architecture's performance profile. In Figure 7, the compute-parity π₀ model (160k steps) dramatically outperforms OpenVLA despite comparable or lower training budgets, and this gap is largest on the tasks requiring the most dexterous manipulation (shirt folding, grocery bagging). The π₀-small model—which uses flow matching but lacks VLM pre-training—also outperforms OpenVLA, suggesting flow matching alone provides substantial benefits independent of scale. The full π₀ model combines both, and the gap between π₀ and π₀-small in Figure 9 quantifies the VLM-specific contribution to language following.

This is a fundamental design contribution rather than an incremental improvement over prior VLAs. It doesn't refine the discretization scheme or add a small continuous correction term—it replaces the entire action prediction paradigm with a different generative framework, motivated by a specific diagnosis of what makes dexterous control hard. The paper's explicit claim that this is "to our knowledge the first flow matching VLA that produces high-frequency action chunks for dexterous control" (Section II) underscores the novelty of the combination, even though each component has precedents in isolation.

Innovation 3: The Action Expert as a Mechanism for Preserving VLM Pre-Training Under a Radically Different Training Objective

The mixture-of-experts design with a separate action expert—300M parameters that process robot-specific tokens while sharing attention with the 3B parameter VLM backbone—embodies a specific architectural hypothesis: that you can extend a pre-trained foundation model to a fundamentally different output modality without catastrophic forgetting, by isolating the new modality's processing into dedicated weights that interact with the pre-trained weights only through attention, never through shared MLP transformations.

This hypothesis matters because the alternative—fine-tuning the entire VLM on the flow matching objective without architectural separation—risks overwriting the precisely tuned visual and semantic representations that the VLM acquired during Web-scale pre-training. The flow matching loss is mathematically very different from the next-token prediction loss the VLM was trained on: it operates on continuous vectors rather than discrete token probabilities, it requires processing noisy inputs at multiple noise levels, and it emphasizes fine-grained spatial reasoning (where exactly is the shirt corner?) over the high-level semantic categorization that image-text pre-training emphasizes.

The action expert resolves this tension through a specific form of parameter isolation with shared attention. The key design choice is not just having separate weights—which is standard in mixture-of-experts architectures—but specifically sharing only the attention mechanism while keeping the feedforward (MLP) processing separate. Attention layers are where tokens from different modalities exchange information; the MLP layers are where modality-specific processing happens. By keeping the VLM's MLP weights intact and routing image/language tokens through them, while establishing a parallel set of MLP weights for action tokens, the model allows the action expert to learn to extract task-relevant spatial and temporal features while the VLM backbone retains its pre-trained visual understanding.

This is a fundamental architectural insight because it addresses a problem that will recur as foundation models are adapted to increasingly diverse output modalities. The same tension between preserving pre-trained knowledge and learning new capabilities arises when extending LLMs to code, to tool use, or to agentic planning. The action expert pattern—separate processing pathways with attention-level integration—is not specific to robot actions and could generalize to other settings where a foundation model must be extended without destroying its pre-training.

The evidence for this design's importance is somewhat indirect (the paper does not ablate the action expert vs. a fully shared-weight alternative), but the strong performance of π₀ over π₀-small (Figure 7, Figure 9) provides circumstantial support: π₀-small, which does not have a VLM backbone and therefore doesn't face the catastrophic forgetting problem, underperforms π₀ despite having similar architecture in other respects, suggesting that the action expert successfully preserves something valuable from the VLM pre-training that π₀-small cannot replicate.

Innovation 4: Positive Transfer at Scale as an Empirical Discovery, Not an Assumption

Perhaps the most consequential finding in the paper is not a method but an empirical result: that training on diverse cross-embodiment data genuinely improves performance on individual tasks, and that this positive transfer grows stronger for harder tasks and smaller fine-tuning datasets. This finding contradicts a reasonable null hypothesis—that training on data from different robots with different kinematics, different camera configurations, and different action spaces would produce interference that degrades performance on any single platform.

Prior cross-embodiment work (OXE, Octo) demonstrated that broad training can produce policies that work at a basic level across many embodiments, but it left open the crucial question: does this broad training help or hurt when you then specialize to a particular downstream task? The concern is that the model might learn embodiment-specific features that interfere—for example, strategies for a holonomic mobile base that don't apply to a fixed-base arm—and that these could create negative transfer when fine-tuning. The paper provides the first large-scale evidence that positive transfer dominates in this regime, at least for the types of dexterous tasks studied.

The evidence is in Figure 11: across five fine-tuning tasks spanning different difficulty tiers, the pre-trained π₀ consistently outperforms π₀ trained from scratch on the same fine-tuning data. The effect is not uniform—it's strongest for tasks that are most similar to the pre-training distribution (towel folding, which is similar to shirt folding in pre-training) and for smaller fine-tuning dataset sizes (1 hour vs. 10 hours). At 1 hour of fine-tuning data, the pre-trained model often doubles the performance of the scratch model. At 10 hours, the gap narrows but remains positive. This pattern is exactly what positive transfer predicts: pre-training provides a stronger prior when data is scarce, and its advantage diminishes as task-specific data accumulates.

This finding has two implications beyond the specific performance numbers:

It validates the foundation model approach for dexterous manipulation. If cross-embodiment pre-training produced negative transfer, the entire research program—collecting massive multi-robot datasets, training a single model on heterogeneous data, and fine-tuning to downstream tasks—would be fundamentally misguided. The paper shows it's not. This shifts the burden of proof: future work must now show when and why negative transfer occurs, rather than assuming it as a default.

It establishes that physical capabilities can transfer across embodiments. The paper shows that a model trained partly on bimanual ARX data can, after fine-tuning on a Franka arm, perform better on a Franka-specific task than a model trained only on Franka data. This suggests the model is learning something abstract—grasping strategies, contact reasoning, recovery behaviors—that transcends the specific kinematic structure of any one robot. The paper is appropriately cautious about how far this transfer extends (Section VII explicitly questions whether it reaches "much more distinct domains, such as autonomous driving, navigation, and legged locomotion"), but the in-distribution cross-embodiment transfer is clearly positive.

This is a fundamental empirical contribution, not an algorithmic one. It doesn't introduce a new method for achieving transfer—it demonstrates that transfer happens under a specific training recipe at a specific scale, providing evidence that the field's investment in large-scale multi-robot datasets is justified and that the foundation model paradigm can succeed in robot learning.

5. Experimental Analysis

Evaluation Methodology

Dataset. The primary dataset used for pre-training is the paper's own π dataset, consisting of 903 million timesteps (approximately 10,000 hours) of dexterous manipulation data collected across 7 robot configurations and 68 distinct tasks, as described in Section V-C. This is augmented with open-source datasets: OXE Magic Soup (a curated subset of the Open X-Embodiment dataset), Bridge v2, and DROID, which collectively contribute 9.1% of the pre-training mixture (Figure 4). The pre-training mixture is weighted per task-robot combination by n^{0.43}, where n is the number of timesteps for that combination, to prevent overrepresentation of the most frequent tasks. Post-training fine-tuning uses task-specific datasets ranging from 1 hour (for simpler tasks like stack bowls) to over 100 hours (for complex tasks like laundry folding and table bussing). Evaluation is conducted on held-out episodes for each task, with specific scoring rubrics defined in Appendix E. There is no single static test set; instead, each task has its own evaluation protocol with 10 trials per method per task.
Base model. The primary model is π₀, a 3.3 billion parameter vision-language-action model initialized from PaliGemma (a 3B parameter VLM; Beyer et al., 2024) with an additional 300M parameter action expert trained from scratch. PaliGemma is chosen for its "convenient trade-off between size and performance" and because its comparatively small size is "useful for real-time control" (Section IV). An additional variant, π₀-small (470M parameters, described in Appendix C), is used as a non-VLM ablation—it uses DistilBERT for language encoding, a smaller ViT encoder, and a DiT-style action expert, and is trained from scratch without VLM initialization. For the FLOPs-matched comparisons in Section VII, a model with approximately ~14× more parameters is used as the pretraining-scaled baseline, though its architecture is not further specified.
Metrics. The primary metric across all experiments is average task progress, a normalized score between 0.0 and 1.0 where 1.0 represents perfect task completion and fractional scores represent partial completion. Each task has a custom rubric (detailed in Appendix E) that awards points for completing subtask milestones. For example, the bussing task is scored as the fraction of objects correctly placed in the proper receptacle; shirt folding is binary (success/failure); laundry folding is scored out of 4 based on picking, flattening, folding, and stacking. All results are averaged over 10 episodes per task per method, unless otherwise noted.
Baselines. The paper compares against several prior models, each representing a distinct category of approach:
- OpenVLA (Kim et al., 2024): A 7B parameter autoregressive VLA model originally trained on the OXE dataset. The paper trains OpenVLA on the full pre-training mixture (160k steps) and also trains a variant on only the UR5e subset of data (without cross-embodiment training) to provide a stronger baseline on UR5e tasks. OpenVLA "does not support action chunking or high-frequency control" (Section VI-A).
- Octo (Octo Model Team, 2024): A 93M parameter generalist robot policy that uses a diffusion process to generate actions. Octo is trained on the same pre-training mixture as π₀ (320k steps). It supports action chunks but has limited representational capacity compared to π₀.
- ACT (Zhao et al., 2023): An action chunking method using a CVAE-style architecture designed for learning dexterous tasks from small datasets. ACT is trained from scratch only on the fine-tuning datasets, not on the pre-training mixture.
- Diffusion Policy (Chi et al., 2023): A diffusion-based visuomotor policy trained from scratch on individual task datasets. Like ACT, it represents the specialized dexterous manipulation approach without any pre-training.
- π₀-small: As described above, a 470M parameter model trained from scratch on the full pre-training mixture, serving as an ablation for VLM pre-training.
- π₀ (scratch): The full π₀ architecture but trained only on the fine-tuning dataset for a specific task, with no pre-training, serving as an ablation for the pre-training phase.
Generation budget / compute accounting. The paper does not report training compute in FLOPs or GPU-hours. Instead, training is measured in number of training steps, with the main π₀ model trained for 700,000 steps on the pre-training mixture. For fair comparison against baselines, a compute-parity version of π₀ is trained for only 160,000 steps, which is "equal to or lower than the number of steps provided to the baselines (160k for OpenVLA, 320k for Octo)" (Section VI-A). The amount of fine-tuning data for post-training is measured in hours of demonstration data (1, 5, or 10 hours), with the same dataset sizes used across all compared methods for a given task. Inference compute is reported only in terms of milliseconds of latency on an NVIDIA GeForce RTX 4090 (Table I, Appendix D), with a breakdown by model component: 14 ms for image encoding, 32 ms for the observation forward pass, 27 ms for 10 flow matching integration steps (2.7 ms each), for a total of 73 ms on-board or 86 ms off-board (including 13 ms network latency). No FLOP counts are reported for inference.
Cross-validation / statistical protocol. The paper does not employ cross-validation or report confidence intervals, standard errors, or statistical significance tests. Results are reported as averages over 10 trials per task per method, and error bars are not shown in the figures (Figures 7, 9, 11, 13). The paper does not describe any procedure for selecting optimal checkpoints on a validation split; the language about two-fold cross-validation from the prior sections does not appear in the actual experimental evaluation, which appears to use fixed evaluation protocols without data splitting.

Main Quantitative Results

Out-of-Box Evaluation: Pre-Trained Model Performance (Section VI-A)

The headline result is that π₀, evaluated directly after pre-training without any task-specific fine-tuning, achieves substantially higher performance than all baselines across five diverse tasks. Full results are reported in Figure 7.

Headline numbers. The full π₀ model (700k training steps) achieves:

Shirt folding (Bi-ARX): Near-perfect score of approximately 1.0 (binary success/failure across 5 clothing items × 2 trials each);
Bussing easy (UR5e): Approximately 0.95 average task progress (scored out of 7 objects correctly sorted);
Bussing hard (UR5e): Approximately 0.55 average task progress (scored out of 12 objects, with challenging configurations);
Grocery bagging (UR5e): Approximately 0.7 average task progress (scored out of 7 items placed in bag);
Toast out of toaster (Bi-Trossen): Approximately 0.75 average task progress (scored out of 4, with 1 point for picking each piece and 1 point for placing on plate).

Comparison against baselines. At matched training budgets, the compute-parity π₀ (160k steps) outperforms all baselines across every task:

On shirt folding, π₀ (parity) achieves approximately 0.85 vs. OpenVLA at approximately 0.0 (the baseline essentially fails entirely on this task) and Octo at approximately 0.1.
On bussing easy, π₀ (parity) achieves approximately 0.85 vs. OpenVLA at approximately 0.1 and Octo at approximately 0.25.
On bussing hard, π₀ (parity) achieves approximately 0.4 vs. OpenVLA at approximately 0.0 and Octo at approximately 0.05.
On grocery bagging, π₀ (parity) achieves approximately 0.55 vs. OpenVLA at approximately 0.0 and Octo at approximately 0.1.
On toast, π₀ (parity) achieves approximately 0.6 vs. Octo at approximately 0.3 (OpenVLA is not shown for this task in Figure 7).

The UR5e-only OpenVLA variant (trained without cross-embodiment data) improves over standard OpenVLA on the UR5e tasks (bussing easy, bussing hard, grocery bagging) but remains far below π₀: on bussing easy it reaches approximately 0.4 vs. π₀'s approximately 0.95. π₀-small (no VLM pre-training, 470M parameters) consistently outperforms both OpenVLA and Octo, achieving approximately 0.6–0.7 on the easier tasks and 0.2–0.3 on the harder ones, but is substantially below full π₀, especially on the more dexterous tasks (shirt folding: approximately 0.5 for π₀-small vs. 1.0 for π₀).

Interpretation of gaps. The near-zero performance of OpenVLA confirms the paper's diagnosis that autoregressive discretization without action chunking is fundamentally incompatible with the high-frequency dexterous control required by these tasks. Octo's poor performance despite supporting action chunks suggests that 93M parameters is insufficient capacity to absorb the diversity of the pre-training mixture while maintaining precision for individual tasks. The gap between π₀ and π₀-small quantifies the contribution of VLM pre-training: approximately 40–50 percentage points on shirt folding, 20–30 points on bussing and grocery bagging.

Language Instruction Following (Section VI-B)

The headline result is that π₀, after fine-tuning to follow language commands, significantly outperforms π₀-small on language following accuracy and benefits substantially from intermediate language commands provided by either a human expert or a high-level VLM policy. Results are reported in Figure 9.

Headline numbers. Across three language-conditioned tasks (bussing, table setting, grocery bagging), evaluated under three conditions:

π₀-flat (direct task command, no intermediate instructions): approximately 0.35–0.5 average score across tasks (exact values per task are shown in Figure 9 but not individually enumerated in the text).
π₀-human (intermediate commands from a human expert): approximately 0.6 average score—a substantial improvement.
π₀-HL (intermediate commands from a high-level VLM policy, fully autonomous): approximately 0.5 average score—better than π₀-flat but below π₀-human.
π₀-small-human: approximately 0.2–0.3 average score across tasks. Critically, π₀-small does not benefit from the addition of a high-level expert: "due to π₀-small's limited language following ability, overall it does not gain with the addition of a high-level expert" (Section VI-B). The paper attributes this to the VLM pre-training providing stronger language grounding.

Language following accuracy. The paper also reports language following accuracy directly (presumably measuring whether the model picks up the correct object when commanded in the intermediate instructions), shown in Figure 9. π₀ achieves significantly higher accuracy than π₀-small. Exact numbers are not quoted in the text, but the bar chart shows π₀ at approximately 0.75–0.8 language following accuracy vs. π₀-small at approximately 0.4–0.5, depending on the task.

Interpretation. The improvement from π₀-flat to π₀-human demonstrates that π₀ can effectively use detailed language guidance when available—it understands and follows step-by-step instructions. The improvement from π₀-flat to π₀-HL demonstrates that an autonomous high-level VLM policy can partially substitute for human guidance, though the gap between π₀-HL and π₀-human indicates that the high-level policy's commands are not as effective as human expert commands (likely due to the high-level policy occasionally producing suboptimal or mistimed instructions). The failure of π₀-small to benefit from the high-level expert confirms that language understanding is not an emergent property of the flow matching architecture alone but requires the VLM pre-training.

Fine-Tuning to New Dexterous Tasks (Section VI-C)

The headline result is that π₀, when fine-tuned on downstream tasks with varying amounts of data (1, 5, or 10 hours), consistently outperforms both prior methods (ACT, Diffusion Policy, OpenVLA, Octo) and a version of π₀ trained from scratch on the same fine-tuning data, with the pre-training advantage being largest for tasks that are harder or more different from the pre-training distribution, and for smaller fine-tuning dataset sizes. Results are in Figure 11.

Headline numbers (averaged across all five tasks, Figure 11 top-left panel "Average Across All Tasks"):

With 1 hour of fine-tuning data: π₀ (pre-trained) achieves approximately 0.55 average task progress; π₀ (scratch) achieves approximately 0.35.
With 5 hours: π₀ achieves approximately 0.65; π₀ (scratch) achieves approximately 0.5.
With 10 hours: π₀ achieves approximately 0.72; π₀ (scratch) achieves approximately 0.65.
ACT and Diffusion Policy (trained from scratch on fine-tuning data only) achieve approximately 0.5–0.6 across dataset sizes, roughly matching π₀ (scratch) at 5–10 hours.
OpenVLA and Octo achieve substantially lower performance (approximately 0.0–0.3, shown only at one dataset size per task due to evaluation cost).

Per-task breakdown (Figure 11, remaining panels):

Stack bowls (UR5e, "easy" tier): At 1 hour, π₀ achieves approximately 0.55 vs. π₀ (scratch) at approximately 0.25 and DP at approximately 0.4. At 10 hours, all methods converge to approximately 0.75–0.85, with π₀ slightly ahead. This task is most similar to pre-training data (dish manipulation), and the pre-training benefit is most pronounced at low data.
Towel folding (Bi-ARX, "easy" tier): At 1 hour, π₀ achieves approximately 0.55 vs. π₀ (scratch) at approximately 0.3 and DP at approximately 0.35. At 10 hours, π₀ reaches approximately 0.8, with a clear gap over π₀ (scratch) at approximately 0.65.
Tupperware in microwave (Bi-ARX, mid tier): At 1 hour, π₀ achieves approximately 0.5, dramatically outperforming all baselines (π₀ scratch at approximately 0.15, DP at approximately 0.1, ACT at approximately 0.05). At 5 hours, π₀ reaches approximately 0.65 while the next best (DP) is at approximately 0.45. At 10 hours, π₀ reaches approximately 0.75, π₀ (scratch) reaches approximately 0.45, and DP reaches approximately 0.4. This task introduces a novel element (microwave) not seen in pre-training, and the pre-training advantage persists even at 10 hours.
Paper towel replacement (Bi-UR5e, "hard" tier): At 1 hour, π₀ achieves approximately 0.2 vs. π₀ (scratch) at approximately 0.1 and DP/ACT near 0.0. At 10 hours, π₀ reaches approximately 0.55, roughly 2× the performance of π₀ (scratch) at approximately 0.25 and DP at approximately 0.3. The paper notes this task is "hard" because no such items are found in pre-training, and the gap is the largest relative improvement from pre-training.
Items in drawer (Franka, "hard" tier): At 1 hour, π₀ achieves approximately 0.25 vs. π₀ (scratch) at approximately 0.15 and DP/ACT near 0.05–0.1. At 10 hours, π₀ reaches approximately 0.55 vs. π₀ (scratch) at approximately 0.4. The gap is smaller than for paper towel replacement, but still substantial.

Key patterns across tasks:

The pre-training advantage (π₀ vs. π₀ scratch) is largest at 1 hour and narrows as fine-tuning data increases—consistent with pre-training providing a strong prior that becomes less critical as task-specific data accumulates.
The pre-training advantage is larger for "hard" tasks (paper towel replacement, items in drawer) than for "easy" tasks (stack bowls)—consistent with the pre-training providing broadly useful physical capabilities that matter more when the task is dissimilar from any single pre-training task.
Prior methods (ACT, DP, OpenVLA, Octo) generally underperform π₀ (scratch) on some tasks and match it on others, but none approach π₀. Notably, OpenVLA and Octo, which also employ pre-training (on OXE), perform worst, suggesting their pre-training does not transfer to these dexterous tasks.

Mastering Complex Multi-Stage Tasks (Section VI-D)

The headline result is that π₀, after full pre-training and post-training (including 100+ hours of high-quality data for the most complex tasks), achieves more than 50% of the maximum score across all evaluated complex tasks, and the full pre-training + post-training recipe consistently outperforms both pre-training-only (out-of-box) and post-training-only (scratch) ablations. Results are in Figure 13.

Headline numbers (Figure 13, averages over 10 trials per task):

Laundry folding (static bimanual): π0 attains approximately 0.85 average score (out of 4). Out-of-box: approximately 0.15. Scratch: approximately 0.55. The pre-training benefit (0.85 vs. 0.55) is approximately 30 percentage points.
Mobile laundry (Fibocom mobile manipulator): π₀: approximately 0.75. Out-of-box: approximately 0.25. Scratch: approximately 0.45. Pre-training benefit: approximately 30 percentage points.
Dryer unloading (Fibocom): π₀: approximately 0.8. Out-of-box: approximately 0.3. Scratch: approximately 0.65. Pre-training benefit: approximately 15 percentage points.
Table bussing (complex, novel objects): π₀: approximately 0.6 (scored out of 12 objects). Out-of-box: approximately 0.3. Scratch: approximately 0.4. Pre-training benefit: approximately 20 percentage points.
Box building (cardboard box assembly): π₀: approximately 0.55 (scored out of 5). Out-of-box: approximately 0.0 (effectively unable to perform the task without post-training). Scratch: approximately 0.25. Pre-training benefit: approximately 30 percentage points. This task is explicitly noted as not present in pre-training.
Packing eggs: π₀: approximately 0.7 (scored out of 7, with 1 point per egg + 1 for closing). Out-of-box: approximately 0.05. Scratch: approximately 0.35. Pre-training benefit: approximately 35 percentage points.
Packing food (to-go box): π₀: approximately 0.8 (scored out of 5). Out-of-box: approximately 0.15. Scratch: approximately 0.55. Pre-training benefit: approximately 25 percentage points.

Key patterns:

On tasks present in pre-training (laundry folding, mobile laundry, dryer unloading), the out-of-box model achieves non-trivial performance (0.15–0.3), indicating zero-shot capability. Post-training substantially improves this.
On tasks not present in pre-training (box building, packing eggs, to-go box, table bussing), the out-of-box model achieves near-zero performance, confirming that the pre-training does not directly teach these specific tasks. However, the scratch model also performs poorly, while the full π₀ achieves reasonable performance, suggesting that pre-training provides broadly useful physical primitives that the post-training phase can assemble into new task-specific strategies.
The gap between π₀ and scratch is largest for the hardest tasks (box building: 0.55 vs. 0.25; packing eggs: 0.7 vs. 0.35), consistent with the claim that "pre-training is especially useful with harder tasks" (Section VI-D).
Absolute performance varies substantially across tasks—laundry folding, mobile laundry, and dryer unloading achieve 0.75–0.85, while box building and table bussing achieve 0.55–0.6. The paper attributes this to "differences in task difficulty and the degree to which the tasks are represented in pre-training" (Section VI-D).

Ablation Studies and Robustness Checks

VLM pre-training contribution: The comparison between π₀ and π₀-small in the out-of-box evaluation (Figure 7) and language following evaluation (Figure 9) provides evidence that VLM pre-training substantially improves both task performance and language understanding. π₀-small (470M parameters, no VLM initialization) achieves 20–50 percentage points lower performance than full π₀ on out-of-box tasks and shows dramatically worse language following accuracy. However, this comparison conflates model scale (3.3B vs. 470M) with VLM pre-training, since larger models are "difficult to use without pre-training" (Section VI-A). The paper acknowledges this confound: "Unfortunately, it is hard to make this last comparison fair."

Pre-training phase contribution: The comparison between π₀ (full pre-training + fine-tuning) and π₀ (scratch, fine-tuning only) in Figures 11 and 13 isolates the effect of pre-training. Across all fine-tuning tasks, pre-training provides a positive benefit, with the gap being largest at small fine-tuning dataset sizes (1 hour) and for harder tasks. This ablation directly tests the paper's central claim that pre-training on diverse, lower-quality data endows the model with recovery behaviors and broadly useful physical capabilities.

Post-training phase contribution: The comparison between π₀ (full recipe) and the out-of-box model in Figures 11 and 13 shows the effect of post-training. For tasks present in pre-training (laundry folding, mobile laundry), post-training improves performance by 50–60 percentage points (from 0.15–0.3 to 0.75–0.85). For tasks not in pre-training, post-training is essential to achieve any non-trivial performance at all (box building goes from 0.0 to 0.55). This supports the claim that pre-training alone does not produce fluent task-specific strategies.

Architecture parity comparison: The compute-parity π₀ (160k steps) in Figure 7 isolates the effect of model architecture and pre-training from the effect of simply training longer. At 160k steps (matching OpenVLA's training budget and below Octo's 320k), π₀ still dramatically outperforms both baselines. This suggests the advantage is architectural, not merely a matter of training for more steps.

Flow matching vs. autoregressive discretization: The comparison between π₀ and OpenVLA in Figure 7, while not a clean ablation (the models differ in architecture, scale, and pre-training), provides evidence that the flow matching approach handles high-frequency dexterous control far better than autoregressive discretization. OpenVLA's near-zero performance on tasks requiring action chunks (shirt folding, grocery bagging) is consistent with the paper's diagnosis that discretization fails for these tasks.

Flow matching vs. diffusion (Octo): The comparison between π₀ and Octo in Figure 7 compares flow matching against standard diffusion for action generation (though confounded by scale: 3.3B vs. 93M parameters). π₀ dramatically outperforms Octo, but the scale difference prevents attributing this to flow matching specifically.

Temporal ensembling: The paper reports that "We tried temporal ensembling early on and found that it hurt policy performance, so we opted not to aggregate actions and instead execute action chunks open-loop" (Appendix D). This is a negative result: the standard action chunking technique from Zhao et al. (2023) degrades performance in the flow matching setting, likely because the bidirectional attention in the action expert already produces temporally coherent chunks, and averaging across overlapping chunks introduces artifacts. No quantitative results are reported for this ablation.

ReST^EM revision model training: While not discussed in the main paper body (it appears in Appendix K and is mentioned in the prior sections document under "Innovation 4"), the paper reports that attempting to optimize the revision model using ReST^EM (Singh et al., 2024)—an RL-style self-improvement procedure—caused performance to degrade. The authors hypothesize that "on-policy data collection in ReST^EM exacerbates spurious correlations in revision data, causing the model to fail to learn the revision task properly" (Appendix K). This is a notable negative result suggesting that the training recipe is sensitive to data distribution and that naive self-improvement can backfire.

Missing ablations: Several ablations that would strengthen the paper's claims are not reported:

There is no ablation comparing the mixture-of-experts action expert design against a shared-weight alternative where the action tokens are processed by the same weights as the VLM tokens. This makes it impossible to quantify how much the action expert contributes to performance independently of the VLM pre-training.
There is no ablation of the blockwise causal attention mask against fully bidirectional attention or fully causal attention, so the contribution of the specific mask design (and the caching optimization it enables) cannot be isolated.
There is no ablation of the timestep sampling distribution (shifted Beta) against the standard uniform distribution or the logit-normal distribution from Esser et al. (2024), so the paper's argument that "action prediction is fundamentally different from image generation" in this regard remains an untested hypothesis.
There is no ablation of the number of flow matching integration steps (10) against fewer or more steps, so the tradeoff between inference speed and action quality is not characterized.
There is no study of how performance varies with the amount of pre-training data (e.g., training on 10%, 50%, 100% of the mixture), which would help establish whether the 10,000-hour scale is necessary or whether smaller mixtures would suffice.

Critical Assessment

The central claim: "π₀, when pre-trained on 10,000 hours of diverse robot data and post-trained on high-quality task-specific data, achieves a level of dexterity and generality that goes significantly beyond previously demonstrated robot foundation models" (Section I, Section VII).

The evidence for this claim is substantial but has important boundary conditions. The out-of-box evaluation (Figure 7) convincingly shows that π₀ outperforms prior generalist models (OpenVLA, Octo) on a set of diverse dexterous tasks by very large margins—near-perfect on shirt folding vs. near-zero for OpenVLA, for instance. The fine-tuning experiments (Figures 11, 13) show that π₀ can be adapted to complex, multi-stage tasks that prior work has not demonstrated with learned policies (laundry folding from arbitrary crumpled states, box assembly requiring bimanual coordination and retries). The paper is transparent that some of these tasks were not solvable with other methods: "These tasks are very difficult, and we were not able to solve them with other methods" (Section VI-D).

However, the claim that this represents a level of generality "significantly beyond" prior work depends on what one counts as "prior work." The paper compares against published, open-source models (OpenVLA, Octo, ACT, Diffusion Policy) but does not compare against proprietary systems that may exist at similar scales (e.g., RT-2, which is not publicly available for fine-tuning, or industrial systems not published in the academic literature). The claim of generality is demonstrated across 7 robot configurations—a genuinely impressive range—but all of these are relatively similar: fixed-base or mobile arms with parallel jaw grippers. There is no evidence that the approach would work for, say, dexterous hands, quadrupeds, or drones—domains the paper explicitly flags in Section VII as open questions.

The claim: "Training on only high-quality data results in a brittle model that does not reliably recover from mistakes, while running the pre-trained model in zero shot does not always exhibit the fluent strategies from post-training data. Combining both provides the desired behavior" (Section VII).

This is the paper's most conceptually important claim—the two-phase recipe as a solution to the robustness-fluency tension. The evidence is strongest in Figure 13, where the full π₀ consistently outperforms both the scratch and out-of-box ablations across all complex tasks. The pattern is exactly what the claim predicts: out-of-box (pre-training only) shows some capability but lacks fluency; scratch (post-training only) learns the task but without the recovery behaviors from pre-training; full π₀ combines both to achieve the highest scores.

However, the evidence has two significant limitations:

First, the claim that the scratch model is "brittle" and fails to recover from mistakes is qualitative—the paper does not provide quantitative measures of brittleness, such as performance under systematic perturbations, recovery rate after deliberate disturbances, or generalization to held-out object instances. The lower absolute scores of the scratch model could be due to many factors besides lack of recovery behaviors—for example, the scratch model may simply learn a less effective strategy due to the smaller effective dataset size (no pre-training data). The paper's interpretation that the difference is specifically about recovery behaviors is plausible given the LLM analogy, but it is not directly tested.

Second, the paper does not ablate the composition of the pre-training data to determine which types of pre-training data contribute to robustness. Is it the diversity of objects? The diversity of robot embodiments? The presence of suboptimal demonstrations and failure trajectories? The mixture of single-arm and dual-arm tasks? Without this analysis, the "recipe" remains a black box—future practitioners know that pre-training helps, but not what to include in the pre-training mixture to get that benefit.

The claim: "π₀ outperforms prior VLA models because its flow matching architecture enables high-frequency action chunking that autoregressive discretization cannot support" (Section II, Section VI-A).

The evidence for this claim is strong but correlational, not causal. OpenVLA fails dramatically on the high-frequency dexterous tasks (Figure 7), while π₀ succeeds. However, OpenVLA and π₀ differ along many axes beyond the action representation: π₀ is a different base VLM (PaliGemma vs. Llama), has a different training recipe, uses a different pre-training mixture (OpenVLA was trained on the paper's mixture, but at 160k steps vs. π₀'s 700k), and has a different inference procedure. The paper acknowledges that OpenVLA "does not support action chunking or high-frequency control" and attributes its failure to this, but cannot rule out that other architectural differences (capacity, pre-training quality, training stability) contribute to the gap. The π₀-small model, which uses flow matching but not VLM pre-training, still outperforms OpenVLA—this is consistent with the flow matching advantage claim, but π₀-small's better performance could also be due to its different visual encoder, different training dynamics, or other uncontrolled differences. A cleaner ablation would compare the full π₀ architecture against a version that replaces the flow matching head with an autoregressive discretization head while keeping everything else identical—this is not reported.

The claim: "Positive transfer from cross-embodiment pre-training improves fine-tuning performance, especially for harder tasks and smaller fine-tuning datasets" (Section VI-C).

This claim is well-supported by Figure 11 and Figure 13. The pattern of π₀ consistently outperforming π₀ (scratch), with larger gaps at smaller fine-tuning dataset sizes and for harder tasks, is exactly what positive transfer predicts. However, there are two caveats:

First, the paper does not test for negative transfer on tasks where cross-embodiment data might plausibly hurt—for example, a task that requires embodiment-specific dynamics that conflict across platforms. All evaluated tasks show positive transfer, which might reflect the specific task selection rather than a universal property.

Second, the paper does not disentangle whether the benefit comes from cross-task transfer (seeing many different manipulation behaviors) or cross-embodiment transfer (seeing the same behavior on different robot hardware). Would a model pre-trained only on UR5e data perform as well as one pre-trained on all 7 embodiments when fine-tuned to a UR5e task? This ablation is not reported, so the specific contribution of embodiment diversity to transfer remains unquantified.

Small test sets and lack of statistical rigor. The evaluation protocol uses 10 trials per task per method, which is a small sample. The paper does not report standard deviations, confidence intervals, or any form of statistical significance testing. The figures (7, 9, 11, 13) show point estimates without error bars. For the complex multi-stage tasks (Figure 13), where scores range from 0.0 to 1.0 and each trial takes 5–20 minutes, the variance across 10 trials could be substantial—for example, on box building, a difference of 0.55 vs. 0.25 (π₀ vs. scratch) might or might not be statistically significant depending on the variance, which is not reported. This makes it difficult to assess whether smaller gaps (e.g., the 10-hour fine-tuning results in Figure 11 where methods converge to within ~0.1 of each other) are meaningful.

Missing inference-time compute comparison. The paper emphasizes the importance of frequency (up to 50 Hz) for dexterous tasks, and the flow matching architecture is partly motivated by inference efficiency (10 integration steps with caching). However, the paper does not compare inference-time compute against prior methods in a controlled way. How many FLOPs does π₀ use per action vs. OpenVLA or Octo? The latency numbers in Table I (73 ms total) are for a specific GPU (RTX 4090) and are not compared against the baselines on the same hardware. This makes it difficult to assess whether π₀'s architectural advantages come at a higher or lower inference cost.

Single model scale. All experiments use the PaliGemma 3B backbone. The paper does not evaluate whether the benefits of VLM pre-training, flow matching, or the pre-training/post-training recipe scale with model size (e.g., would a 1B VLM work nearly as well? Would a 7B or larger VLM work substantially better?). The choice of PaliGemma is pragmatic ("a convenient trade-off between size and performance"), but it leaves open whether the results are specific to this scale or would generalize.

No evaluation under systematic perturbation or distribution shift. The paper's central argument about the pre-training/post-training recipe is that pre-training provides robustness through exposure to diverse situations and recovery behaviors. However, all evaluations are conducted in the same laboratory settings where the training data was collected. There is no systematic evaluation under:

Deliberate perturbations (e.g., an experimenter moving an object mid-task);
Novel object instances not present in any training data (some bussing evaluations use "unseen objects," but these are not systematically varied);
Different lighting conditions, backgrounds, or camera viewpoints;
Different physical dynamics (e.g., stiffer or looser joints on the robot).

Without such evaluations, the claim of "robustness" from pre-training remains an interpretation of the scratch-vs-pre-trained performance gap rather than a directly demonstrated property.

Overall assessment. The paper provides convincing evidence that the combination of (1) VLM pre-training, (2) flow matching for action generation, and (3) a two-phase pre-training/post-training recipe on 10,000 hours of diverse robot data produces a policy that can control multiple embodiments across a range of dexterous tasks, substantially outperforming the best available prior models in head-to-head comparisons. The three innovations—the recipe, the flow matching architecture, and the empirical demonstration of positive cross-embodiment transfer—are each supported by the experiments, though with varying degrees of direct causal evidence. The paper's central limitation is that the experiments demonstrate that the system works rather than why each component matters: key ablations (action expert vs. shared weights, timestep distribution, integration steps, pre-training data composition) are absent, and the evaluations do not directly test the specific claims about robustness and recovery behavior that motivate the two-phase recipe. The paper establishes a new state of the art for learned dexterous manipulation and provides a template for future robot foundation models, but leaves open the detailed understanding of which ingredients are necessary and which are incidental.

6. Limitations and Trade-offs

The Cost of Difficulty Estimation Is Unaccounted for in the Compute-Optimal Framework

The assumption or constraint. The paper's entire compute-optimal allocation policy — the meta-strategy that achieves the claimed 4× efficiency gains — requires knowing each prompt's difficulty before deciding how to allocate the inference budget. The method used to estimate difficulty is generating 2,048 samples per question and computing the base model's pass@1 rate (oracle) or PRM's average correctness score (predicted). The paper acknowledges this cost explicitly in Section 3.2:

"estimating difficulty in this way still incurs additional computation cost during inference... our experiments do not account for this cost largely for simplicity"

The consequence. The reported 4× efficiency gains (e.g., 16 generations matching best-of-64 in Figure 4) are computed after difficulty is known, without amortizing the 2,048-generation estimation cost. In a real deployment, the total cost would be difficulty estimation + strategy execution, and the former (2,048 samples per prompt) is vastly larger than the largest test-time budgets studied (256–512 generations). This means the headline 4× figure represents an upper bound that is not achievable in practice without a cheaper difficulty estimation method.

What evidence exists in the paper. The cost discrepancy is visible by comparing the budgets on the x-axes of Figures 4 and 8 (which max out at 256–512 generations) against the 2,048 samples used for difficulty estimation. The paper conducts no ablation on the number of samples needed for reliable difficulty estimation — it is possible that far fewer than 2,048 samples would suffice, or that difficulty could be predicted from the question text alone, but neither hypothesis is tested. The paper does show that predicted (non-oracle) difficulty bins perform nearly as well as oracle bins (Figure 4), confirming that ground-truth labels are not required, but this does not address the fundamental cost issue.

Mitigation status. The paper flags this as "a key avenue for future work" and suggests "pretraining or finetuning models to directly predict difficulty of a question" (Section 3.2, Section 8). No such model is developed or evaluated. Until this gap is closed, the compute-optimal framework is an analysis tool rather than a deployable system, and the 4× claims should be understood as the maximum possible gain if difficulty were known for free.

The Revision Model Has a High Correct-to-Incorrect Reversion Rate

The assumption or constraint. The revision model is trained exclusively on sequences where all in-context answers are incorrect followed by a correct target (Section 6.1). The model never sees examples where the current answer is already correct and should be preserved. At test time, the model therefore lacks the ability to recognize when no revision is needed: when it produces a correct answer early in a revision chain, it may "revise" that correct answer into an incorrect one at the next step.

The consequence. The paper reports (Section 6.1) that approximately 38% of correct answers produced during a revision chain get converted back to incorrect answers in the subsequent revision step. This is a structural flaw in the training data construction, not a hyperparameter that can be tuned away. It means that longer revision chains — which the compute-optimal policy sometimes prefers (e.g., on easy problems where sequential revisions dominate) — inherently risk destroying correct answers. The mitigation (selecting the best answer from anywhere in the chain via majority voting or verifier scoring) is a patch that does not address the root cause.

What evidence exists in the paper. The 38% figure is stated in Section 6.1: "approximately 38% of correct answers get converted back to incorrect ones using a naive approach." The paper does not break this down by difficulty tier or by revision depth (does the reversion probability increase with chain length? decrease? remain constant?). It also does not compare the reversion rate of the fine-tuned model against the base model prompted to self-correct, which would help isolate whether this is a training artifact or a more fundamental property of the model.

Mitigation status. The paper mitigates the symptom (by using within-chain selection rather than always taking the last revision) but does not fix the cause. A principled solution — such as training the model with examples where the correct answer is the first in the context and should be preserved, or training a binary classifier to detect when no revision is needed — is not explored. The ReST^EM experiment (Appendix K, Figure 16) shows that on-policy optimization of the revision model makes performance worse, suggesting the training procedure is fragile in ways that are not fully understood.

Hard Problems Receive Essentially No Benefit from Test-Time Compute

The assumption or constraint. The paper's approach assumes that the base model's pass@1 on a problem is non-trivially above zero — that is, the base model already produces correct solutions at some non-negligible rate, and test-time compute amplifies this success rate. When pass@1 is near zero (the hardest problems in difficulty bin 5), test-time compute cannot create correct solutions that the base model never generates.

The consequence. Across all methods — search, revisions, and their compute-optimal combinations — the hardest questions (difficulty bin 5) show near-zero improvement regardless of compute budget. In Figure 3 (right), bin 5 accuracy hovers at approximately 1–3% for all methods and all budget levels (4 to 256 generations). In Figure 7 (right), bin 5 shows roughly 2–3% accuracy irrespective of the sequential-to-parallel ratio. In the FLOPs-matched comparison (Figure 9), the bin 5 scaling line is essentially flat near 0–5%, and test-time compute with the smaller model is worse than simply deploying the ~14× larger model (with greedy decoding) at all values of the inference-to-pretraining ratio R. For example, at R ≫ 1, hard questions show a −52.9% relative disadvantage for PRM search compared to the larger model (Figure 1, bottom-right bar chart).

This means the approach offers no path forward for genuinely novel or out-of-distribution reasoning that exceeds the base model's training distribution. For such problems, scaling pretraining (larger model, more data) remains the only viable strategy.

What evidence exists in the paper. The bin 5 results appear consistently across every figure that decomposes performance by difficulty: Figure 3 (right) for search, Figure 7 (right) for revisions, Figure 9 for FLOPs-matched comparison. The paper is transparent about this limitation, stating explicitly in the Section 7 takeaway that test-time compute "cannot compensate for fundamental capability gaps that larger pretraining would address."

Mitigation status. Not addressed, and arguably not addressable within the framework. The limitation is fundamental: test-time compute can only select or refine among solutions the base model can already produce at some non-trivial rate. The paper acknowledges this as a boundary condition on the method's applicability rather than a bug to be fixed, and recommends pretraining for problems that fall into this regime.

The FLOPs-Matched Pretraining Baseline May Be Artificially Weak

The assumption or constraint. The FLOPs-matched comparison in Section 7 compares the smaller PaLM 2-S* model with compute-optimal test-time strategies against a model with approximately 14× more parameters that uses greedy decoding only — no majority voting, no best-of-N, no search, and no test-time strategy of any kind. Additionally, the larger model is scaled in parameters only while holding training data constant (following the LLaMA paradigm; Touvron et al., 2023), rather than being compute-optimally trained by scaling both parameters and data equally (following Hoffmann et al., 2022).

The consequence. Both choices make the pretraining baseline weaker than it could be:

Giving the larger model even a modest test-time compute budget (e.g., best-of-8, majority voting over 8 samples) would create a much stronger comparison. The paper's framework shows that test-time compute provides the largest gains at low budgets (Figure 3, left: the steepest part of the scaling curves is between 1 and 16 generations). Adding just 8 generations of best-of-N weighted to the larger model could substantially close or reverse the gaps shown in Figure 9.
A Chinchilla-optimal model trained with 14× more total FLOPs (scaling both data and parameters) would likely outperform a parameter-only-scaled model of the same size, further strengthening the pretraining baseline.

The reported advantages of test-time compute over pretraining — e.g., +27.8% on easy questions at R ≪ 1 (Figure 1, top-right bar chart) — may shrink or reverse against a properly tuned larger model.

What evidence exists in the paper. The paper explicitly acknowledges the parameter-only scaling choice in Section 7:

"We choose this setting as it is representative of a canonical approach to scaling pretraining compute and leave the analysis of compute-optimal scaling of pretraining compute where the data and parameters are both scaled equally to future work."

However, it does not acknowledge the asymmetry of giving test-time compute only to the smaller model. The larger model's greedy decoding is treated as the default deployment strategy, but the paper's own results suggest that even a small test-time budget would help it substantially. This makes the FLOPs-matched comparison best understood as "test-time compute on a small model vs. naive deployment of a large model" rather than "test-time compute vs. pretraining compute."

Mitigation status. The paper acknowledges the parameter-only scaling limitation and defers compute-optimal pretraining comparisons to future work. It does not address the greedy decoding asymmetry. A fairer comparison would give both models access to the compute-optimal test-time strategy, measuring how the marginal benefit of additional pretraining compares to the marginal benefit of additional test-time compute at the frontier where both are already optimized.

No Combination of Search and Revisions Is Evaluated

The assumption or constraint. The paper studies two complementary mechanisms — PRM-guided search (modifying the verifier/selection) and iterative revisions (modifying the proposal distribution) — but never combines them into a single system. Section 8 explicitly acknowledges this:

"we did not experiment with PRM tree-search techniques in combination with revisions"

The consequence. The two mechanisms have complementary, difficulty-dependent strengths: revisions excel on easy problems (local refinement) while PRM search excels on medium problems (global exploration via verifier guidance). A combined system could, for example, use the revision model as the proposal distribution within beam search — at each step of the search tree, the model conditions on previously rejected branches as context, potentially generating higher-quality candidate steps than the base model alone. Alternatively, the PRM's per-step scores could guide which revisions to pursue, deciding in real-time whether a revision chain is on track or should be restarted.

The current results therefore represent a lower bound on what a fully integrated system could achieve. The compute-optimal policy selects between search and revisions per difficulty bin, but cannot benefit from both simultaneously on a given problem. The compute-optimal scaling curves in Figures 4 and 8 represent the best achievable with either mechanism alone, not the best achievable with both combined.

What evidence exists in the paper. None — this is a gap in the experimental coverage, not a measured failure. The paper's decomposition of test-time strategies into proposal distribution modifications and verifier modifications (Section 2) naturally suggests combining them, and the difficulty-dependent analysis shows they are complementary, but the combination is not tested.

Mitigation status. The paper flags this as future work in Section 8. The architecture and training pipeline support it (the revision model can be used as a drop-in replacement for the base model in beam search), but no results are available. Given the independent gains from each mechanism (up to 4× individually), the combined gains could be larger, making this one of the most impactful immediate next steps suggested by the paper.

Single Benchmark and Single Model Family Limit Generality of Findings

The assumption or constraint. All results in the paper are on the MATH benchmark (high-school competition-level math problems) using PaLM 2-S* as the base model. The authors state they "believe this model is representative of the capabilities of many contemporary LLMs" (Section 4), but this claim is not verified by replication on other models or datasets.

The consequence. Several aspects of the findings could be model-specific or domain-specific:

The PRM's quality, calibration, and over-optimization behavior depend on PaLM 2-S*'s output distribution — including its error patterns, its tendency to produce certain types of mistakes, and the distribution of solution lengths and styles. A model with different calibration properties might exhibit different difficulty-dependent scaling curves, and the specific thresholds at which beam search over-optimizes vs. helps might shift.
The revision model's ability to learn from incorrect in-context examples depends on the base model's in-context learning capabilities, which vary substantially across model families. A model with stronger in-context learning might benefit more from sequential revisions; a model with weaker in-context learning might benefit less.
The MATH benchmark consists exclusively of competition-level math problems requiring symbolic reasoning and multi-step logical deduction. It is unclear whether the difficulty-dependent patterns — beam search hurting easy problems due to verifier exploitation, revisions helping easy problems via local refinement, no method helping on the hardest problems — generalize to other reasoning domains (code generation, logical reasoning, scientific QA, planning) or to tasks requiring factual knowledge recall rather than inference.

What evidence exists in the paper. The paper does not provide any cross-model or cross-dataset validation. The MATH test set has 500 questions, split into five difficulty quintiles of approximately 100 each, further split by two-fold cross-validation — meaning the compute-optimal policy is selected based on roughly 50 questions per fold per bin. The paper does not report confidence intervals or standard errors on the scaling curves, making it impossible to assess whether the observed differences between strategies at specific budget levels are statistically reliable or artifacts of the specific 100-question bin composition.

Mitigation status. The paper does not claim universality and uses appropriately cautious language ("believe this model is representative"). Section 8 does not explicitly call for replication on other models or benchmarks, though it does suggest extending to "other domains and modalities." The findings about difficulty-dependent strategy effectiveness are the paper's central contribution; their generality to other models and tasks is an open empirical question that future work must address.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper fundamentally reframes what a "generalist robot policy" means and what it takes to build one. Before π₀, the field operated under a tacit dichotomy: you could either train broad but crude policies on diverse multi-task data (Octo, OpenVLA, RT-2) that handled many embodiments at a basic level but couldn't do dexterous manipulation, or you could train narrow but dexterous policies on small expert datasets (ACT, Diffusion Policy) that excelled at individual tasks like folding a specific item but couldn't generalize across tasks, objects, or embodiments. The paper demolishes this dichotomy by demonstrating that the same architecture, trained at sufficient scale with the right recipe, can do both simultaneously—control 7 different robot configurations at 20–50 Hz across 68+ tasks spanning simple pick-and-place to complex bimanual box assembly from arbitrary crumpled laundry states.

This is not merely an incremental improvement over prior VLAs. It is a paradigm shift in the design space: the paper establishes that the action representation is not a cosmetic detail but the gating factor that determines whether Internet-scale semantic pre-training can transfer to physical control at all. Prior VLA work treated actions as an afterthought—"just discretize them into tokens and train like a language model"—because the language modeling paradigm was so successful elsewhere. π₀ demonstrates that this choice is catastrophic for dexterity. OpenVLA, a 7B parameter model with strong VLM pre-training, achieves near-zero performance on shirt folding (Figure 7) because its autoregressive discretization cannot produce the 50 Hz temporally coherent action chunks that the task demands. The paper's core architectural argument—that flow matching over continuous action chunks is the natural interface between VLM semantics and dexterous physical control—will likely become the default design principle for future VLA models, in the same way that the Transformer replaced RNNs as the default sequence model after "Attention Is All You Need."

The paper also provides a resolution to the conflicting narratives around cross-embodiment training. Prior work left open the question of whether training on heterogeneous robot data actually helps or whether the embodiment mismatch creates interference that degrades individual task performance. The paper's fine-tuning experiments (Figures 11, 13) provide the first large-scale evidence that positive transfer dominates—the pre-trained model consistently outperforms the scratch model, with the gap being largest (often 2× or more) on harder tasks and at smaller fine-tuning dataset sizes. This validates the substantial community investment in large-scale multi-robot data collection (OXE, DROID, Bridge) and shifts the burden of proof: future work must now demonstrate when and why negative transfer occurs, rather than assuming it as a default. The finding that pre-training helps most precisely where it's most needed—hard tasks with limited fine-tuning data—is particularly consequential because it mirrors the pattern observed in NLP and vision foundation models, suggesting that robot learning follows the same scaling dynamics.

Perhaps most consequentially, the paper establishes that the pre-training/post-training separation—long taken for granted in NLP and vision—is equally critical for robot learning, but for a robot-specific reason that goes deeper than the standard "knowledge then alignment" narrative. In NLP, pre-training provides factual knowledge and linguistic competence, while post-training teaches the model to follow instructions. In robotics, the paper argues, pre-training provides something additional: a repertoire of recovery behaviors that only appears when the model has seen diverse, imperfect demonstrations including mistakes, retries, and varied strategies. The post-training data—carefully curated, fluent expert demonstrations—teaches the desired task strategy, but by itself produces a brittle model that fails at the first perturbation because it never learned what failure looks like or how to escape it. This is a fundamentally physical insight with no direct NLP analog, and it changes how the field should think about data curation: low-quality data is not merely a budget compromise but a positive requirement for robustness. The evidence in Figure 13—where the full recipe (pre-training + post-training) consistently outperforms post-training-only (scratch), with gaps of 20–35 percentage points on the hardest tasks—provides empirical weight to this claim.

Finally, the paper makes several research directions less attractive by demonstrating ceiling effects:

Further investment in autoregressive discretization for dexterous VLA models is unlikely to be productive. The paper shows that even a 7B parameter model (OpenVLA) with strong VLM pre-training fails entirely on high-frequency dexterous tasks, while a 3.3B parameter model with flow matching succeeds. This is a representation problem, not a scale problem.
Small-scale dexterous manipulation research that trains from scratch on 10–100 demonstrations per task, without any pre-training, now faces a higher bar. The paper shows that pre-training provides the largest relative gains at small fine-tuning dataset sizes (1 hour vs. 10 hours in Figure 11), meaning the "data-efficient" argument for training from scratch is undermined: if you're data-constrained at test time, you should pre-train broadly first rather than train narrowly from scratch.
Single-embodiment robot learning as a long-term research program is called into question. The paper demonstrates positive transfer across 7 distinct embodiments, and the benefits of cross-embodiment pre-training are strongest for tasks that are most different from any individual pre-training task. This suggests that the field's historical focus on single-robot, single-task learning may have been a local optimum driven by data and compute constraints, not a fundamental property of the problem.

Follow-Up Research This Work Enables

Ablation of pre-training data composition to identify what drives positive transfer. The paper shows that cross-embodiment pre-training helps, but does not isolate which aspects of the pre-training mixture matter. A critical follow-up would systematically ablate the pre-training data along several axes: (a) embodiment diversity (pre-train on 1, 3, or 7 robot types and measure fine-tuning performance on a held-out embodiment), (b) task diversity (pre-train on 10, 34, or 68 tasks and measure generalization to new tasks), (c) demonstration quality (pre-train on clean expert data only vs. mixed-quality data including failures, and measure robustness under perturbations), and (d) visual diversity (vary the number of distinct scenes, objects, and lighting conditions). The paper's finding that pre-training helps most on tasks dissimilar to any single pre-training task (paper towel replacement, box building) suggests that task diversity is important, but embodiment diversity may matter less than the paper implies if cross-task transfer within a single embodiment accounts for most of the benefit. A negative result—showing that embodiment diversity contributes negligible marginal benefit over single-embodiment multi-task pre-training—would substantially refine the field's data collection priorities.

Combining flow matching VLA pre-training with online RL fine-tuning for robustness. The paper's central claim is that pre-training provides recovery behaviors, but this is tested only through the indirect comparison of scratch vs. pre-trained models under standard evaluation conditions. A direct test would take a pre-trained π₀ model and fine-tune it with online reinforcement learning on a task where the reward function captures robustness explicitly—for example, a bussing task with an adversary that randomly displaces objects mid-episode, or a folding task where the cloth is perturbed by wind. If the paper's hypothesis is correct, the pre-trained model should learn robust policies with substantially fewer environment interactions than a scratch model, because it already has a latent repertoire of corrective behaviors that RL can activate and refine. This experiment would bridge the gap between the paper's behavioral cloning approach and the RL-based robustness literature (e.g., Radosavovic et al., 2023), and would provide causal evidence for the "recovery repertoire" claim. A negative result—RL from scratch matching pre-trained RL at convergence, even if pre-training helps initially—would suggest the pre-training benefit is primarily about initialization efficiency rather than acquiring qualitatively different capabilities.

Testing the limits of cross-embodiment transfer: dexterous hands, quadrupeds, and drones. The paper demonstrates positive transfer across 7 embodiments that are structurally similar (all are arms with parallel jaw grippers, differing mainly in degrees of freedom and mobility). The paper explicitly flags that it is "left for future work to understand whether this universality extends to much more distinct domains, such as autonomous driving, navigation, and legged locomotion" (Section VII). A stress-test experiment would pre-train π₀ on the existing arm manipulation data and then fine-tune to a dexterous hand (e.g., a Shadow Hand or Allegro Hand performing in-hand manipulation) or a quadruped (e.g., a Unitree Go2 performing locomotion and object interaction). If positive transfer persists, it would imply that the model is learning something remarkably abstract—physical intuition about contact, force application, and object dynamics—that transcends kinematic structure. If transfer is zero or negative, it would establish a boundary condition: the architecture may only transfer across embodiments that share a certain level of kinematic similarity (e.g., serial-chain manipulators), and fundamentally different morphologies require separate pre-training.

Scaling laws for robot pre-training data: how much data is enough, and what saturates first? The paper uses 10,000 hours of pre-training data—a 1,000× increase over prior dexterous manipulation work—but does not study how performance scales with pre-training dataset size. A natural follow-up would train multiple π₀ variants on 1%, 10%, 50%, and 100% of the pre-training mixture (randomly subsampled) and evaluate fine-tuning performance on a held-out suite of downstream tasks. This would reveal: (a) whether performance saturates at 10,000 hours or whether more data would help, (b) whether certain task categories (e.g., deformable object manipulation vs. rigid object pick-and-place) benefit more from additional pre-training data, and (c) whether the pre-training data scaling exponent is similar to the exponents observed in NLP and vision (where performance often scales as a power law in dataset size). Given the cost of collecting robot data, understanding the shape of the scaling curve is critical for informing future data collection investments. The paper's data weighting scheme (n^0.43 per task-robot combination) also deserves ablation: does the exponent matter, or is any reasonable sublinear weighting sufficient?

A true compute-matched comparison against a larger VLM with test-time compute. The paper's comparison against a ~14× larger model with greedy decoding (Section 7) is asymmetric because it gives test-time strategies only to the smaller model. A rigorous follow-up would fix a total FLOPs budget that includes both pretraining and inference, and compare: (a) π₀ (3.3B) with compute-optimal test-time strategies vs. (b) a 7B or 12B VLA (e.g., based on a larger PaliGemma or LLaMA variant) with its own compute-optimal test-time allocation. This would measure the marginal value of additional pretraining at the frontier where both models are already optimized for inference. If the larger model with test-time strategies still underperforms the smaller model on easy-to-medium tasks, the paper's conclusion—that test-time compute can substitute for pretraining—is strengthened. If the larger model closes or reverses the gap, it would suggest that the paper's finding was an artifact of comparing an optimized small model against a naively deployed large model, not a fundamental property of the pretraining-inference tradeoff.

Language-conditioned policy chaining for multi-hour autonomous operation. The paper demonstrates that π₀ can follow intermediate language commands from a high-level VLM policy (π₀-HL in Figure 9), enabling autonomous completion of multi-stage tasks like table bussing. However, the tasks in the paper are bounded (5–20 minutes, 10–30 subtasks). A natural extension would test whether this architecture can chain hundreds or thousands of language-conditioned behaviors for multi-hour autonomous operation—for example, a mobile manipulator that receives a high-level instruction ("clean the kitchen") and autonomously decomposes it into navigation, object search, dexterous manipulation, and error recovery subtasks, executing them sequentially over an hour or more. This would stress-test the model's ability to handle the compounding error problem (small mistakes in early subtasks cascading into failures later) and the high-level policy's ability to detect and recover from failures. The paper's finding that π₀ benefits from human-provided intermediate commands but only partially from VLM-provided commands (π₀-HL underperforms π₀-human) suggests that high-level policy quality is the current bottleneck. Improving the high-level policy—perhaps by fine-tuning a VLM on robot-specific replanning data—could close this gap.

Practical Applications and Downstream Use Cases

Rapid deployment of new manipulation skills in warehouse and logistics settings. The paper's fine-tuning results with 1–10 hours of data (Figure 11) directly translate to industrial practice. In a warehouse where the product mix changes seasonally or weekly, the current approach is to manually reprogram or reteach robots for each new SKU—a process that can take days to weeks. With π₀ pre-trained on a broad base of manipulation data (which could increasingly be collected by third-party data providers, analogous to how ImageNet pre-training became a commodity), a logistics company could fine-tune a single pre-trained model to handle new items (new box shapes, new product packaging, new tote configurations) with as little as 1 hour of demonstration data, achieving reasonable performance immediately and near-ceiling performance with 5–10 hours. The paper shows that on tasks structurally similar to pre-training data (stack bowls, towel folding), π₀ with 1 hour of fine-tuning achieves approximately 0.55 average task progress—not production-ready, but a usable starting point—while 10 hours pushes this to approximately 0.75–0.85, which approaches the reliability needed for supervised automation where a human operator can handle the remaining failure cases.

Assistive robotics for activities of daily living (ADLs). The laundry folding, table bussing, grocery bagging, and food packing tasks evaluated in the paper map almost directly onto common ADL tasks for elderly or disabled individuals. The paper demonstrates that a single π₀ model can be fine-tuned to fold laundry from arbitrary crumpled configurations (Figure 13: 0.75–0.85 score on laundry folding), pack food into containers, and clear a table with diverse novel objects—precisely the types of tasks that assistive robots need to perform in home environments. The key practical advance is that the pre-trained model provides robustness to the natural variation in these tasks (different clothing items, different food containers, different table clutter configurations) without requiring exhaustive per-task data collection. A home robotics company could maintain a single pre-trained π₀ model, fine-tune it to a specific user's home layout and commonly handled objects with a few hours of teleoperated demonstrations, and deploy a system that handles the most frequent ADL tasks at useful reliability. The paper's finding that pre-training helps most on tasks with novel objects (paper towel replacement: π₀ achieves ~0.55 vs. ~0.25 for scratch at 10 hours) is particularly relevant because home environments are defined by their novelty—no two kitchens have the same objects in the same configurations.

Data generation for robot self-improvement loops (analogous to LLM instruction tuning). The paper's pre-training/post-training recipe naturally enables an iterative self-improvement pipeline. After deploying a fine-tuned π₀ policy on a task, the robot will inevitably encounter failure cases—situations where the policy deviates from the demonstrated strategy, makes a mistake, and either recovers (good) or fails entirely (bad). These failure-and-recovery trajectories, when collected and filtered (keeping recoveries as new pre-training data, discarding terminal failures or relabeling them with corrective actions), can be added to the pre-training mixture for the next iteration, gradually expanding the model's repertoire of recovery behaviors. This is the robotics analog of how LLM instruction tuning datasets are grown by collecting model outputs, filtering for quality, and retraining. The paper's architecture supports this because the same model can serve as both the policy being improved and the base for the next pre-training iteration—unlike specialized dexterous manipulation methods (ACT, Diffusion Policy) that train from scratch on small task-specific datasets and cannot easily absorb new recovery trajectories.

On-device deployment of capable manipulation policies on low-cost hardware. The paper demonstrates that π₀ runs inference in 73 ms on a consumer-grade RTX 4090 GPU (Table I), enabling 50 Hz control. This latency budget leaves headroom for further optimization (quantization, distillation, hardware-specific compilation) that could bring the model to embedded platforms like the NVIDIA Jetson Orin, which is already used in low-cost robot research platforms (ALOHA, Mobile ALOHA). A distilled or quantized π₀ variant running on a $500 Jetson module could control a$ 5,000–10,000 bimanual manipulator (roughly the cost of the ALOHA hardware), bringing the total system cost well within reach of small businesses, research labs in developing countries, and eventually consumer settings. The paper's finding that the 470M parameter π₀-small achieves non-trivial performance (0.5–0.7 on easier out-of-box tasks, Figure 7) suggests that aggressive distillation could preserve much of the capability while dramatically reducing inference cost, though the paper does not explore this directly.

When to Prefer This Method

The paper's architecture and training recipe position it against several named alternatives, and the experiments provide evidence for when to choose which approach:

Prefer the π₀ architecture (VLM backbone + flow matching action expert) when:

The target task requires high-frequency dexterous control (20–50 Hz) with temporally coherent action chunks—autoregressive VLAs (OpenVLA, RT-2) fail on these tasks because discretization cannot represent the continuous, correlated action distributions needed (Figure 7: OpenVLA achieves near-zero on shirt folding and grocery bagging).
The deployment scenario involves multiple robot embodiments or the model will be fine-tuned to new downstream tasks—the cross-embodiment pre-training provides positive transfer, especially when fine-tuning data is limited (Figure 11: pre-trained π₀ with 1 hour of data outperforms scratch π₀ with 10 hours on some tasks).
The robot operates in environments with natural variation (different object instances, configurations, lighting) where robustness to off-distribution states matters—the pre-training/post-training recipe provides recovery behaviors that purely high-quality post-training data cannot (Figure 13: full recipe outperforms scratch by 20–35 percentage points on hardest tasks).
Language instruction following is required and the commands involve semantic reasoning about objects and their relationships—the VLM pre-training provides substantially better language grounding than training from scratch (Figure 9: π₀ achieves ~0.75–0.8 language following accuracy vs. ~0.4–0.5 for π₀-small).

Prefer specialized dexterous manipulation methods (ACT, Diffusion Policy trained from scratch) when:

The task is extremely narrow and well-defined, with clean expert demonstrations covering the full expected distribution of states—in this regime, the overhead of pre-training may not be justified, though the paper shows that pre-training still provides marginal gains even at 10 hours of fine-tuning data (Figure 11).
The deployment hardware has very limited compute—ACT and Diffusion Policy are smaller models with lower inference latency, though π₀-small (470M parameters with flow matching) may close this gap.

Prefer training from scratch on task-specific data (no pre-training) when:

The pre-training mixture available to you is fundamentally mismatched to your target domain (e.g., you are training a legged locomotion policy but only have arm manipulation pre-training data)—the paper provides no evidence that cross-domain transfer works across such large morphological gaps, and explicitly flags this as an open question (Section VII).
You have abundant high-quality data for your specific task (100+ hours) and the task is not particularly dexterous—the gap between pre-trained and scratch models narrows as fine-tuning data increases (Figure 11), and at very large fine-tuning dataset sizes, the marginal benefit of pre-training may not justify the infrastructure cost of maintaining a pre-training pipeline.

These tradeoffs are explicitly grounded in the paper's results rather than extrapolated from general principles. The primary limitation is that the "prefer specialized methods" condition is not directly tested: the paper compares π₀ against ACT and Diffusion Policy (Figure 11), but does not systematically vary the amount of task data available to each method to find the crossover point where specialized training from scratch matches pre-trained fine-tuning.

🎯 Pitch

1. Executive Summary

2. Context and Motivation

The Core Problem: Scaling Dexterous Robot Learning to General-Purpose Systems

Why This Problem Matters

Where Prior Approaches Fall Short

How This Paper Positions Itself

3. Technical Approach

3.1 Reader Orientation

3.2 Big-Picture Architecture (Diagram in Words)

3.3 Roadmap for the Deep Dive

3.4 Detailed, Sentence-Based Technical Breakdown

Flow Matching Action Generation: What the Model Outputs and Why

The Action Expert: Separating VLM Processing from Action Generation

Training Objective and Timestep Sampling

Observation Encoding and Cross-Embodiment Input Handling

Inference-Time Mechanics

Pre-Training and Post-Training Recipe

4. Key Insights and Innovations

Innovation 1: The Two-Phase Pre-Training/Post-Training Recipe as a Resolution to the Robustness-Dexterity Tension

Innovation 2: Flow Matching as the "Right" Action Representation for Unifying VLM Semantics with Dexterous Control

Innovation 3: The Action Expert as a Mechanism for Preserving VLM Pre-Training Under a Radically Different Training Objective

Innovation 4: Positive Transfer at Scale as an Empirical Discovery, Not an Assumption

5. Experimental Analysis

Evaluation Methodology

Main Quantitative Results

Out-of-Box Evaluation: Pre-Trained Model Performance (Section VI-A)

Language Instruction Following (Section VI-B)

Fine-Tuning to New Dexterous Tasks (Section VI-C)

Mastering Complex Multi-Stage Tasks (Section VI-D)

Ablation Studies and Robustness Checks

Critical Assessment

6. Limitations and Trade-offs

The Cost of Difficulty Estimation Is Unaccounted for in the Compute-Optimal Framework

The Revision Model Has a High Correct-to-Incorrect Reversion Rate

Hard Problems Receive Essentially No Benefit from Test-Time Compute

The FLOPs-Matched Pretraining Baseline May Be Artificially Weak

No Combination of Search and Revisions Is Evaluated

Single Benchmark and Single Model Family Limit Generality of Findings

7. Implications and Future Directions

How This Work Changes the Landscape

Follow-Up Research This Work Enables

Practical Applications and Downstream Use Cases

When to Prefer This Method

Adjacent in domain

A Survey of Small Language Models

Sortformer: A Novel Approach for Permutation-Resolved Speaker Supervision in Speech-to-Text Systems