ArXiv: 2507.16815

🎯 Pitch

ThinkAct discovers that giving a VLM reinforcement learning–based visual rewards grounded in physical trajectories—not just QA-style correctness—is the key to unlocking effective chain-of-thought reasoning for robot actions. This allows a 7B model to surpass prior VLAs by over 15% on long-horizon manipulation tasks and, remarkably, to self-correct its plans without any explicit error-recovery training.

1. Executive Summary

ThinkAct proposes a dual-system framework that bridges high-level embodied reasoning with low-level action execution via reinforced visual latent planning—an MLLM (Qwen2.5-VL 7B) generates chain-of-thought reasoning plans reinforced by action-aligned visual rewards (goal completion and trajectory consistency scores derived from 2D gripper trajectories), which are then compressed into a visual plan latent to condition a downstream DiT-based action model for robust manipulation. Evaluated across robot manipulation (SimplerEnv, LIBERO) and embodied reasoning (EgoPlan-Bench2, RoboVQA, OpenEQA) benchmarks, ThinkAct improves over its DiT-Policy baseline by 11–17% on SimplerEnv and outperforms prior VLAs like OpenVLA and CoT-VLA, achieving state-of-the-art overall scores—particularly a 15.3% success rate advantage on LIBERO-Long long-horizon tasks. The framework further demonstrates few-shot adaptation and self-correction behaviors, establishing that reinforced visual latent planning enables deliberative reasoning to substantively improve action execution only when the reasoning MLLM receives RL-based visual feedback that grounds plans in physical trajectories, rather than relying solely on QA-style accuracy rewards or supervised CoT annotations.

2. Context and Motivation

The Core Problem: VLAs Lack Deliberative Reasoning Before Action

The fundamental challenge this paper addresses is straightforward to state but deeply pernicious in practice: existing vision-language-action (VLA) models map perception directly to action without intermediate reasoning about what to do, why, and in what order. This end-to-end shortcut—training a model to predict the next 7-DOF control vector directly from camera pixels and a natural language instruction—works adequately for short-horizon tasks in narrow training distributions but fundamentally fails when agents must plan over multiple steps, adapt to novel environments, or recover from execution errors.

The authors articulate this gap explicitly in the introduction:

"Despite promising on short-horizon skills, the crucial capabilities to reason in diverse visual scenes and enable long-horizon planning remain limited due to the end-to-end fashion from visual and textual inputs to low-level actions."

This is not merely an accuracy problem; it is a structural deficiency. An end-to-end policy trained to regress action deltas from pixel embeddings has no mechanism for decomposing "put the book in the back compartment of the drawer" into sequential subgoals (approach book, grasp, move to drawer, open drawer, place, close drawer). It is a single function call from an embedding vector to a motor trajectory. When the environment changes—different lighting, a novel object shape, a distractor object—the mapping breaks because there is no intermediate semantic representation to absorb the variation.

Why This Problem Matters: The Gap Between Perception and Deployable Autonomy

The significance of this gap extends well beyond benchmark numbers. Systems trained end-to-end on demonstration data exhibit three characteristic failure modes that make them brittle in real-world deployment, and the paper explicitly targets each:

1. Long-horizon planning collapse. As task horizons grow, the number of possible action sequences explodes combinatorially. An end-to-end model must implicitly represent all of these possibilities in the geometry of a fixed-size embedding space. Without explicit decomposition into subgoals, the model's effective planning horizon is bounded by the temporal receptive field of its training distribution. The LIBERO benchmark directly tests this: its LIBERO-Long suite consists of tasks requiring 4–6 sequential subgoals (e.g., "pick up the book and place it in the back compartment"), and existing models struggle disproportionately on this split. The paper reports that OpenVLA achieves only 53.7% on LIBERO-Long versus 84.7% on LIBERO-Spatial (Table 1)—a 31 percentage point gap that reflects the planning horizon penalty.

2. Brittle generalization to novel environments. When an end-to-end policy encounters a scene with different lighting, a rearranged table, or objects it hasn't seen in training, there is no semantic buffer. The pixel-to-action mapping must generalize holistically, which fails catastrophically when multiple factors vary simultaneously. The SimplerEnv benchmark operationalizes this: its Variant Aggregation setting introduces simultaneous changes in color, material, lighting, and camera pose. The paper's baseline DiT-Policy drops from 56.0% (Visual Matching) to 48.2% (Variant Aggregation) on Google Robot tasks (Table 1). A reasoning model that explicitly represents "the coke can is on the left side of the table" is better equipped to handle the can being a different color than a model that has only learned a correlational mapping.

3. Inability to recover from errors. End-to-end policies lack metacognition. When an action fails—a grasp slips, an object drops, a collision occurs—there is no mechanism to detect the failure, diagnose its cause, and replan. The robot continues executing the remainder of a now-impossible trajectory. This is not merely an inconvenience; in deployment settings, it means a single early error cascades into complete task failure. The paper explicitly names this as a motivation (Section 1, Figure 1): enabling "self-correction behavior in physical AI scenarios."

These three failure modes are interconnected. Long-horizon tasks increase the probability of encountering a novel situation mid-execution, which increases the probability of an error, which—without recovery—dooms the entire sequence. Addressing any one in isolation leaves the others unresolved. The paper's dual-system architecture is designed to tackle all three simultaneously by interposing a reasoning layer between perception and action.

Prior Approaches and Where They Fall Short

The paper identifies three broad families of prior work, each with characteristic limitations that ThinkAct aims to surpass.

Approach 1: End-to-End VLA Models Trained on Demonstration Data

This family includes OpenVLA, RT-2, Octo, TraceVLA, and GR00T N1—models that initialize from pretrained vision-language models (typically Prismatic, Llama, or similar VLMs) and then fine-tune on large-scale robot demonstration datasets such as Open X-Embodiment (OXE) or DROID to directly predict actions. The key architectural pattern is: encode vision and language into a shared embedding space, then decode into continuous action coordinates (or discretized action tokens).

Where they fall short: The paper's critique is not that these models perform poorly—on the contrary, they achieve strong results on standard benchmarks—but that their architecture imposes a ceiling. The authors note (Section 2.1):

"these models predict actions directly from vision and language inputs, often bypassing structured planning or intermediate reasoning. As a result, their capability to handle complex instructions, long-horizon goals, or out-of-distribution scenarios remains limited."

There is an important nuance here that the paper does not state explicitly but is implied by the experimental design: end-to-end VLAs conflate two distinct competencies that benefit from different training signals. Producing a reasonable action trajectory requires (a) understanding what needs to be done (semantic planning) and (b) knowing how the specific robot kinematics map to motor commands (motor control). By training a single model on demonstration data alone, the semantic planning competence is bottlenecked by the coverage of the robot dataset, which is typically narrow—a few environments, a few dozen task types—compared to the breadth of semantic knowledge available in internet-scale vision-language data. The paper's dual-system design explicitly separates these: the MLLM handles semantic reasoning (trained with multimodal data including human videos and QA tasks), while the action model handles motor control (trained on OXE). This separation allows each component to be trained on the data best suited to its competence.

Approach 2: Supervised CoT for VLAs

This family—including ECoT, RAD, and CoT-VLA—attempts to inject reasoning into VLAs by generating chain-of-thought annotations (subgoal decompositions, spatial reasoning traces, visual subgoal images) and then training models to predict these intermediate outputs via supervised fine-tuning (SFT) before producing actions.

ECoT (Zawalski et al., 2024) generates textual subgoals by prompting an off-the-shelf MLLM and trains the VLA to predict these subgoals as an auxiliary task. RAD (Clark et al., 2025) extends this by using action-free human videos to generate reasoning traces, then mapping the reasoning to real actions using robot data. CoT-VLA (Zhao et al., 2025) replaces textual reasoning with visual subgoal frames generated by a video model, which are input to the action policy.

Where they fall short: The paper identifies two fundamental limitations:

First, data curation is expensive and introduces distributional bias. Generating high-quality CoT annotations requires prompting large MLLMs, filtering outputs for correctness, and ensuring alignment with actual executable actions. The paper states (Section 2.2):

"due to the high cost of producing high-quality reasoning traces, the resulting models are prone to overfitting to specific visual scenes or reasoning patterns."

This is a deep problem. When SFT data is constructed by prompting a frozen MLLM on the training distribution, the resulting reasoning traces reflect that MLLM's specific biases and limitations. The VLA trained on these traces learns to mimic the form of reasoning (e.g., "First, I will approach the object. Second, I will grasp it. Third...") without necessarily learning the substance (why those steps, in that order, for this scene). The model overfits to the surface patterns of the prompted CoT rather than developing genuine planning capability.

Second, SFT on static annotations provides no mechanism for the model to learn which reasoning leads to successful actions. The supervision signal is "did the model's output match the reference annotation?" rather than "did the reasoning lead to task completion?" This is the standard credit assignment problem: an intermediate reasoning step might be suboptimal or even incorrect, but if the reference annotation contains it, the model is trained to reproduce it. Conversely, a novel and effective reasoning strategy receives zero supervision signal if it deviates from the reference.

CoT-VLA's visual subgoal approach partially addresses the brittleness of textual reasoning by generating images, but introduces its own limitation: the visual subgoal generator is a separate frozen model trained on task-specific video data, which limits scalability to new task types.

Approach 3: RL-Based Reasoning for Vision-Language Tasks

Inspired by DeepSeek-R1 and related work, several recent approaches (Video-R1, Cosmos-Reason1, Reason-RFT, Visual-RFT) apply reinforcement learning—specifically GRPO (Group Relative Policy Optimization)—to incentivize chain-of-thought reasoning in multimodal LLMs for question-answering tasks. The core idea: rather than supervising the intermediate reasoning with human-written CoT traces, define a reward function based on answer correctness (and optionally format compliance), and let the model discover effective reasoning strategies through exploration.

Where they fall short: The paper identifies a critical mismatch for embodied domains (Section 2.2):

"their QA-formatted rewards cannot fully support long-horizon planning or establish grounding between reasoning and action execution."

This is the crux of the problem. QA-style rewards—"did the model output the correct multiple-choice letter?" or "does the predicted answer match the ground truth?"—are fundamentally outcome-based and operate in the space of text tokens. For embodied reasoning, the outcome is not a text answer but a physical action trajectory, and the quality of reasoning cannot be reduced to whether the final textual prediction matches a reference. A reasoning trace could correctly identify the subgoals but assign them in a physically impossible order. It could correctly plan the spatial trajectory but fail to account for a specific object's grasp affordances. It could describe the scene accurately but fail to ground descriptions in pixel coordinates that the action model can use.

Moreover, QA-style rewards are sparse: they provide feedback only at the end of the reasoning chain, with no signal about whether intermediate steps are progressing toward a physically feasible plan. This makes credit assignment across long reasoning traces (potentially dozens of reasoning steps for a long-horizon manipulation task) extremely difficult.

The paper's key insight is that the reward function for embodied reasoning must itself be grounded in physical action. Rather than rewarding textual correctness, reward the reasoning based on whether the planned trajectory matches demonstrated successful trajectories—in pixel space, using metrics that capture both endpoint accuracy (goal reward) and path feasibility (trajectory consistency reward).

How ThinkAct Positions Itself Relative to Existing Work

The paper constructs ThinkAct as a synthesis that addresses the limitations of each prior family while combining their respective strengths:

From end-to-end VLAs, ThinkAct inherits the action model trained on large-scale robot demonstration data (pre-training π_φ on OXE provides strong motor control priors). But instead of using this model in isolation, ThinkAct augments it with a reasoning-conditioned planning latent—turning an end-to-end policy into a reasoning-conditioned policy while keeping the action model's architecture intact.

From supervised CoT approaches, ThinkAct inherits the idea that intermediate reasoning—whether textual or visual—improves action prediction. But instead of training on static, prompted annotations, ThinkAct uses RL with action-aligned rewards to let the model discover effective reasoning strategies through exploration. This addresses both the overfitting problem (the model isn't forced to match a specific annotation style) and the credit assignment problem (the reward is based on task-relevant trajectory metrics, not token matching).

From RL-based visual reasoning, ThinkAct inherits the GRPO optimization framework and the idea of using reinforcement learning to elicit reasoning without step-level supervision. But instead of QA-style rewards, ThinkAct introduces action-aligned visual feedback—a reward function composed of goal completion (Equation 1: comparing predicted vs. detected start and end gripper positions) and trajectory consistency (Equation 2: Dynamic Time Warping distance between predicted and demonstrated trajectories). This grounds the reasoning in the physical constraints of the task.

The paper's title—"Reinforced Visual Latent Planning"—encodes this positioning precisely. "Reinforced" signals the RL-based training paradigm (as opposed to SFT). "Visual" signals that the planning operates in pixel-space trajectory coordinates (as opposed to abstract text subgoals or task-specific reward signals from a simulator). "Latent" signals the architectural choice to compress the reasoning into a fixed-dimensional embedding that conditions the action model (as opposed to generating raw actions autoregressively). "Planning" signals the focus on multi-step reasoning for long-horizon tasks (as opposed to single-step perception-to-action mapping).

The Deeper Theoretical Motivation: Two Competences Require Two Training Regimes

Framing the paper's motivation at a more fundamental level: ThinkAct is built on the premise that embodied reasoning and motor execution are qualitatively different competencies that benefit from different learning signals, and a single end-to-end training pipeline on demonstration data cannot optimize both simultaneously.

Motor execution is well-supervised by behavioral cloning: given an observation and a demonstrated action trajectory, the model learns to predict what an expert would do. The learning signal is dense (per-timestep action error) and directly relevant to the output space. Large-scale demonstration datasets provide broad coverage of manipulation skills across embodiments.

Embodied reasoning—decomposing a high-level instruction into subgoals, anticipating obstacles, allocating attention to task-relevant objects, planning spatial trajectories—is poorly supervised by behavioral cloning on demonstration data. The "correct" reasoning intermediate is ambiguous: multiple valid reasoning paths can lead to the same action sequence, and the demonstration data contains only actions, not reasoning. Supervised fine-tuning on prompted CoT annotations bootstraps this signal but introduces annotation bias. RL with action-aligned rewards offers a third way: define what makes a plan good (it ends at the right place, follows a feasible path, leads to task completion) and let the model discover effective reasoning through exploration.

The experimental results bear out this theoretical framing. In the ablation study (Table 3 and Table A6), removing the trajectory reward or the goal reward individually causes noticeable performance drops, and removing both—leaving only QA-style rewards—reduces ThinkAct to near the SFT cold-start baseline. The paper states (Section 4.4):

"When both r_traj and r_goal are removed, leaving only QA-style reward from QA datasets, the model shows only marginal improvements over the SFT baseline, confirming that action-aligned visual feedback is critical for effective multi-step planning in embodied settings."

This is the paper's central empirical claim about motivation: the form of the reward function matters decisively. QA-style rewards, while effective for eliciting reasoning in text domains, are insufficient for embodied domains because they fail to ground reasoning in the physical constraints of the task. The action-aligned visual feedback provides this grounding by comparing planned trajectories to demonstrated trajectories in pixel space.

A Note on Scope: Where ThinkAct Does and Does Not Apply

The paper is explicit that ThinkAct is designed for tasks where:

  • The observation space includes visual input from which 2D trajectories can be extracted or predicted (camera-based manipulation settings).
  • There exists a dataset of demonstrated trajectories (even action-free human videos) from which reward signals can be computed via trajectory matching.
  • The task requires multi-step planning rather than reactive control (long-horizon goals with identifiable subgoals).

ThinkAct is not designed for tasks where reasoning is purely linguistic (the paper does not claim to improve on R1-style reasoning for math or coding), tasks where actions cannot be represented as 2D gripper trajectories on image frames (e.g., full-body navigation), or tasks where the action model's training data is insufficient for the motor control demands (the paper relies on OXE pre-training for the base action model). These boundary conditions are implied by the experimental design rather than stated explicitly, but they matter for understanding the scope of the contribution.

3. Technical Approach

This is an architecture and training methodology paper whose core idea is that a dual-system design—a slow-thinking reasoning MLLM connected to a fast-acting diffusion policy through a learned visual plan latent—plus an action-aligned RL reward grounded in physical trajectories, enables deliberative reasoning to substantively improve robot manipulation, whereas end-to-end VLAs and QA-style RL rewards are structurally incapable of this.

3.1 Reader Orientation

ThinkAct builds a two-part agent for vision-language-action tasks: an MLLM (multimodal large language model) that looks at the camera image and the task instruction, thinks through a plan in natural language and predicts a 2D trajectory on the image, and then compresses that thinking into a compact embedding vector; and a separate action model (a diffusion policy) that takes that embedding plus the raw perception inputs and outputs a sequence of executable 7-DOF robot actions. The problem it solves is that prior VLA models map perception directly to low-level actions without explicit planning, which fails catastrophically on long-horizon tasks, novel environments, and error recovery scenarios—ThinkAct solves this by forcing the system to reason before acting, where the quality of the reasoning is improved through reinforcement learning with a reward signal that measures whether the planned 2D trajectory matches demonstrated successful trajectories.

3.2 Big-Picture Architecture (Diagram in Words)

ThinkAct has five major components:

  1. Reasoning MLLM $\mathcal{F}_\theta$ — a multimodal LLM (initialized from Qwen2.5-VL 7B) that takes a visual observation $o_t$ and a language instruction $l$ as input. It autoregressively generates two output sequences: (a) reasoning steps $v_t$ (text chain-of-thought, e.g., "First I need to locate the book, then approach it, then grasp..."), and (b) a visual plan $c_t$ (a latent embedding vector distilled from the reasoning that captures the high-level intent and the predicted 2D gripper trajectory). During RL training, the visual plan is decoded into explicit 2D trajectory waypoints $\tau = [p_k]_{k=1}^K$ for reward computation; during action adaptation, the latent $c_t$ is used directly.

  2. Action-Aligned Visual Reward Function — computes two scalar scores from the MLLM's predicted trajectory $\tau$ and the ground-truth trajectory $\hat{\tau}$ (extracted from demonstration videos using an off-the-shelf gripper detector): a goal reward $r_{\text{goal}}$ that measures whether the predicted start and end positions match the reference, and a trajectory reward $r_{\text{traj}}$ that uses Dynamic Time Warping distance to measure whether the overall path shape matches the reference. Combined with a format correctness reward, this produces the total reward used for GRPO training.

  3. GRPO Reinforcement Learning Loop — samples a group of $M$ distinct responses from the MLLM, scores each with the action-aligned reward, computes an advantage $A_i$ for each response relative to the group mean, and updates $\mathcal{F}_\theta$ to increase the probability of above-average responses while regularizing against deviation from the original model via KL penalty.

  4. Visual Plan Latent $c_t$ — the hidden-state representation produced by the MLLM at the boundary between reasoning and trajectory prediction. This compact embedding (dimension $|c_t| \times d$) is used as a conditioning signal for the action model, bridging the gap between high-level semantic planning and low-level motor execution by injecting reasoning context into the action model's input space through a Q-Former-based latent projector.

  5. Action Model $\pi_\phi$ — a DiT-based (Diffusion Transformer) policy with 432M parameters, pre-trained on the Open X-Embodiment dataset. It takes as input: (a) the current visual observation $o_t$ encoded by a DINOv2 image encoder, (b) the language instruction $l$ encoded by a CLIP text encoder, and (c) the visual plan latent $c_t$ projected into its input space. It outputs a sequence of $N$ executable 7-DOF actions $[a_t]_{t}^{t+N}$. The action model is trained via imitation learning with frozen MLLM weights.

Information flows through the system in three distinct phases:

Phase 1 (MLLM Cold-Start Fine-Tuning): The MLLM $\mathcal{F}_\theta$ is initialized from Qwen2.5-VL 7B and fine-tuned on a supervised dataset containing 2D visual trajectories from OXE, QA tasks from RoboVQA and EgoPlan-IT, and chain-of-thought data from Video-R1-CoT. This teaches the model to (a) interpret visual trajectory representations, (b) produce reasoning in the correct output format with <reasoning> and <answer> tags, and (c) predict 2D gripper trajectory waypoints when given manipulation scenes.

Phase 2 (RL Training): The cold-started MLLM undergoes GRPO training on a mixture of manipulation trajectory data (OXE + Something-Something V2 human videos) and embodied QA data. For each input observation-instruction pair, the MLLM generates $M=5$ distinct responses (each containing reasoning text + predicted trajectory waypoints). Each response is scored using the action-aligned visual reward (Equation 3). The GRPO objective updates the model to favor responses with higher relative reward while constraining divergence from the cold-start model.

Phase 3 (Action Adaptation): The trained MLLM is frozen. For each observation-instruction pair in the target environment, the MLLM produces a visual plan latent $c_t$. This latent is pre-computed offline for efficiency and cached. The action model $\pi_\phi$ is then trained via imitation learning on the target environment's action demonstrations, conditioning on the visual plan latent through a Q-Former projector that is updated alongside the action model and base encoder. At inference time, the MLLM and action model can operate asynchronously: one reasoning step produces a latent that conditions $N$ actions (e.g., $N=75$ on LIBERO), enabling slow thinking with fast control.

3.3 Roadmap for the Deep Dive

  • First, the problem formulation (Section 3.1 in the paper) — the formal definition of the VLA reasoning task, the notation for observations, instructions, and actions, and the core architectural split between $\mathcal{F}_\theta$ (reasoning MLLM) and $\pi_\phi$ (action model). This establishes the interfaces between components.
  • Second, the reward function design (Section 3.2) — the goal reward (Equation 1), the trajectory reward (Equation 2), and the overall reward (Equation 3). This is the most conceptually novel component because it grounds RL in physical trajectory metrics rather than text correctness. Understanding the reward is prerequisite to understanding the RL objective.
  • Third, the GRPO training procedure (Section 3.2) — how the reward is used to update the MLLM, including the advantage computation, the clipped objective, and the KL penalty. This explains how the MLLM learns to reason without step-level supervision.
  • Fourth, the visual plan latent extraction (bridging Sections 3.2 and 3.3) — how the MLLM's textual reasoning is compressed into an embedding $c_t$, what that embedding represents, and why a compact latent is preferred over passing raw reasoning text to the action model.
  • Fifth, the reasoning-enhanced action adaptation (Section 3.3) — the architecture of the action model $\pi_\phi$, the Q-Former latent projector that injects $c_t$ into the action model's input, the imitation learning objective (Equation 5), the asynchronous inference design, and the training data and hyperparameters.
  • Sixth, the multi-stage training pipeline (Section 3.4) — the sequential cold-start → RL → action adaptation stages, what data is used at each stage, what parameters are updated, and the rationale for this ordering.

3.4 Detailed, Sentence-Based Technical Breakdown

Problem Formulation: The VLA Reasoning Task Structure

The paper defines the VLA reasoning task as follows: at each timestep $t$, the agent receives a visual observation $o_t$ (specifically, a single $224 \times 224$ RGB image from a third-person camera, following OpenVLA's convention) and a textual task instruction $l$ (e.g., "Pick up the book and place it in the back compartment"). The goal is to predict an action $a_t$, which can be either a textual command or—more critically for this paper—a 7-DOF control vector $[\Delta_x, \Delta_y, \Delta_z, \Delta_{\theta_x}, \Delta_{\theta_y}, \Delta_{\theta_z}, \Delta_{\text{Grip}}]$ representing the delta motion of the robot end-effector and the gripper state.

The key architectural decision is to decompose this prediction into two stages handled by separate models: an MLLM $\mathcal{F}_\theta$ that reasons about the high-level plan, and an action model $\pi_\phi$ that executes the low-level control. Formally:

ct=Fθ(ot,l)c_t = \mathcal{F}_\theta(o_t, l)

[at]tt+N=πϕ(ct,ot,l)[a_t]_{t}^{t+N} = \pi_\phi(c_t, o_t, l)

where $c_t$ is the visual plan latent (an embedding vector representing the high-level planning intent), and $[a_t]_{t}^{t+N}$ is a sequence of $N$ executable actions predicted from that single reasoning step.

Why this decomposition: the paper implicitly argues that semantic planning (understanding the task, decomposing into subgoals, anticipating spatial arrangements) and motor execution (producing smooth, kinematically feasible trajectories for a specific robot) require fundamentally different training signals and architectures. The MLLM benefits from internet-scale vision-language data and from RL with trajectory-matching rewards. The action model benefits from dense imitation learning on robot demonstration data. By separating them, each can be trained on the data best suited to its competence, and the interface—the visual plan latent—provides a learned bottleneck that captures the planning-relevant information while abstracting away embodiment-specific details.

A crucial design note: the reasoning and action execution can be asynchronous. One MLLM reasoning step produces a latent $c_t$ that conditions $N$ subsequent actions, where $N$ is set based on the average task length in each environment ($N=15$ for SimplerEnv, $N=75$ for LIBERO). This means the MLLM—which runs autoregressive text generation—can be invoked infrequently (every $N$ steps), while the action model—which is a feed-forward diffusion sampling process—runs at every timestep. This architectural choice directly enables the paper's framing of "slow thinking, fast control": the MLLM does the heavy cognitive lifting at a low frequency, and the action model handles real-time execution.

Reward Design: Action-Aligned Visual Feedback

The core technical innovation of the paper is the reward function used for RL training of the MLLM. Unlike prior RL-based reasoning work that uses QA-style rewards (binary correctness on a text answer), ThinkAct defines rewards that measure how well the MLLM's predicted 2D visual trajectory matches demonstrated successful trajectories.

What the MLLM Predicts During RL Training

During the RL training phase, the MLLM $\mathcal{F}_\theta$ is prompted to generate two outputs for each input $(o_t, l)$:

  1. Reasoning text $v_t$ — a natural language chain-of-thought describing the scene analysis, task decomposition, subgoals, and trajectory planning. The model is prompted to use expressions such as "let me think," "Hmm," "let's break it down," and to include self-reflection and verification.

  2. 2D trajectory waypoints $\tau = [p_k]_{k=1}^{K}$ — a sequence of $K$ points on the image plane, where $K=8$ (specified in Appendix A.1). Each $p_k \in [0,1]^2$ is a normalized pixel coordinate representing a keypoint along the gripper's planned trajectory from its current position ($p_1$, the start point) to the task completion position ($p_K$, the end point). The coordinates are normalized to $[0,1]^2$ relative to the image dimensions.

The trajectory is decoded from the model's text output: the model is trained to output the waypoints inside <answer></answer> tags in the format [(x1, y1), (x2, y2), ..., (x8, y8)].

Ground-Truth Trajectories

To compute rewards, the paper needs a reference trajectory $\hat{\tau} = [\hat{p}_k]_{k=1}^{K}$ for each training example. These are extracted from demonstration videos using an off-the-shelf gripper detector (from LLARVA, Niu et al., 2024) applied to each frame. For robot data from OXE, this detector identifies the gripper position. For human data from Something-Something V2, a hand detector (Shan et al., 2020) is used instead; if two hands appear, the one with the largest movement is selected.

From each video, the authors randomly sample 3 starting frames and simplify the subsequent gripper trajectories into $K=8$ keypoints using the Ramer–Douglas–Peucker (RDP) algorithm (following HAMSTER). Camera motion is handled through stabilization (following Magma's approach).

The Goal Reward

The goal reward measures whether the MLLM correctly predicts the spatial start and end positions of the gripper:

rgoal=12(f(p1,p^1)+f(pK,p^K)),where f(p,p′)=max⁡(0,1−∥p−p′∥22)r_{\text{goal}} = \frac{1}{2}\left(f(p_1, \hat{p}_1) + f(p_K, \hat{p}_K)\right), \quad \text{where } f(p, p') = \max\left(0, 1 - \|p - p'\|_2^2\right)

where $p_1$ is the predicted start position (the first trajectory waypoint from the MLLM), $p_K$ is the predicted end position (the last trajectory waypoint), $\hat{p}_1$ is the ground-truth start position from the detector, and $\hat{p}_K$ is the ground-truth end position.

What it computes: the function $f(p, p')$ is a similarity score for two normalized 2D points. When the Euclidean distance $\|p - p'\|_2^2$ is zero (perfect match), $f = 1$. As the distance increases, $f$ decreases linearly, reaching zero when the distance equals 1 (the width of the normalized image space). The $\max(0, \cdot)$ ensures the score never becomes negative. The overall goal reward $r_{\text{goal}}$ is the average of the start-point similarity and the end-point similarity.

Why this form: the squared-distance scoring function $1 - \|p - p'\|_2^2$ provides a smoother gradient than a binary within-threshold reward; predictions that are close to the target receive partial credit, which helps the RL optimization. The choice to score only the start and end positions (rather than all $K$ waypoints in the goal reward) reflects the intuition that task completion is fundamentally about where the gripper begins and ends—the intermediate trajectory shape is captured separately by the trajectory reward. Clamping to zero prevents the optimization from wasting capacity on precisely matching start/end for trajectories that are already far from the target.

The Trajectory Reward

The trajectory reward measures whether the overall shape of the predicted trajectory matches the demonstrated trajectory:

rtraj=max⁡(0,1−d(τ,τ^))r_{\text{traj}} = \max\left(0, 1 - d(\tau, \hat{\tau})\right)

where $d(\tau, \hat{\tau})$ is the Dynamic Time Warping (DTW) distance between the two trajectories $\tau = [p_1, ..., p_K]$ and $\hat{\tau} = [\hat{p}_1, ..., \hat{p}_K]$.

What it computes: DTW is a sequence alignment algorithm that finds the optimal non-linear warping path between two time series. Unlike Euclidean distance (which requires one-to-one point correspondences and penalizes temporal misalignment), DTW allows a point in one trajectory to match multiple points in the other, effectively handling variations in execution speed and temporal sampling. The resulting distance $d(\tau, \hat{\tau})$ is normalized between 0 (identical trajectories under optimal alignment) and some upper bound. The reward is $1 - d(\tau, \hat{\tau})$, clamped to non-negative values.

Why DTW: the paper explicitly states (Section 3.2) that the trajectory reward is designed to "regularize the predicted $\tau$ to match the distribution of demonstrated trajectory $\hat{\tau}$." DTW is chosen because gripper trajectories in manipulation tasks can vary in execution speed—the robot might move faster or slower through certain segments—but the spatial path shape should be consistent for successful completion. DTW is robust to these temporal variations. Alternative metrics like mean squared error between corresponding waypoints would penalize temporal shifts that are physically irrelevant to task success.

The Total Reward

The overall reward combines visual feedback with format compliance:

r=0.9rvisual+0.1rformat,where rvisual=ωgoalrgoal+ωtrajrtrajr = 0.9 r_{\text{visual}} + 0.1 r_{\text{format}}, \quad \text{where } r_{\text{visual}} = \omega_{\text{goal}} r_{\text{goal}} + \omega_{\text{traj}} r_{\text{traj}}

where $r_{\text{format}}$ is a format correctness score (rewarding proper use of <reasoning> and <answer> tags, following the convention from DeepSeek-R1), and $\omega_{\text{goal}} = \omega_{\text{traj}} = 0.5$ are the weighting coefficients for the two visual reward components.

What it computes: a scalar score in $[0, 1]$ where 0.9 of the weight comes from the equally-weighted combination of goal and trajectory rewards, and 0.1 from format compliance. The format reward serves as a regularizer to prevent the RL optimization from collapsing the output structure into unstructured or malformatted generations.

Why this weighting: the 0.9/0.1 split between visual and format rewards signals that task-relevant trajectory accuracy is vastly more important than output formatting. Within the visual reward, equal weighting of goal and trajectory components reflects the authors' finding (from the ablation study, Table 3) that both rewards are individually important—removing either causes a performance drop, and removing both causes the model to collapse to near the SFT baseline. The equal weighting is a simple design choice rather than a carefully tuned optimum; the paper does not report sweeps over alternative weightings.

Contrast with QA-Style Rewards

The paper's ablation study (Table 3) directly compares the full action-aligned reward against a version with only QA-style rewards (the w/o r_traj, r_goal variant). With only QA rewards, the model "shows only marginal improvements over the SFT baseline" on SimplerEnv (56.9 vs. 56.4), EgoPlan-Bench2 (47.2 vs. 46.4), and RoboVQA (58.3 vs. 57.9). This demonstrates that the trajectory-based visual feedback, not the RL framework itself, is responsible for the performance gains—GRPO with text-based QA rewards on embodied tasks produces minimal improvement because the reward signal is not grounded in the physical constraints of the manipulation task.

GRPO Training: How the Reward Updates the MLLM

The MLLM $\mathcal{F}_\theta$ is updated using Group Relative Policy Optimization (GRPO), a variant of policy gradient methods designed for language model fine-tuning. The procedure at each training step is:

  1. Sampling a response group. For a given input $(o_t, l)$, the current reference model $\mathcal{F}_{\theta_{\text{old}}}$ generates $M$ distinct responses $\{z_1, z_2, ..., z_M\}$ via autoregressive sampling with temperature 1.0 and top-p sampling with $p = 0.99$. Each $z_i$ is the full generated text, containing both the reasoning steps and the trajectory prediction. The paper uses $M = 5$ rollouts per input (Section 4.1: "rollout size 5").

  2. Computing rewards. Each response $z_i$ is evaluated using the reward function (Equation 3), producing a set of scalar rewards $\{r_1, r_2, ..., r_M\}$.

  3. Computing advantages. The advantage $A_i$ for response $i$ is defined as:

Ai=ri−mean({r1,...,rM})std({r1,...,rM})A_i = \frac{r_i - \text{mean}(\{r_1, ..., r_M\})}{\text{std}(\{r_1, ..., r_M\})}

where $r_i$ is the reward for the $i$-th response, $\text{mean}(\{r_1, ..., r_M\})$ is the group average reward, and $\text{std}(\{r_1, ..., r_M\})$ is the group standard deviation.

What it computes: the advantage is a standardized score measuring how much better or worse response $i$ is relative to the group average, expressed in standard deviation units. An advantage of +1 means the response was one standard deviation above the group mean; -1 means one standard deviation below.

Why standardization: by normalizing relative to the group rather than using absolute rewards, GRPO eliminates the need for a learned value function (unlike PPO). The relative comparison within each group of responses provides a stable learning signal even as the absolute reward scale changes during training. This is particularly important for ThinkAct because the reward function depends on distance metrics in pixel space, whose absolute scale can vary across different scenes, camera positions, and object scales—standardization per group makes the optimization invariant to these variations.

  1. The GRPO objective. The model $\mathcal{F}_\theta$ is updated by maximizing:

JGRPO(θ)=1M∑i=1M(Fθ(zi∣ot,l)Fθold(zi∣ot,l)Ai−βDKL(Fθ(zi∣ot,l)∥Fθold(zi∣ot,l)))\mathcal{J}_{\text{GRPO}}(\theta) = \frac{1}{M}\sum_{i=1}^{M} \left( \frac{\mathcal{F}_\theta(z_i | o_t, l)}{\mathcal{F}_{\theta_{\text{old}}}(z_i | o_t, l)} A_i - \beta D_{KL}(\mathcal{F}_\theta(z_i | o_t, l) \parallel \mathcal{F}_{\theta_{\text{old}}}(z_i | o_t, l)) \right)

where $\frac{\mathcal{F}_\theta(z_i | o_t, l)}{\mathcal{F}_{\theta_{\text{old}}}(z_i | o_t, l)}$ is the probability ratio (importance weight) between the current and reference model for response $z_i$, $A_i$ is the standardized advantage, $D_{KL}(\mathcal{F}_\theta \parallel \mathcal{F}_{\theta_{\text{old}}})$ is the Kullback-Leibler divergence between the current and reference model's output distributions, and $\beta = 1 \times 10^{-2}$ is the KL penalty coefficient.

What it computes: for each response, the first term increases the probability of responses with positive advantage (relative to the reference model) and decreases the probability of responses with negative advantage. The importance weight $\frac{\mathcal{F}_\theta}{\mathcal{F}_{\theta_{\text{old}}}}$ prevents the update from over-weighting responses where the current model has already diverged significantly from the reference. The second term, $-\beta D_{KL}$, acts as a regularizer that penalizes the current model for deviating too far from the reference distribution overall, regardless of which responses had high advantage.

Why this form: the importance-sampled advantage term is the standard policy gradient objective adapted for language generation—it allows off-policy updates using responses generated by the old model while correcting for the distribution shift. The KL penalty is critical for RL-based fine-tuning of language models because unconstrained policy gradient can cause the model to collapse to degenerate outputs that exploit reward function loopholes (reward hacking). In the context of ThinkAct, this might mean the model learning to output trajectory waypoints that score well under DTW but are physically nonsensical (e.g., all points clustered at the target location). The KL penalty keeps the model close to the cold-start distribution, which already produces reasonable trajectories. The specific value $\beta = 0.01$ is a standard choice from the GRPO literature.

Training details. The GRPO training runs for 6K iterations with a batch size of 64, learning rate $1 \times 10^{-6}$, and a maximum response length of 1024 tokens. For computational efficiency, videos are processed with up to 16 frames at a maximum resolution of $128 \times 28 \times 28$ pixels (temporal × spatial), while single images use $256 \times 28 \times 28$ pixels.

Data Mixture for RL Training

The RL training dataset is carefully constructed to provide both action-grounded and reasoning-intensive data (Section 4.1, Appendix A.2.2):

  • 2D visual trajectories (12.5K samples): from two robot datasets in OXE (fractal20220817_data and bridge subsets, selected for high quality and visually clear trajectories) and from Something-Something V2 human manipulation videos. These samples provide the primary action-aligned visual reward signal. The robot data grounds the reward in actual manipulation kinematics; the human video data broadens the diversity of manipulation patterns while being action-free (no robot actions, only visual trajectories for the hand).

  • RoboVQA (10K samples): long-horizon reasoning QA about robot manipulation videos. These are scored using the standard QA accuracy reward (ROUGE scores for free-form answers, following Video-R1).

  • EgoPlan-IT/Val (10K samples): egocentric planning QA for daily-life tasks. These use accuracy reward for multiple-choice questions.

  • RoboFail (0.5K samples): failure detection data reformatted as multiple-choice questions (300 question-answer pairs total, from 100 simulated + 30 real-world failure cases). This data teaches the model to reason about errors.

  • LLaVA-Video-178K (10K samples): general video understanding data including captions, open-ended questions, and multiple-choice questions.

Why this mixture: the manipulation trajectory data provides the action-aligned visual reward signal that the paper argues is essential for grounded reasoning. The embodied QA data (RoboVQA, EgoPlan, RoboFail) provides additional reasoning training that is still domain-relevant but uses the QA reward format—this maintains the model's general reasoning capabilities while the visual reward shapes its trajectory prediction quality. The LLaVA-Video-178K data serves as a general-purpose video understanding regularizer, preventing the model from over-specializing to manipulation scenes and losing its broader visual reasoning competence. The paper emphasizes that this diverse mixture "enables the model to plan both physically executable and semantically coherent, while also improving generalization to diverse real-world tasks" (Appendix A.2.2).

The prompt templates for each data type are provided in Table A4. For trajectory data, the model is prompted to output reasoning in <reasoning> tags and the 2D waypoints in <answer> tags. For QA data, the model is prompted to reason in <reasoning> tags and output the answer in <answer> tags, with the instruction to use natural thought expressions and include self-reflection.

The Visual Plan Latent: Bridging Reasoning and Action

After RL training, the MLLM $\mathcal{F}_\theta$ produces both reasoning text $v_t$ and a visual plan latent $c_t \in \mathbb{R}^{|c_t| \times d}$ (where $d$ is the hidden dimension of the MLLM, and $|c_t|$ is the sequence length of the latent representation). The paper is deliberately sparse on how exactly $c_t$ is extracted from the MLLM's forward pass, but the description in Section 3.2 establishes that:

"the MLLM $\mathcal{F}_\theta$ autoregressively generates a sequence of latent embeddings for reasoning $v_t \in \mathbb{R}^{|v_t| \times d}$ and visual plan $c_t \in \mathbb{R}^{|c_t| \times d}$, where the former is decoded to reasoning steps while the latter would be inferred into a text string of 2D points $\tau = [p_k]_{k=1}^K$"

What this means operationally: during training, the MLLM outputs text that includes both the reasoning and the trajectory waypoints. The "visual plan latent" $c_t$ is the hidden-state representation at the specific layers/tokens that correspond to the trajectory prediction portion of the output—essentially, the MLLM's internal state just before or during the generation of the waypoint coordinates. During RL, this latent is decoded (via the LLM's language modeling head) into text waypoints for reward computation. During action adaptation, it is the raw hidden-state representation that is fed to the action model without textual decoding.

Why a latent instead of text: passing the raw text of the predicted trajectory to the action model would be possible but introduces two problems. First, text is a variable-length, discrete representation that would require the action model to learn its own text encoding, adding unnecessary complexity. Second, the text representation loses the rich distributional information in the MLLM's hidden states—the latent captures not just the predicted waypoints but the model's uncertainty, the spatial context, and the planning rationale that led to those waypoints. The latent provides a fixed-dimensional, continuous, information-dense conditioning signal.

Why this is "planning": the latent $c_t$ is computed from the MLLM after it has gone through its full chain-of-thought reasoning process, including scene analysis, task decomposition, and trajectory planning. It therefore encodes not just a spatial path but the reasoning context that produced that path—the identification of task-relevant objects, the decomposition into subgoals, the anticipation of obstacles. When the action model conditions on $c_t$, it receives the distilled output of the MLLM's planning process.

Reasoning-Enhanced Action Adaptation: Training the Action Model

With the visual plan latent $c_t$ produced by the frozen MLLM, the second stage trains the action model $\pi_\phi$ to use this latent as a conditioning signal for action prediction.

Action Model Architecture

The action model $\pi_\phi$ is a DiT-based (Diffusion Transformer) policy, specifically a Transformer-based diffusion policy following the Diffusion Policy framework (Chi et al., 2023). It has 432M parameters. The diffusion process works as follows:

  1. Forward diffusion (training): A clean action sequence is corrupted by adding Gaussian noise over 1000 timesteps following a DDPM noise schedule. The model is trained to predict the noise added at each timestep.

  2. Reverse diffusion (inference): Starting from pure Gaussian noise, the model iteratively denoises over 20 DDIM steps to produce a clean action sequence.

Input Encoding

The action model's state encoder processes three input modalities:

  • Visual observation: A single $224 \times 224$ RGB image in third-person view (following OpenVLA's convention) is encoded by a frozen DINOv2 image encoder (Oquab et al., 2023).

  • Language instruction: The task instruction text $l$ is encoded by a frozen CLIP text encoder (Radford et al., 2021).

  • Visual plan latent: The latent $c_t$ from the MLLM is projected into the action model's input space through a Q-Former latent projector.

The paper specifies (Section 4.1) that the state encoder jointly encodes these inputs into 1024-dimensional embeddings.

The Q-Former Latent Projector

The Q-Former (Li et al., 2023) is a transformer-based module that uses a small set of learnable query tokens to attend to the visual plan latent $c_t$ and produce a fixed-size embedding. The paper specifies (Section 4.1) that the Q-Former uses 32 queries. This design:

  • Compresses variable-length latents into a fixed-size representation regardless of the MLLM's output sequence length.
  • Selectively attends to planning-relevant information in the latent through learned query embeddings.
  • Provides a learned interface between the MLLM's hidden state space and the action model's input space.

Why Q-Former: compared to simply mean-pooling the latent or using a linear projection, the Q-Former with learnable queries can learn to extract specific planning information (e.g., "where should the gripper go?" vs. "what object is being manipulated?") into distinct query slots. The 32 queries are a design choice balancing representation capacity with computational efficiency.

Training Objective and Procedure

The action model is trained via imitation learning with annotated action demonstrations:

LIL(ϕ)=E(oi,l,ai)[ℓ(πϕ(ct,oi,l),ai)]\mathcal{L}_{\text{IL}}(\phi) = \mathbb{E}_{(o_i, l, a_i)}\left[\ell\left(\pi_\phi(c_t, o_i, l), a_i\right)\right]

where $(o_i, l, a_i)$ are observation-instruction-action triples from the target environment's demonstration data, $\pi_\phi(c_t, o_i, l)$ is the action model's predicted action (conditioned on the visual plan latent, the observation, and the instruction), and $\ell(\cdot, \cdot)$ is the standard diffusion loss (mean squared error between predicted noise and added noise in the forward diffusion process).

What it computes: the model learns to predict, from noisy action sequences, the noise that was added during the forward diffusion process. By learning to denoise, the model implicitly learns the distribution of expert actions conditioned on the inputs. The conditioning on $c_t$ means the model learns which actions are appropriate given the high-level planning intent encoded in the latent.

Why diffusion for actions: diffusion models have proven effective for visuomotor policy learning because they (a) naturally handle multi-modal action distributions (multiple valid ways to achieve a task), (b) provide stable training compared to GANs or VAEs, and (c) allow flexible conditioning through cross-attention. The iterative denoising during inference (20 DDIM steps) adds negligible latency compared to the environment step time.

Training details (Section 4.1, Appendix A.1): the action model is pre-trained on OXE data for 120K iterations with batch size 256 and learning rate $2 \times 10^{-5}$ in a general-purpose phase. This pre-training uses the visual plan latents pre-computed by the cold-started MLLM (before RL). The model is then fine-tuned on the target environment (LIBERO subtasks) for 75K iterations with batch size 128. During action adaptation, the MLLM $\mathcal{F}_\theta$ is kept frozen—only the state encoder, Q-Former latent projector, and DiT-based action model are updated. The visual plan latents are pre-computed offline for all training observations to accelerate training.

Asynchronous Inference Design

A key architectural property that the paper emphasizes (Section 3.3, Figure 2 caption) is that "during inference, $\pi_\phi$ and $\mathcal{F}_\theta$ could operate asynchronously to enable slow thinking and fast control for VLA reasoning tasks."

This means: one forward pass of the MLLM $\mathcal{F}_\theta$ produces a visual plan latent $c_t$ that conditions the action model for the next $N$ timesteps (where $N=15$ on SimplerEnv and $N=75$ on LIBERO). The action model runs at every timestep to produce the next action, while the MLLM is only invoked every $N$ steps to produce a new plan. Between MLLM invocations, the robot executes $N$ actions using the same planning context.

Why asynchronous: the MLLM generates text autoregressively, which is inherently slow—each token requires a full forward pass. The action model uses iterative denoising (20 DDIM steps), which is also iterative but operates on a much smaller model (432M parameters vs. 7B). If the MLLM had to run at every timestep, the robot would be idle for the duration of text generation, making real-time control impossible. The asynchronous design decouples planning frequency from control frequency: planning happens at 1/$N$ times the rate of action execution. The paper's ablation (Appendix B.6) shows that $N=75$ on LIBERO achieves the best balance—$N=100$ degrades because reasoning becomes too stale (failures aren't detected in time), while $N=25$ incurs extra inference cost without substantial gains.

Multi-Stage Training Pipeline

The paper adopts a sequential three-stage training strategy (Section 3.4), following the paradigm established by Video-R1.

Stage 1: Cold-Start Supervised Fine-Tuning (SFT)

The MLLM $\mathcal{F}_\theta$ is initialized from Qwen2.5-VL 7B and fine-tuned on a supervised dataset for 20K iterations with batch size 32 and learning rate $1 \times 10^{-5}$ using DeepSpeed ZeRO-3.

The SFT dataset (Appendix A.2.2) comprises:

  • 30K 2D visual trajectories from OXE robot manipulation data.
  • 50K RoboVQA samples (embodied QA about robot manipulation).
  • 50K EgoPlan-IT samples (egocentric planning QA for daily tasks).
  • 165K Video-R1-CoT samples (chain-of-thought video reasoning data across math, spatial, OCR, and chart domains).

For the Video-R1-CoT data, which already includes CoT annotations, the model is prompted to output in <reason>...</reason><answer>...</answer> format. For the remaining datasets, the model receives the instruction: "Please directly provide your text answer within the <answer></answer> tags, without any reasoning process." This encourages the model to learn both reasoning (from the CoT data) and direct answering (from the QA data) with the appropriate output structure.

What this stage accomplishes: the SFT cold-start teaches the MLLM three competencies: (a) interpreting visual trajectory representations from robot manipulation scenes, (b) producing outputs in the correct format with reasoning and answer tags, and (c) maintaining general video understanding capabilities from the Video-R1-CoT data. The model learns to predict 2D gripper trajectory waypoints in the expected output format, providing a reasonable starting point for RL optimization.

Simultaneously (independently), the action model $\pi_\phi$ is pre-trained on the full Open X-Embodiment dataset (OXE) to establish strong motor control priors. This pre-training is done without the visual plan latent—the action model learns to predict actions from visual observations and language instructions alone.

Stage 2: Reinforced Fine-Tuning

The cold-started MLLM undergoes GRPO training as described above (Section 3.2, Section 3.4 detailed breakdown). This stage:

  • Runs for 6K iterations.
  • Uses batch size 64, learning rate $1 \times 10^{-6}$, rollout size $M=5$.
  • Uses the action-aligned visual reward (Equation 3) for trajectory data and QA-style accuracy/ROUGE rewards for embodied QA and video understanding data.
  • Uses $\beta = 0.01$ for the KL penalty in GRPO.
  • Sets a maximum response length of 1024 tokens.
  • Uses temperature 1.0 and top-p $p=0.99$ for rollout diversity.

The training data mixture is as described in the "Data Mixture for RL Training" section.

Stage 3: Reasoning-Enhanced Action Adaptation

The action model $\pi_\phi$ is fine-tuned on the target environment with the visual plan latent conditioning, as described in the "Reasoning-Enhanced Action Adaptation" section. During this stage, the MLLM $\mathcal{F}_\theta$ is frozen, and only the state encoder, Q-Former latent projector, and action model parameters are updated.

Why this ordering: the cold-start stage provides a reasonable initialization so the RL stage doesn't start from a model that produces random-format outputs (which would give no meaningful reward signal). The RL stage improves the MLLM's reasoning and trajectory prediction quality using action-aligned feedback. The action adaptation stage then trains the action model to leverage the improved planning latents. Critically, the action model is not exposed to the RL training—it only sees the frozen MLLM's outputs, which means the action model learns a policy conditioned on whatever planning quality the RL-trained MLLM provides, without the risk of co-adaptation between the two models during training.

This staged approach also has a practical benefit: the expensive GRPO training (which requires online sampling, reward computation, and policy gradient updates on a 7B model) only needs to be run once to produce the reasoning MLLM. Different action models can then be trained for different embodiments or environments by repeating only Stage 3 (action adaptation) with the same frozen MLLM, enabling what the paper calls "few-shot adaptation" (Section 4.5).

4. Key Insights and Innovations

The dominant paradigm for eliciting reasoning in multimodal LLMs—exemplified by Video-R1, Cosmos-Reason1, and Visual-RFT—applies GRPO-style reinforcement learning with outcome-based text rewards: did the model's final answer match the ground truth? Is the output format correct? These rewards are fundamentally decoupled from the physical world. A reasoning trace can be linguistically coherent, correctly identify subgoals, and still produce a trajectory that is kinematically impossible or spatially misaligned with the actual scene. The reward function does not know—and therefore cannot penalize—this disconnect.

ThinkAct's conceptual move is to recognize that the reward function for embodied reasoning must itself be embodied. The paper operationalizes this through two trajectory-based metrics computed entirely in pixel space: goal reward (Equation 1, start/end position accuracy) and trajectory reward (Equation 2, DTW path consistency with demonstrated trajectories). These rewards do not check whether the text reasoning sounds right; they measure whether the predicted gripper path aligns with what a successful execution actually looks like in image coordinates.

This is a fundamental shift in how we define the training objective for reasoning models in physical domains. Prior work either (a) used QA-style accuracy rewards that ignore physical feasibility, or (b) relied on task-specific simulator success rates that are unavailable for real-world deployment and provide sparse terminal feedback. The action-aligned visual reward occupies a middle ground: it is dense (every trajectory prediction produces a score), domain-general (computed from 2D trajectories detectable by off-the-shelf models), and physically grounded (the reward responds to spatial alignment, not linguistic similarity).

The ablation study (Table 3, Table A6) provides the decisive evidence that this conceptual shift matters empirically, not just philosophically. Removing both the goal and trajectory rewards—leaving only QA-style format and accuracy rewards—reduces ThinkAct to near the SFT cold-start baseline on SimplerEnv (56.9 vs. 56.4), EgoPlan-Bench2 (47.2 vs. 46.4), and LIBERO (81.6 vs. 79.1). The GRPO framework itself is not the contributor; the specific form of the reward is. This is not an incremental refinement of prior RL-based reasoning methods—it is a categorical change in the reward representation from text-space to action-space.

The significance extends beyond this paper's results. The action-aligned visual reward provides a template for future work on embodied reasoning RL: define reward functions in terms of physically measurable quantities (trajectory shapes, contact events, affordance predictions) rather than text outputs, and let the model discover reasoning strategies that optimize those quantities. This reframes RL for embodied reasoning away from "verify the answer" toward "verify the action."

Innovation 2: Dual-System Architecture as Learned Separation of Semantic Planning and Motor Control

The field's dominant approach to VLA models—OpenVLA, RT-2, Octo, TraceVLA, GR00T N1—treats the mapping from perception to action as a single end-to-end function approximated by one large model. This architectural choice conflates two qualitatively different competencies: semantic planning (understanding the task, decomposing into subgoals, identifying task-relevant objects, reasoning about spatial relationships) and motor execution (producing smooth, kinematically feasible joint trajectories for a specific robot embodiment). Training a single model on demonstration data forces the semantic planning competence to be learned from whatever planning-relevant signal incidentally exists in robot action trajectories—which is minimal, because demonstrations contain what the robot did, not why it chose those actions in that order.

ThinkAct's dual-system architecture makes this conflation explicit and then resolves it through learned separation. The Reasoning MLLM (7B parameters, trained with RL on trajectory-matching rewards) handles semantic planning. The Action Model (432M parameters, trained with imitation learning on OXE demonstrations) handles motor execution. The interface between them—the visual plan latent compressed through a Q-Former—serves as a learned bottleneck that captures exactly the planning-relevant information the action model needs, while abstracting away both the variable-length reasoning text and the embodiment-specific motor details.

This is not the same as prior pipeline approaches where an LLM produces text subgoals and a separate policy executes them. Critically, ThinkAct's MLLM and action model are jointly optimized through the latent interface. The Q-Former projector is trained during action adaptation—it learns to extract from the MLLM's hidden states whatever information most improves action prediction. The MLLM is frozen during this stage, but its RL training was already shaped by action-aligned rewards that correlate with downstream execution success. There is no hand-designed subgoal vocabulary or manually specified interface; the latent is a fully learned representation of "what the MLLM knows that the action model needs."

The significance of this separation is demonstrated most clearly in the few-shot adaptation results (Figure 5, Figure A9). When the action model is fine-tuned on only 5–10 demonstrations per task on LIBERO, ThinkAct substantially outperforms Magma (an end-to-end model that also conditions on visual traces) by 7.3–9.5% on LIBERO-Goal and LIBERO-Spatial. This is because the MLLM's reasoning competence—learned from diverse manipulation and human video data—transfers to the new task without retraining, while the action model only needs to adapt its motor execution to the new embodiment or environment. In an end-to-end model, both competencies would need to adapt simultaneously from the same 5–10 demonstrations, which is fundamentally harder.

This is a diagnostic finding with implications beyond this paper. It suggests that semantic planning and motor execution have fundamentally different sample complexity requirements—planning benefits from broad semantic data (human videos, embodied QA, internet-scale vision-language), while motor execution requires dense, embodiment-specific demonstrations. A single model trained end-to-end on demonstration data will be bottlenecked by whichever competence requires more data, and in practice, that bottleneck is almost always semantic planning (because robot demonstration datasets are orders of magnitude smaller than internet vision-language data). The dual-system design allows each component to be trained on data scaled to its needs.

Innovation 3: The Overfitting Diagnosis for Supervised CoT in Embodied Domains

Prior work on reasoning for VLAs—ECoT, RAD, CoT-VLA—treats chain-of-thought as an annotation problem: prompt a frozen MLLM to generate subgoals or reasoning traces, filter for quality, and train the VLA to reproduce these traces via supervised fine-tuning. This is the standard recipe from text-domain reasoning (e.g., training on DeepSeek-R1 distilled CoT outputs), and on its surface, it makes sense: if reasoning helps, provide examples of good reasoning and train the model to imitate them.

ThinkAct identifies a deeper problem with this approach that is specific to embodied domains and is not addressed by simply improving annotation quality. The paper frames it as an overfitting diagnosis (Section 2.2, expanded in the Technical Approach motivation): when CoT annotations are produced by a frozen MLLM prompted on the training distribution, they reflect that specific MLLM's visual interpretation biases, scene parsing tendencies, and reasoning style. A VLA trained via SFT on these traces learns to mimic the surface form of reasoning—the vocabulary, the subgoal decomposition pattern, the spatial description conventions—rather than developing a causal understanding of which reasoning leads to successful actions.

The consequence is a fundamental credit assignment blind spot. SFT provides a per-token supervision signal of "did the model's output match the reference annotation?" It cannot distinguish between (a) a reasoning step that is genuinely informative for action selection and (b) a reasoning step that is merely stylistically consistent with the CoT annotation but contributes nothing to downstream execution quality. Worse, it actively penalizes novel reasoning strategies that might be more effective than the prompted reference, because those strategies—by definition—deviate from the annotation.

ThinkAct's RL-based approach resolves this blind spot by replacing the annotation-matching objective with an action-grounded reward. The model is not trained to reproduce a specific reasoning trace; it is trained to produce reasoning that leads to trajectory predictions scoring well on goal and path alignment metrics. If the model discovers a novel reasoning strategy—a different subgoal ordering, a more efficient spatial decomposition—that produces a better trajectory, it receives higher reward and is reinforced, regardless of whether that strategy appeared in any training annotation.

The evidence for this diagnosis is both qualitative and quantitative. Qualitatively, Figure 4 and Figure A7 show that the RL-trained model produces reasoning that is substantively different from the SFT cold-start model—the RL model demonstrates future-state reasoning ("the robot is done with the first step, now it needs to...") and scene-grounded spatial analysis that the SFT model lacks, despite the SFT model having been trained on similar reasoning examples. Quantitatively, the ablation (Table 3) shows that the RL training provides gains beyond SFT only when the action-aligned rewards are present—QA-style rewards during RL do not substantially improve over the SFT baseline, suggesting that the annotation-matching signal (which is implicit in QA rewards) is insufficient.

This is a conceptual advance rather than a method advance: it reframes the problem of training VLAs to reason not as "how do we get better CoT annotations?" but as "how do we define a training objective that distinguishes effective reasoning from stylistic imitation?" This question generalizes beyond the specific GRPO + trajectory reward implementation. Any training method that replaces per-token SFT loss with an action-grounded outcome reward—whether RL, direct preference optimization, or reward-weighted regression—inherits this conceptual advantage. The paper does not make this generalization explicit, but it is the logical implication of the diagnosis.

Innovation 4: Verifier Over-Optimization Avoidance Through Dual-System Decoupling

The reference paper you analyzed earlier identified verifier over-optimization as the primary bottleneck for test-time compute scaling in LLMs: as search budgets increase, beam search and best-of-N begin exploiting quirks in the verifier (PRM) that produce high scores but incorrect solutions, causing performance to plateau or degrade. This phenomenon is well-documented in the RLHF and LLM-reasoning literatures.

ThinkAct encounters a structurally analogous problem but solves it through architecture rather than policy. The potential over-optimization loop would work as follows: if the same model both generates reasoning/trajectories AND executes actions, then RL training on trajectory-matching rewards could cause the model to produce reasoning outputs that score highly under DTW and goal metrics (e.g., degenerate trajectories that trivially match the reference in path shape) but fail to condition the action model effectively—because the reasoning has become decoupled from the physical constraints of motor execution. This is the same structural problem as verifier over-optimization: the reward signal can be exploited by optimizing the proxy metric (trajectory alignment) at the expense of the true objective (task success).

ThinkAct's dual-system architecture prevents this failure mode through decoupled optimization. The MLLM is trained with RL on trajectory-matching rewards (Stage 2). The action model is then trained separately via imitation learning on demonstration data, conditioning on the frozen MLLM's outputs (Stage 3). Critically, the action model's loss function is not the trajectory reward—it is standard behavioral cloning loss against expert actions. This means the action model serves as an implicit verifier of the MLLM's reasoning: if the MLLM produces a visual plan latent that is poorly aligned with executable actions, the action model will have high imitation loss, and the system will fail. But this failure is detected at the action model's training stage, not at the MLLM's RL stage, preventing the RL optimization from exploiting the trajectory reward in ways that degrade action quality.

Put differently: the trajectory reward shapes the MLLM's planning, but the action model's imitation loss is the true downstream constraint. By training the action model on the frozen MLLM's outputs rather than jointly optimizing both models, ThinkAct ensures that the MLLM cannot gamify the action model's loss—the action model is always trained to match expert demonstrations given whatever planning input the MLLM provides, so degenerate planning inputs simply produce poor imitation and don't contaminate the MLLM's optimization.

This is a novel architectural solution to a known optimization pathology. Prior approaches to avoiding reward hacking in RL-tuned LLMs focus on regularization (KL penalties, reward clipping, constrained optimization) or reward model ensembling. ThinkAct's solution is structural: separate the model that receives the reward from the model that executes the actions, and optimize them with different objectives in sequence rather than jointly. This is only possible because of the dual-system architecture; a single end-to-end model receiving both trajectory-matching and imitation losses would face the standard over-optimization tension.

The paper does not explicitly frame this as verifier over-optimization avoidance, but the structural parallel is precise, and the empirical results support it. The RL-trained model substantially outperforms the SFT cold-start model across all benchmarks (Tables 1, 2, 3), with no evidence of the performance degradation that would indicate over-optimization. The asynchronous inference design—where the MLLM reasons at 1/N times the action frequency—further decouples the planning horizon from the control horizon, preventing the MLLM from needing to predict fine-grained trajectory details that would be most susceptible to reward hacking.

5. Experimental Analysis

Evaluation Methodology

  • Dataset. ThinkAct is evaluated on five benchmarks spanning two categories: robot manipulation (SimplerEnv, LIBERO) and embodied reasoning (EgoPlan-Bench2, RoboVQA, OpenEQA). SimplerEnv is a simulation benchmark built on WidowX and Google Robot setups, providing two evaluation settings per environment: Visual Matching (same objects, varied lighting/textures/backgrounds) and Variant Aggregation (introducing color, material, lighting, and camera pose variations simultaneously). It spans Google-VM, Google-VA, and Bridge-VM setups. LIBERO (Liu et al., 2023a) tests generalization across four structured task suites: LIBERO-Spatial (layout variation), LIBERO-Object (object diversity), LIBERO-Goal (goal variation), and LIBERO-Long (long-horizon planning with mixed variations, 4–6 subgoals per task). Each task suite is evaluated over 500 trials using 3 random seeds, following the protocol of CoT-VLA. EgoPlan-Bench2 (Qiu et al., 2024) evaluates egocentric planning across 24 scenarios in 4 daily-life domains (Daily life, Work, Recreation, Hobbies) using 1,321 high-quality multiple-choice QA pairs. RoboVQA (Sermanet et al., 2024) focuses on long-horizon reasoning in robotic manipulation with 1,893 video-text pairs in free-form QA format, evaluated using BLEU-1 through BLEU-4 scores and their average. OpenEQA (Majumdar et al., 2024) measures zero-shot embodied understanding across over 1,600 human-authored questions from 180+ real-world environments, spanning 7 subcategories (Object State, Object Recognition, Functional Reasoning, Spatial, Attribute Recognition, World Knowledge, Object Localization), evaluated using an LLM-based scoring metric aligned with human judgment.

  • Base model(s). The Reasoning MLLM $\mathcal{F}_\theta$ is initialized from Qwen2.5-VL 7B (Bai et al., 2025), a 7-billion-parameter multimodal LLM. The authors argue this model is representative of contemporary frontier MLLMs. Additionally, they demonstrate generalizability by reporting results with Qwen2.5-VL-3B (a smaller 3B variant) in Appendix B.4. The action model $\pi_\phi$ is a DiT-based Diffusion Policy (Chi et al., 2023) with 432M parameters, using a DINOv2 image encoder (Oquab et al., 2023) for visual observations and a CLIP text encoder (Radford et al., 2021) for language instructions.

  • Metrics. For manipulation tasks, the primary metric is task success rate (%), computed as the fraction of evaluation trials in which the robot successfully completes the specified task. For embodied reasoning tasks, the metrics vary by benchmark: EgoPlan-Bench2 uses accuracy (%) on multiple-choice questions (exact match between predicted and ground-truth option letters). RoboVQA uses BLEU-1 through BLEU-4 scores (Papineni et al., 2002) measuring n-gram overlap between predicted and reference free-form answers, with the overall score being the average across all four BLEU levels. OpenEQA uses an LLM-based scoring metric (0–100 scale for each question, averaged over all questions) where a judge LLM evaluates the semantic equivalence of the model's predicted answer to the ground-truth answer, following the benchmark's original protocol.

  • Baselines. ThinkAct is compared against two categories of methods. For robot manipulation (Table 1), baselines include: (1) DiT-Policy (the base action model $\pi_\phi$ without visual plan latent conditioning—this is the most direct ablation since it isolates the effect of the reasoning latent), (2) Octo-Base (Team et al., 2024b), (3) RT1-X (Brohan et al., 2022), (4) OpenVLA (Kim et al., 2024) — a widely-used end-to-end VLA built on Prismatic VLMs, (5) TraceVLA (Zheng et al., 2024) — which augments VLAs with visual trace prompting, (6) CoT-VLA (Zhao et al., 2025) — which uses visual subgoal frames as reasoning intermediates, and (7) Magma (Yang et al., 2025) — a foundation model for multimodal AI agents. For embodied reasoning (Table 2), ThinkAct is compared against: (1) GPT-4V (Achiam et al., 2023), (2) LLaVA-Video (Li et al., 2024a), (3) InternVL2.5 (Zhu et al., 2025), (4) InternVL3, (5) NVILA (Liu et al., 2024), (6) Qwen2.5-VL, (7) Qwen2.5-VL* (fine-tuned on EgoPlan-IT and RoboVQA datasets), and (8) Magma.

  • Generation budget / compute accounting. ThinkAct does not use a "generation budget" in the test-time compute scaling sense. Instead, the reasoning compute is determined by the architectural design: the MLLM is invoked once every $N$ actions, where $N=15$ on SimplerEnv and $N=75$ on LIBERO (chosen based on average task length in each environment). The action model runs at every timestep using 20 DDIM denoising steps per action prediction during inference. The paper reports inference speed comparison in Appendix B.7: on LIBERO tasks using an A100 GPU, ThinkAct takes 17% longer execution time than end-to-end OpenVLA, primarily due to the autoregressive reasoning process. This overhead is framed as a test-time compute investment that yields significant downstream performance gains (2.8–15.3% success rate improvements over OpenVLA across LIBERO tasks). The action model was trained on 100K OXE samples for pre-training (120K iterations) plus 75K iterations for LIBERO fine-tuning, all on 16 NVIDIA A100 GPUs with 80 GB memory, using batch size 256 (pre-training) or 128 (LIBERO fine-tuning) and learning rate $2 \times 10^{-5}$. The MLLM cold-start ran for 20K iterations with batch size 32 and learning rate $1 \times 10^{-5}$, and the RL stage ran for 6K iterations with batch size 64 and learning rate $1 \times 10^{-6}$. These training budgets are reported but not treated as variables in a scaling analysis.

  • Cross-validation / statistical protocol. For LIBERO, each task suite is evaluated over 500 trials using 3 random seeds, following prior work (CoT-VLA). For the few-shot adaptation experiments, the action model is fine-tuned on 10 demonstrations per task (and additionally 5 demonstrations in Appendix B.5) and evaluated over 100 trials. There is no cross-validation for the main benchmark results (Tables 1, 2); the models are evaluated on fixed test splits following each benchmark's standard protocol. The paper does not report confidence intervals, standard deviations, or statistical significance tests for any results.

Main Quantitative Results

Robot Manipulation Results

Table 1 reports task success rates on SimplerEnv and LIBERO. ThinkAct achieves the highest overall scores across all evaluation settings against all compared methods.

SimplerEnv — Google Robot Setup

On Google-VM (Visual Matching), ThinkAct achieves 71.5% overall, outperforming DiT-Policy (56.0%) by 15.5 percentage points and the next-best competitor CoT-VLA (68.4%) by 3.1 points. Breaking this down by task:

  • Open/Close Drawer: ThinkAct achieves 50.0% vs. DiT-Policy 44.9%, CoT-VLA 56.0%, and TraceVLA 57.0%. ThinkAct underperforms TraceVLA and CoT-VLA on this specific task.
  • Move Near: ThinkAct achieves 72.4%, substantially outperforming DiT-Policy (58.9%), CoT-VLA (65.4%), and all other methods. The improvement over DiT-Policy is 13.5 points.
  • Pick Coke Can: ThinkAct achieves 92.0%, dramatically outperforming DiT-Policy (64.3%) by 27.7 points and the next-best CoT-VLA (83.7%) by 8.3 points. This task shows the largest absolute improvement from reasoning conditioning.

On Google-VA (Variant Aggregation) — where visual conditions are deliberately varied to test robustness — ThinkAct achieves 65.1% overall, outperforming DiT-Policy (48.2%) by 16.9 points and CoT-VLA (62.6%) by 2.5 points. The gap between ThinkAct and DiT-Policy is larger in the Variant Aggregation setting (16.9 points) than in Visual Matching (15.5 points), suggesting that the reasoning latent is particularly helpful under distribution shift. By task:

  • Open/Close Drawer: ThinkAct achieves 47.6% vs. DiT-Policy 35.5% (+12.1 points). Magma achieves 53.4%.
  • Move Near: ThinkAct achieves 63.8% vs. DiT-Policy 52.8% (+11.0 points). CoT-VLA achieves 65.7%, slightly outperforming ThinkAct.
  • Pick Coke Can: ThinkAct achieves 84.0% vs. DiT-Policy 56.4% (+27.6 points) and CoT-VLA 68.8% (+15.2 points).

The pattern is consistent: ThinkAct's gains over DiT-Policy are largest on Pick Coke Can (27.6–27.7 points) and smallest on Open/Close Drawer (5.1–12.1 points), suggesting that reasoning provides the most benefit for tasks that require fine-grained manipulation of specific objects rather than broad spatial motions.

SimplerEnv — Bridge Setup

On Bridge-VM (Visual Matching), ThinkAct achieves 43.8% overall, outperforming DiT-Policy (32.4%) by 11.4 points. This is a harder environment (all methods score lower), and several baselines score near zero on specific tasks. By task:

  • Put Carrot on Plate: ThinkAct achieves 37.5% vs. DiT-Policy 29.4% (+8.1 points). OpenVLA achieves only 4.2%.
  • Stack Blocks: ThinkAct achieves 8.7% vs. DiT-Policy 0.0%. CoT-VLA and Magma are not evaluated on Bridge tasks (dashes in Table 1). The low absolute scores across all methods indicate this is an extremely challenging task.
  • Put Spoon on Towel: ThinkAct achieves 58.3% vs. DiT-Policy 34.5% (+23.8 points). This is the largest relative improvement on Bridge tasks.
  • Put Eggplant in Basket: ThinkAct achieves 70.8% vs. DiT-Policy 65.5% (+5.3 points).

A noteworthy detail: on Bridge tasks, the action model baselines that are not evaluated (CoT-VLA, Magma) have dashes in the table. The paper does not explain why these methods were not evaluated on Bridge, but it likely relates to embodiment-specific training requirements (Bridge uses a WidowX robot, while Google Robot tasks use a different platform).

LIBERO Benchmark

ThinkAct achieves the best overall success rate of 84.4% on LIBERO (Table 1, bottom section), outperforming DiT-Policy (76.8%) by 7.6 points and CoT-VLA (83.9%) by 0.5 points. By task suite:

  • LIBERO-Spatial: ThinkAct achieves 88.3% vs. DiT-Policy 82.6% (+5.7 points) and CoT-VLA 87.5% (+0.8 points). Magma is not evaluated on LIBERO-Spatial.
  • LIBERO-Object: ThinkAct achieves 91.4% vs. DiT-Policy 84.7% (+6.7 points) and CoT-VLA 91.6% (−0.2 points). CoT-VLA marginally outperforms ThinkAct on object variation tasks.
  • LIBERO-Goal: ThinkAct achieves 87.1% vs. DiT-Policy 75.1% (+12.0 points) and CoT-VLA 87.6% (−0.5 points). The gap between ThinkAct and DiT-Policy is largest in Goal tasks, where the task specification (rather than spatial layout or object appearance) varies, suggesting reasoning is particularly beneficial for understanding goal variations.
  • LIBERO-Long: ThinkAct achieves 70.9% vs. DiT-Policy 57.6% (+13.3 points) and CoT-VLA 69.0% (+1.9 points). OpenVLA achieves only 53.7%. The 13.3-point improvement over DiT-Policy on long-horizon tasks is the single largest gain across all LIBERO suites, directly supporting the paper's central claim that reasoning improves long-horizon planning.

The LIBERO-Long result deserves emphasis: the gap between ThinkAct and the baseline DiT-Policy grows from +5.7 points (Spatial) to +13.3 points (Long) as the task horizon increases. This is a monotonic relationship between planning horizon and reasoning benefit, which is precisely what the dual-system architecture is designed to produce.

A cross-comparison reveals an interesting pattern: on LIBERO, CoT-VLA and ThinkAct perform similarly overall (84.4% vs. 83.9%), with CoT-VLA slightly outperforming ThinkAct on Object and Goal suites while ThinkAct leads on Spatial and Long. Both reasoning-augmented methods substantially outperform the non-reasoning DiT-Policy baseline (76.8%). However, ThinkAct's advantage over CoT-VLA on LIBERO-Long (70.9% vs. 69.0%) suggests that the RL-based reasoning approach may scale better to longer horizons than CoT-VLA's visual subgoal generation.

Embodied Reasoning Results

Table 2 reports results on EgoPlan-Bench2, RoboVQA, and OpenEQA. ThinkAct achieves state-of-the-art or near-state-of-the-art performance across all three benchmarks.

EgoPlan-Bench2

ThinkAct achieves 48.2% overall accuracy, outperforming the second-best method Qwen2.5-VL* (45.7%) by 2.5 percentage points and the base Qwen2.5-VL (29.1%) by 19.1 points. This is a massive improvement over the base model, indicating that the RL-based reasoning training substantially enhances planning capabilities even in QA-style tasks where the action-aligned reward is not directly applicable (these tasks use accuracy rewards during RL). By domain:

  • Daily life: 50.1% (vs. Qwen2.5-VL* 47.9%, Qwen2.5-VL 31.4%)
  • Work: 49.8% (vs. Qwen2.5-VL* 46.3%, Qwen2.5-VL 26.7%)
  • Recreation: 44.8% (vs. Qwen2.5-VL* 44.3%, Qwen2.5-VL 29.5%)
  • Hobbies: 45.2% (vs. Qwen2.5-VL* 44.2%, Qwen2.5-VL 28.6%)

ThinkAct leads across all four domains, with the largest margin in Daily life and Work. Notably, Qwen2.5-VL* (which is the base Qwen2.5-VL fine-tuned on EgoPlan-IT and RoboVQA) already achieves 45.7%, a 16.6-point improvement over the base model—indicating that even supervised fine-tuning on embodied planning data provides substantial gains. ThinkAct's additional 2.5-point improvement comes from the RL training with action-aligned and QA rewards.

The paper also reports results with the 3B model variant (Appendix B.4, Table A5): ThinkAct-3B achieves 44.0% overall on EgoPlan-Bench2, outperforming the Qwen2.5-VL-3B* baseline (43.0%) and substantially outperforming InternVL2.5-2B (30.1%), InternVL3-2B (33.4%), and NVILA-2B (31.4%). This confirms that the approach generalizes to smaller model scales.

RoboVQA

ThinkAct achieves an overall BLEU score of 59.8 (averaged across BLEU-1 through BLEU-4), outperforming Qwen2.5-VL* (55.7) by 4.1 BLEU points and Qwen2.5-VL base (39.7) by 20.1 points. Breaking this down:

  • BLEU-1: 69.1 (vs. Qwen2.5-VL* 65.3, GPT-4V 32.2)
  • BLEU-2: 61.8 (vs. Qwen2.5-VL* 57.3, GPT-4V 26.5)
  • BLEU-3: 56.0 (vs. Qwen2.5-VL* 52.2, GPT-4V 24.7)
  • BLEU-4: 52.4 (vs. Qwen2.5-VL* 48.0, GPT-4V 23.9)

ThinkAct shows the largest relative improvement over Qwen2.5-VL* at higher n-gram orders (BLEU-4: +4.4 points; BLEU-1: +3.8 points), suggesting that the RL training improves not just content relevance (which BLEU-1 captures) but also the structural and phrasal quality of the generated answers (which higher BLEU orders capture better).

The ThinkAct-3B variant achieves 55.3 BLEU overall (Table A5), outperforming Qwen2.5-VL-3B* (53.6) and substantially outperforming InternVL2.5-2B (32.7), InternVL3-2B (33.8), and NVILA-2B (33.3). The gap between ThinkAct-3B and competing 2–3B models (21.5–22.6 BLEU points) is substantially larger than the gap between ThinkAct-7B and competing 7B models on this benchmark, suggesting that the reasoning training provides an even greater relative advantage at smaller model scales.

OpenEQA

ThinkAct achieves an overall score of 56.2%, outperforming InternVL3 (55.5) by 0.7 points and Qwen2.5-VL (50.8) by 5.4 points. By subcategory:

  • Object State: 70.0 (vs. InternVL2.5 70.2, InternVL3 68.9)
  • Object Recognition: 47.2 (vs. InternVL3 49.1)
  • Functional Reasoning: 53.2 (vs. InternVL2.5 56.2, InternVL3 54.6)
  • Spatial: 47.6 (vs. LLaVA-Video 46.2, InternVL2.5 44.1) — ThinkAct leads on spatial reasoning.
  • Attribute Recognition: 71.1 (vs. InternVL3 74.4)
  • World Knowledge: 58.6 (vs. LLaVA-Video 60.5)
  • Object Localization: 45.9 (vs. InternVL3 45.0)

ThinkAct does not dominate OpenEQA in the same way it does on EgoPlan-Bench2 and RoboVQA. It achieves the highest score on Spatial reasoning (47.6%) and Object Localization (45.9%)—both tasks that benefit directly from spatial trajectory planning—but falls short of InternVL3 on Object Recognition, Functional Reasoning, Attribute Recognition, and World Knowledge. This pattern is revealing: ThinkAct's training emphasizes spatial planning and trajectory prediction, and it excels on the subcategories that draw on these capabilities. On subcategories requiring factual knowledge or general visual recognition, the specialized training provides less benefit, and generalist models like InternVL3 maintain an advantage.

The overall 56.2% is a relatively narrow margin over several strong baselines (InternVL3 55.5%, InternVL2.5 54.4%, NVILA 54.0%), suggesting that on zero-shot embodied QA tasks, the reasoning training provides modest improvements over already-strong generalist MLLMs. This contrasts with the larger gains on manipulation and domain-specific reasoning benchmarks (SimplerEnv: +15.5% over DiT-Policy; RoboVQA: +20.1 BLEU over Qwen2.5-VL base).

Few-Shot Adaptation Results

Figure 5 and Figure A9 (Appendix B.5) evaluate ThinkAct's ability to adapt to new environments and tasks with limited demonstrations. On LIBERO with 10 demonstrations per task (Figure 5):

  • LIBERO-Spatial: ThinkAct achieves the highest success rate (exact value not stated in text; determined from bar chart comparison), outperforming Magma by 9.5 percentage points. This represents adaptation to novel spatial layouts.
  • LIBERO-Object: ThinkAct again leads, outperforming Magma. This represents adaptation to novel object types.
  • LIBERO-Goal: ThinkAct outperforms Magma by 7.3 percentage points. This represents adaptation to novel goal specifications.

With 5 demonstrations per task (Figure A9), the pattern holds: ThinkAct consistently outperforms comparative methods across all three LIBERO suites. The exact success rates are not reported in the text.

What these results demonstrate: the MLLM's reasoning competence—learned from diverse manipulation data during SFT and RL training—provides a strong prior that transfers to new environments with minimal action-model fine-tuning. The action model only needs to learn the motor mapping for the new setting, while the MLLM already provides task-decomposition and spatial-planning guidance. This is the key architectural advantage of the dual-system design: the reasoning module is environment-agnostic (operates in 2D pixel space), while the action model is embodiment-specific but can adapt quickly because it receives high-quality planning latents.

Self-Correction Capability

Figure 6 and Figure A8 (Appendix B.3) provide qualitative demonstrations of ThinkAct's self-correction behavior. In the main example (Figure 6), the robot attempts to place a box in a basket but accidentally drops the box midway. The reasoning MLLM receives a short video segment $o_{t-N:t}$ (extended from single-image to multi-frame input for this capability), identifies the failure ("Let's reconsider how to complete the task"), and generates a revised plan guiding the gripper back to the dropped object for regrasping. The robot then successfully completes the task. Additional examples in Figure A8 show: (a) failed mug grasp with adjustment suggestion, and (b) failed pickup detection with replanning.

Important caveat: the self-correction behavior is demonstrated qualitatively through selected examples. There is no quantitative evaluation of self-correction rates, such as: what fraction of failures does the system detect? What is the recovery success rate after detection? How does the recovery rate compare to baselines without reasoning? The paper claims self-correction as a capability in the abstract and introduction, but provides only anecdotal evidence. This is a significant experimental gap—the examples are compelling but do not constitute systematic evaluation.

Ablation Studies and Robustness Checks

Reward function components (Table 3, Table A6): The paper ablates the goal reward $r_{\text{goal}}$ and trajectory reward $r_{\text{traj}}$ on three benchmarks (SimplerEnv, EgoPlan-Bench2, RoboVQA in Table 3) and two additional benchmarks (LIBERO, OpenEQA in Table A6). The full ThinkAct achieves 60.1 (SimplerEnv), 48.2 (EgoPlan), 59.8 (RoboVQA), 84.4 (LIBERO), and 56.2 (OpenEQA). Removing the trajectory reward (w/o r_traj) reduces performance to 59.2 (−0.9), 47.9 (−0.3), 58.5 (−1.3), 82.1 (−2.3), and 55.9 (−0.3). Removing the goal reward (w/o r_goal) reduces performance to 59.1 (−1.0), 47.6 (−0.6), 58.9 (−0.9), 81.7 (−2.7), and 55.6 (−0.6). Removing both action-aligned rewards (w/o r_traj, r_goal)—leaving only QA-style and format rewards—reduces performance to 56.9 (−3.2), 47.2 (−1.0), 58.3 (−1.5), 81.6 (−2.8), and 55.7 (−0.5). The SFT cold-start baseline (no RL at all) achieves 56.4, 46.4, 57.9, 79.1, and 53.3 respectively. The critical finding: removing both trajectory rewards causes performance on SimplerEnv to drop from 60.1 to 56.9—almost entirely back to the SFT baseline (56.4), while on LIBERO the drop is from 84.4 to 81.6 (still above the SFT baseline of 79.1). On EgoPlan-Bench2 and RoboVQA, removing action-aligned rewards causes smaller but consistent degradation, likely because these QA-style benchmarks benefit partially from the improved reasoning the RL training induces through the QA reward component. On OpenEQA, the degradation from full ThinkAct to QA-only RL (56.2 → 55.7) is small, and the SFT baseline (53.3) is substantially lower, suggesting that even RL with QA rewards improves general embodied understanding over SFT alone.

Number of actions per reasoning step $N$ (Appendix B.6): The paper ablates the asynchronous frequency $N$—how many actions the action model executes between MLLM reasoning invocations—on LIBERO. Setting $N$ to 25, 50, 75, and 100 results in average success rates of 84.0%, 84.6%, 84.4%, and 83.7%, respectively. The sweet spot is $N=50$ (84.6%), with $N=100$ causing a 0.9-point drop and $N=25$ causing a 0.6-point drop from the $N=50$ optimum. The paper interprets the $N=100$ degradation as "overly sparse reasoning might cause the model to be unable to detect the failure and perform self-correction in time." The $N=25$ degradation is attributed to "additional inference cost without yielding substantial performance gains." The variation across $N$ values is modest (0.9 percentage point range), suggesting the system is relatively robust to this hyperparameter, with $N=75$ chosen as the reported setting at only 0.2 points below optimum.

Model scale generalization (Table A5): The paper evaluates ThinkAct with a smaller backbone, Qwen2.5-VL-3B, on EgoPlan-Bench2, RoboVQA, and OpenEQA. ThinkAct-3B achieves 44.0% (EgoPlan), 55.3 BLEU (RoboVQA), and 48.9 (OpenEQA), consistently outperforming Qwen2.5-VL-3B* (43.0%, 53.6, 44.6) and substantially outperforming InternVL2.5-2B, InternVL3-2B, and NVILA-2B. This provides evidence that the ThinkAct training recipe is not specific to the 7B model scale, though the paper does not evaluate ThinkAct-3B on manipulation tasks (SimplerEnv, LIBERO), which would require retraining the action model for the smaller MLLM's latent dimension, and this evaluation is absent.

RL training impact on reasoning quality (Figures 4, A7): Qualitative comparisons of ThinkAct with and without RL show that the RL-trained model produces substantively different reasoning—it engages in future-state reasoning, scene-grounded spatial analysis, and error anticipation that the SFT model lacks. In Figure 4(a) (RoboVQA), the SFT model "focuses only on the current state and fails to reason over future steps," while the RL-tuned model "successfully infers the correct answer." In Figure A7 (EgoPlan-Bench2), the RL model correctly predicts the next action while the SFT model's reasoning is described as insufficient. However, these are cherry-picked examples; there is no systematic classification of reasoning improvements or quantitative measurement of reasoning quality.

Baseline ablation (Tables 1, 2): The most important baseline for isolating the effect of the visual plan latent is DiT-Policy (the same action model architecture without $c_t$ conditioning). Across all SimplerEnv settings, ThinkAct outperforms DiT-Policy by 11.4–16.9 points (Table 1). On LIBERO, the improvement is 7.6 points overall. These gaps are the purest measure of the reasoning latent's contribution, since they hold the action model architecture constant.

Critical Assessment

Does ThinkAct Demonstrate That Reasoning Improves Action Execution?

The paper's central claim—that reinforced visual latent planning enables deliberative reasoning to substantively improve action execution—is supported by the DiT-Policy vs. ThinkAct comparison (Table 1), which shows 11.4–16.9 percentage point improvements on SimplerEnv and 7.6 points on LIBERO. This is a clean comparison: same action model architecture, same training data for the action model, differing only in whether the visual plan latent conditions the policy.

However, the claim is qualified by what the experiments do not test. ThinkAct is compared against DiT-Policy, which is its own action model without reasoning. But it is not compared against an alternative that adds reasoning through a different mechanism—for instance, an end-to-end model trained on CoT annotations (ECoT or RAD), or an action model conditioned on text-based reasoning outputs rather than a visual plan latent, or an action model conditioned on CoT-VLA's visual subgoals. The comparison to CoT-VLA (Table 1) partially addresses this, but CoT-VLA uses a fundamentally different architecture (visual subgoal generation rather than text reasoning). A more direct test would be: take the same DiT-Policy action model, condition it on the raw text reasoning output from ThinkAct's MLLM (without latent compression), and measure whether the latent compression provides benefits over raw text conditioning. This ablation is not reported.

Additionally, ThinkAct and CoT-VLA achieve nearly identical performance on LIBERO overall (84.4% vs. 83.9%). Both substantially outperform DiT-Policy (76.8%). This means that on the benchmark where both reasoning-enhanced VLAs are directly comparable, ThinkAct's RL-based visual latent planning provides only a 0.5-point advantage over CoT-VLA's visual subgoal generation—well within the range of statistical noise given the 500-trial evaluation. The advantage is more pronounced on LIBERO-Long (70.9% vs. 69.0%) and SimplerEnv (71.5% vs. 68.4% on Google-VM), but the overall picture is that reasoning—regardless of the specific mechanism—provides large gains over no reasoning, with the choice between RL-based latent planning and visual subgoal generation being secondary.

Does the Paper Demonstrate That Action-Aligned Visual Feedback Is Superior to QA-Style Rewards?

The ablation study (Table 3) shows that removing both $r_{\text{goal}}$ and $r_{\text{traj}}$ reduces SimplerEnv performance from 60.1 to 56.9 (near the SFT baseline of 56.4). This is the strongest evidence for the necessity of action-aligned rewards. However, the ablation is incomplete:

  • The paper does not ablate the DTW trajectory reward against simpler alternatives (e.g., mean squared error between corresponding waypoints, which would be computationally cheaper and easier to implement). The claim that DTW is essential is stated but not tested.

  • The format reward ($r_{\text{format}}$) is always present (weight 0.1), but its importance is not ablated. It is possible that the format reward alone—which encourages structured reasoning—contributes non-trivially to the RL gains.

  • The data mixture during RL training includes both trajectory data (which uses the action-aligned reward) and embodied QA data (which uses QA-style accuracy/ROUGE rewards). The ablation results reflect the contribution of the reward function on trajectory data, but do not disentangle whether the performance gains come from (a) the action-aligned reward signal, (b) the inclusion of trajectory data in the RL training mixture, or (c) the interaction between these two. A control where trajectory data is included in RL but scored with QA-style rewards would clarify whether it is the data or the reward that matters. This control is not reported.

On EgoPlan-Bench2 and RoboVQA—benchmarks that use QA-style evaluation—removing action-aligned rewards causes only small degradation (−1.0 and −1.5 points, respectively). This suggests that the action-aligned reward's importance is largest for manipulation tasks, which is expected but confirms that the benefit is domain-specific.

Does ThinkAct Actually Demonstrate Few-Shot Adaptation?

The few-shot adaptation experiments (Figures 5, A9) show ThinkAct outperforming Magma on LIBERO suites with 5–10 demonstrations. This is meaningful and supports the claim that reasoning aids adaptation. However, two limitations apply:

First, the adaptation is only tested on LIBERO, where ThinkAct already achieves strong full-dataset performance (84.4%). A more stringent test would be few-shot adaptation to a completely different embodiment or environment not represented in the MLLM's training data. Since the MLLM operates in 2D pixel space, it should theoretically generalize to any camera-based manipulation setting, but this is not verified empirically.

Second, the few-shot baseline is Magma, which is itself a reasoning-conditioned model (using visual trace conditioning). It is unclear how Magma was fine-tuned in the few-shot setting—whether its trace-conditioning module was also adapted, or only the action head. A comparison against DiT-Policy (the same action model as ThinkAct, without latent conditioning) would clarify how much of the few-shot advantage comes from the MLLM's planning prior versus from the action model's architecture.

Is Self-Correction Demonstrated or Merely Illustrated?

The paper claims that ThinkAct "enables capabilities of... self-correction behaviors in complex embodied AI tasks" (abstract) and that "failure detection and self-correction are critical for robust robot manipulation" (Section 4.5). The evidence consists of three qualitative examples (Figures 6, A8) showing the system recovering from dropped objects and failed grasps. These examples are selected and presumably represent successful cases. There is no systematic evaluation of:

  • Detection rate: What fraction of execution failures does the MLLM identify? What is the false positive rate (flagging non-errors as failures)?
  • Recovery rate: Given a detected failure, what fraction of recovery attempts succeed?
  • Comparison to baselines: Do other reasoning VLAs (CoT-VLA, ECoT) exhibit self-correction? Does an action model without reasoning but with a failure detection classifier achieve comparable recovery rates?
  • Video context ablation: The self-correction capability requires extending the MLLM's input from a single image to a video segment $o_{t-N:t}$ to provide temporal context. The paper does not ablate whether self-correction works with single-image input (presumably it would not, since failure detection requires observing the gripper's motion relative to the object). This design change is mentioned in passing but its necessity is not evaluated.

The self-correction claim should be interpreted as an existence proof—the system can, under some conditions, detect and recover from errors—rather than a demonstrated, reliable capability. The paper would be strengthened by even a modest quantitative evaluation (e.g., on the RoboFail dataset, which the model was trained on for failure detection).

Strengths of the Experimental Design

Despite the gaps noted above, the experimental design has several genuine strengths:

  • Multi-benchmark evaluation across two categories (manipulation and reasoning) provides evidence of generality beyond a single task type. The inclusion of benchmarks that test different aspects of embodied intelligence—long-horizon planning (LIBERO-Long), robustness to visual variations (SimplerEnv Variant Aggregation), spatial reasoning (OpenEQA Spatial), and free-form embodied QA (RoboVQA)—means that consistent gains across these settings are unlikely to be artifacts of a single benchmark's idiosyncrasies.

  • Clean within-architecture ablation (DiT-Policy vs. ThinkAct) isolates the contribution of the reasoning latent using the same action model, avoiding confounding from architectural differences. This is the strongest evidence in the paper.

  • The RL ablation (Table 3) directly tests the paper's core conceptual contribution—that action-aligned visual rewards matter more than QA-style rewards for embodied reasoning. The result (removing action-aligned rewards collapses performance to near SFT baseline on SimplerEnv) is clean and informative.

  • Model scale generalization (Table A5) shows the approach works with a 3B MLLM, though the gap between full ThinkAct and the SFT baseline is smaller at 3B scale (44.0 vs. 43.0 on EgoPlan) than at 7B scale (48.2 vs. 46.4), suggesting that the RL benefits may scale with model capacity—an interesting observation that is not discussed.

  • Inference cost transparency (Appendix B.7): reporting that ThinkAct takes 17% longer inference time than OpenVLA while providing larger performance gains helps practitioners assess the cost-benefit tradeoff. This is an uncommon and valuable disclosure.

Missing Experiments and Analyses

Several experiments would strengthen the paper's claims:

  1. Text reasoning conditioning baseline: Condition the action model on the raw text output of ThinkAct's MLLM (the reasoning text and trajectory text) instead of the visual plan latent, to isolate whether the latent compression provides benefits beyond text conditioning.

  2. Joint training ablation: Train the MLLM and action model jointly (with both RL and imitation losses) to test whether the decoupled training (Stage 2 → Stage 3) is necessary to avoid verifier over-optimization, or whether joint training with appropriate regularization would perform equivalently or better.

  3. Reward function sweep: Test alternative trajectory distance metrics (MSE between corresponding waypoints, Hausdorff distance, Fréchet distance) to determine whether DTW is genuinely advantageous or chosen for convenience.

  4. Difficulty stratification: Report performance stratified by task difficulty (e.g., short-horizon vs. long-horizon within each LIBERO suite) to quantify whether reasoning benefits scale with complexity, as the paper implies.

  5. Negative results with SFT-only CoT: Compare ThinkAct against an action model conditioned on CoT annotations from the same MLLM but trained only via SFT (without RL). This would test the claim that RL-based reasoning training provides benefits beyond SFT-based reasoning training.

  6. Statistical significance: Report confidence intervals or standard deviations for all main results, especially given the 500-trial LIBERO evaluation, where a 0.5-point difference between ThinkAct and CoT-VLA is reported as meaningful without statistical context.

  7. Quantitative self-correction evaluation: Even a modest evaluation on 20–50 failure cases, reporting detection rate and recovery rate, would transform the self-correction claim from anecdotal to empirical.

Overall, the experimental analysis provides strong evidence for the core claim that reasoning—operationalized as a visual plan latent from an RL-trained MLLM—improves robot manipulation performance over an equivalent non-reasoning action model. The experiments provide suggestive but incomplete evidence for the superiority of action-aligned visual rewards over alternative reasoning training approaches, for genuine few-shot adaptation (as opposed to fine-tuning on standardized benchmark splits), and for reliable self-correction behavior.

6. Limitations and Trade-offs

Limitation 1: Hallucinated Visual Reasoning from the Base MLLM Persists Through RL Training

The assumption or constraint. ThinkAct inherits Qwen2.5-VL 7B as its reasoning backbone, and with it inherits that model's tendency toward visual hallucination—incorrect object attributes, phantom spatial relationships, misidentified objects. The paper acknowledges this explicitly in Section 5 (Limitations):

"Since ThinkAct builds on pretrained multimodal LLMs, it inevitably inherits their limitations, particularly hallucinations in visual or spatial reasoning. This can lead to generated plans that reference incorrect object attributes or spatial relationships, affecting downstream execution."

This is not a minor edge case. The MLLM must correctly identify task-relevant objects in the scene, locate them in pixel coordinates, reason about their spatial relationships, and predict a physically plausible trajectory—all in one autoregressive generation. A hallucinated object location produces a trajectory pointing to empty space; a hallucinated object identity produces a plan to grasp the wrong item. The paper's proposed mitigation—"grounding-aware training or hallucination suppression in MLLMs"—is deferred entirely to future work, with no action taken in the current system.

The consequence. The failure mode has a specific structure: when the MLLM hallucinates, the visual plan latent $c_t$ encodes a plan that is spatially or semantically misaligned with the actual scene. The action model $\pi_\phi$, conditioned on this latent, will execute actions consistent with the hallucinated plan—grasping at empty coordinates, approaching the wrong object, or sequencing subgoals in an impossible order. Because the action model is trained via imitation learning on expert demonstrations conditioned on correct MLLM latents, it has no mechanism to detect or override hallucinated plans at inference time. The train-time distribution (correct plans → expert actions) diverges from the test-time distribution when hallucinations occur, and the action model's behavior under this distribution shift is uncontrolled.

Critically, the self-correction capability demonstrated in Figures 6 and A8—where the MLLM detects dropped objects and replans—does not address this failure mode. Self-correction detects execution errors (the gripper dropped the object). Hallucinated reasoning produces planning errors—the plan itself is wrong, even if perfectly executed. There is no mechanism for the MLLM to detect its own hallucinations, because the hallucinated perception is all it has access to. The system would confidently execute a hallucinated plan, and the self-correction module would see nothing to correct (the gripper followed the hallucinated trajectory correctly).

What evidence exists in the paper. There is no quantitative measurement of hallucination rates or their impact on task success. The qualitative examples (Figures 3, 4, 6, A7, A8) are all success cases where the MLLM correctly perceived the scene. The paper does not report a failure analysis categorizing error types (hallucination vs. motor error vs. planning error), which would be necessary to assess the prevalence of this limitation. The acknowledgment in the Limitations section is the only mention; it is a statement of awareness without empirical characterization.

Mitigation status. Not addressed in the current system. The paper suggests future work on "grounding-aware training or hallucination suppression in MLLMs" but proposes no concrete mechanism. The dual-system architecture provides some implicit mitigation—if the MLLM hallucinates an object location, the action model might still succeed if the hallucinated location happens to be close to the true location (the trajectory reward during RL training encourages spatial consistency). But this is incidental, not engineered, and there is no evidence it works in practice.

Limitation 2: The Difficulty Estimation Cost Is Not Accounted for in Any Efficiency Calculation

The assumption or constraint. ThinkAct's reasoning MLLM produces a visual plan latent at inference time through autoregressive text generation—a process fundamentally slower than the feed-forward action model. The paper reports this overhead quantitatively in Appendix B.7:

"On average, ThinkAct takes 17% longer execution time than OpenVLA, primarily due to the autoregressive reasoning process."

The 17% figure compares ThinkAct's total inference time against OpenVLA on LIBERO using an A100 GPU. However, this comparison has two critical features that limit its interpretability. First, the "17% longer" metric is expressed relative to total task execution time, which conflates model inference latency with environment step time (the robot moving, the simulator stepping). If environment step time is large relative to model inference—as is typical in robotic manipulation, where physical actions take hundreds of milliseconds—then a 17% increase in total time could represent a 2–5× increase in model inference time specifically. The paper does not report raw inference latencies for the MLLM and action model separately, making it impossible to assess whether the reasoning overhead would be prohibitive in low-latency settings (high-frequency control loops, real-time human-robot interaction).

Second, the 17% overhead is measured against OpenVLA, which is itself an autoregressive VLM (it generates discretized action tokens sequentially). A comparison against a feed-forward policy like DiT-Policy—ThinkAct's own action model without reasoning—would be more informative about the specific cost of adding reasoning, since it isolates the MLLM latency from the action model latency. This comparison is not reported.

The consequence. Two deployment scenarios are affected differently. For throughput-oriented batch evaluation (e.g., running LIBERO benchmarks on a GPU server), the 17% overhead is modest and likely acceptable given the performance gains (7.6–15.5 points on manipulation tasks). For latency-sensitive interactive deployment (e.g., a robot arm serving a human user, where each additional 100ms of planning time degrades the interaction quality), the absolute MLLM latency matters critically. A 7B-parameter model generating 100+ reasoning tokens with autoregressive decoding on an A100 will take on the order of hundreds of milliseconds to seconds per reasoning step—potentially unacceptable for real-time control loops.

The asynchronous inference design (one reasoning step every $N=75$ actions on LIBERO) partially mitigates this by amortizing the MLLM cost over many action steps. But this amortization comes at a cost: if the MLLM's plan becomes stale partway through the $N$ action steps (e.g., an unexpected obstacle appears, the gripper slips), the robot continues executing a now-invalid plan until the next MLLM invocation. The ablation on $N$ (Appendix B.6) shows that $N=100$ degrades performance compared to $N=50$, consistent with stale reasoning. This creates a fundamental accuracy-latency tradeoff that the paper does not characterize: smaller $N$ (more frequent reasoning) improves responsiveness but increases total inference cost; larger $N$ (less frequent reasoning) reduces cost but increases staleness risk.

What evidence exists in the paper. Only the single 17% figure in Appendix B.7, with no breakdown by MLLM vs. action model, no latency measurements at different $N$ values, no comparison against a non-reasoning feed-forward baseline, and no measurement on SimplerEnv (where $N=15$—more frequent reasoning—would incur higher relative overhead).

Mitigation status. The asynchronous design ($N > 1$) is the paper's primary mitigation. The $N$ ablation provides some guidance (choose $N$ based on task length), but this guidance is benchmark-specific and would require tuning for each new deployment. The paper does not explore techniques like speculative decoding, model distillation, or KV-caching across reasoning steps that could reduce the MLLM latency without increasing $N$.

Limitation 3: The Approach Severely Degrades on the Hardest Manipulation Tasks, and No Stratification Analysis Exists to Diagnose Why

The assumption or constraint. The paper's qualitative text and figures emphasize ThinkAct's strong performance on long-horizon and spatially complex tasks. But the raw numbers in Table 1 reveal substantial variation in absolute success rates that the paper never discusses. On SimplerEnv Bridge tasks:

  • Stack Blocks: 8.7% (ThinkAct) vs. 0.0% (DiT-Policy). Even with reasoning, the absolute performance is near floor.
  • Put Carrot on Plate: 37.5% (ThinkAct) vs. 29.4% (DiT-Policy).
  • Put Spoon on Towel: 58.3% (ThinkAct) vs. 34.5% (DiT-Policy).
  • Put Eggplant in Basket: 70.8% (ThinkAct) vs. 65.5% (DiT-Policy).

The gap between the easiest Bridge task (Put Eggplant: 70.8%) and the hardest (Stack Blocks: 8.7%) is 62.1 percentage points within the same benchmark, using the same model. On LIBERO, the spread between LIBERO-Object (91.4%) and LIBERO-Long (70.9%) is 20.5 points. This is not a uniform improvement envelope; ThinkAct helps substantially on some tasks and minimally on others, and the paper provides no framework for predicting which tasks will benefit.

The consequence. A practitioner deploying ThinkAct cannot predict whether the reasoning overhead (Section 6, Limitation 2) will be justified for a given task without running their own evaluation. The paper's headline numbers (71.5% Google-VM overall, 84.4% LIBERO overall) obscure the fact that ThinkAct is not uniformly effective—it provides large gains on some task types and near-zero gains on others. Without a difficulty-stratified analysis, it is impossible to determine whether failure on Stack Blocks (8.7%) represents a fundamental capability ceiling (the base action model cannot perform precise stacking regardless of reasoning quality) or an inadequate reasoning representation (the MLLM cannot represent the spatial precision needed for block stacking in 2D trajectory coordinates). These have different implications for future work: the former suggests investing in better action models; the latter suggests improving the reasoning representation.

More critically, the absence of stratification makes it unclear whether the observed gains are concentrated in easy-to-medium tasks, as was the case in the reference paper on compute-optimal test-time scaling (where gains on the hardest difficulty bin were near zero across all methods). If ThinkAct's reasoning benefits are largest on tasks where DiT-Policy already achieves moderate success (e.g., Pick Coke Can at 64.3% → 92.0%) and near-zero on tasks where DiT-Policy fails (e.g., Stack Blocks at 0.0% → 8.7%), this would indicate that reasoning amplifies existing capability rather than creating new capability—a boundary condition with significant practical implications.

What evidence exists in the paper. None. The paper reports per-benchmark averages and per-task scores on SimplerEnv but does not categorize tasks by difficulty, stratify results by any complexity metric (number of subgoals, spatial precision required, number of distractors), or analyze failure modes by task type. The qualitative examples are all success cases. There is no error taxonomy, no confusion matrix, and no analysis of which reasoning steps fail on which tasks.

Mitigation status. Not addressed. The paper treats all tasks within a benchmark as a homogeneous evaluation set and reports only aggregate metrics. This is standard practice in the VLA literature but limits the informativeness of the results for understanding where and why the method works.

Limitation 4: Self-Correction Is Demonstrated Qualitatively but Never Quantitatively Evaluated

The assumption or constraint. The paper prominently claims self-correction as an emergent capability. The abstract states ThinkAct "enables capabilities of few-shot adaptation, long-horizon planning, and self-correction behaviors in complex embodied AI tasks." Section 4.5 devotes a subsection to "Reasoning Elicit Self-Correction" and presents qualitative examples in Figures 6 and A8. The paper also trains the MLLM on the RoboFail dataset (failure detection data) and mentions that the MLLM receives "a short video segment $o_{t-N:t}$" (rather than a single image) to enable failure detection.

However, there is no quantitative evaluation of any self-correction metric anywhere in the paper. The claims rest entirely on three cherry-picked examples showing: (a) a dropped box being regrasped (Figure 6), (b) a failed mug grasp with an adjustment (Figure A8a), and (c) a failed pickup with replanning (Figure A8b).

The consequence. A practitioner evaluating ThinkAct for deployment cannot answer any of the following essential questions:

  • Detection sensitivity: What types of failures does the MLLM detect? Only dropped objects, or also collisions, incorrect grasps, missed approaches, incorrect object selection? The examples show only dropped-object failures. What is the false negative rate—what fraction of actual execution failures go undetected?
  • Recovery efficacy: Given a detected failure, what is the probability that the revised plan succeeds? The examples show 100% recovery (by selection). In practice, some failures are unrecoverable (the object fell off the table, the gripper damaged the target). What is the actual recovery rate?
  • False positive rate: Does the MLLM sometimes incorrectly flag successful execution as a failure and replan unnecessarily, potentially disrupting a trajectory that would have succeeded? This is not demonstrated but is a realistic failure mode.
  • Comparison to simpler alternatives: Could a simpler failure detection mechanism—a classifier trained on the RoboFail data, or a heuristic based on gripper-object distance—achieve comparable recovery rates without the MLLM inference cost? This baseline is absent.
  • Video context necessity: The paper notes that self-correction requires extending the MLLM's input from single image to video segment $o_{t-N:t}$. This is a non-trivial design change (it increases input tokens by a factor of up to $N$) that is not ablated. Could single-image input with explicit state tracking achieve similar results? Does the video input introduce its own latency and cost?

Without answers to these questions, the self-correction claim is an existence proof—the system can recover from some errors under some conditions—rather than a demonstrated capability with characterized reliability.

What evidence exists in the paper. Three qualitative examples (Figures 6, A8) and a textual description of the video-context design change (Section 4.5). No detection rates, recovery rates, false positive rates, baseline comparisons, or ablations related to self-correction. The RoboFail dataset used during training contains 300 QA pairs (100 simulated + 30 real-world failures, reformatted as multiple choice); this data exists but is not used for any downstream self-correction evaluation.

Mitigation status. Not addressed. The paper presents the qualitative examples as sufficient evidence for the claim, which they are not. The acknowledgment that the MLLM's hallucination tendencies could "result in unintended behaviors, especially in safety-critical settings" (Broader Impacts) implicitly recognizes that self-correction is unreliable, but this caveat is buried in a forward-looking impact statement rather than presented as a measured limitation of the current system.

Limitation 5: The 2D Trajectory Representation Is Both the Key Innovation and a Fundamental Bottleneck for 3D Manipulation

The assumption or constraint. The action-aligned visual reward—the paper's central technical innovation—is computed entirely in 2D image space. The MLLM predicts $K=8$ waypoints $p_k \in [0,1]^2$, normalized pixel coordinates on the camera image. The goal reward (Equation 1) measures start/end point distance in pixel space; the trajectory reward (Equation 2) measures DTW distance between 2D trajectory shapes. The visual plan latent $c_t$ encodes a 2D plan. The entire reasoning pipeline operates on 2D projections of a 3D world.

This representation is sufficient for tasks where depth variation is minimal relative to the task structure—pick-and-place on a tabletop, sliding objects across a surface, moving between fixed-height locations. But it is fundamentally insufficient for tasks requiring 3D spatial reasoning: stacking objects vertically (the height of placement matters), reaching into containers (occlusion + depth), manipulating articulated objects (opening a drawer requires reasoning about the drawer's axis of motion), or any task where the critical spatial relationship is orthogonal to the camera plane. The Stack Blocks task on Bridge (8.7% success for ThinkAct) is a potential victim of this limitation: block stacking requires precise 3D positioning that a 2D trajectory cannot fully specify.

The consequence. Two failure modes emerge from the 2D representation. First, depth ambiguity: the 2D trajectory indicates where the gripper should appear in the image, but not how far from the camera it should be. The action model must infer depth from monocular cues in the DINOv2 image embedding—a notoriously difficult problem. If the MLLM predicts a 2D point that corresponds to the correct pixel location but the action model misestimates the depth, the gripper will reach to the wrong 3D position. Second, occlusion blindness: when the target location is occluded in the camera view (e.g., inside a drawer, behind another object), the MLLM cannot predict a meaningful 2D trajectory to that location. The 2D representation has no mechanism for reasoning about occluded space.

These failures are not diagnosable from the aggregate metrics in Table 1. They would manifest as task-specific failures (low success on tasks requiring depth precision or handling occluded targets) that the paper does not stratify (see Limitation 3). The paper's choice of evaluation benchmarks may inadvertently mask this limitation: SimplerEnv and LIBERO tasks are predominantly tabletop manipulation with minimal occlusion and depth variation, where 2D trajectories are a reasonable approximation.

What evidence exists in the paper. Indirect evidence only. The low Stack Blocks performance (8.7%) on Bridge is consistent with a 2D representation failing to capture the vertical precision needed for stacking. The paper does not evaluate on any benchmark that explicitly tests 3D spatial reasoning (e.g., tasks requiring depth estimation, occluded grasping, or 3D trajectory specification). There is no ablation comparing 2D trajectory conditioning against 3D alternatives (e.g., predicting depth-augmented waypoints, conditioning on point clouds, or using multi-view trajectories). The choice of 2D representation is presented as a design decision, not as a limitation requiring justification.

Mitigation status. Not addressed. The 2D representation is presented as a feature (it enables the action-aligned visual reward using off-the-shelf 2D detectors) rather than as a constraint. The paper does not discuss extending to 3D, the tradeoffs involved, or which task types are excluded by this design choice. The use of human hand trajectories from Something-Something V2 (which are inherently 2D projections of 3D hand motion) further reinforces the 2D commitment without interrogation.

Limitation 6: The Approach Is Validated on a Single Model Family with No Evidence of Cross-Architecture Transfer

The assumption or constraint. Every experiment in the paper—manipulation and reasoning benchmarks, all ablations, all qualitative analyses—uses Qwen2.5-VL (7B or 3B) as the reasoning MLLM backbone and a DiT-based Diffusion Policy as the action model. The paper states in Section 4.1:

"We initialize $\mathcal{F}_\theta$ with Qwen2.5-VL 7B"

and provides only a single cross-model experiment: scaling down to Qwen2.5-VL-3B (Table A5). While this shows the approach works at a smaller scale within the same model family, it provides no evidence that the method transfers to MLLMs with different architectures (e.g., LLaVA-style models with different vision encoders, InternVL with dynamic resolution, or proprietary models like GPT-4V), different pretraining distributions, or different visual reasoning capabilities.

The consequence. Several components of ThinkAct may depend on properties specific to Qwen2.5-VL:

  • The visual plan latent extraction uses the MLLM's hidden states at the trajectory prediction tokens. The dimensionality, representational structure, and information content of these hidden states depend on Qwen2.5-VL's architecture (tokenizer, hidden dimension, number of layers, attention pattern). The Q-Former latent projector is trained to interface with this specific hidden state distribution. Switching to a different MLLM would require retraining the projector and possibly the action model.
  • The GRPO training hyperparameters—learning rate $1 \times 10^{-6}$, KL penalty $\beta = 0.01$, rollout size $M = 5$, 6K iterations—were tuned for Qwen2.5-VL 7B. Different models may have different optimal hyperparameters, different sensitivity to KL penalty, or different learning dynamics under RL. The paper provides no hyperparameter sensitivity analysis.
  • The reward function (Equations 1–3) uses normalized 2D coordinates and fixed weights ($\omega_{\text{goal}} = \omega_{\text{traj}} = 0.5$). These weights were not swept; they may be near-optimal for Qwen2.5-VL but suboptimal for other models with different spatial reasoning biases.
  • The cold-start SFT recipe uses a specific mixture of datasets (30K trajectories, 50K RoboVQA, 50K EgoPlan-IT, 165K Video-R1-CoT) and a specific prompting format (Table A4). A different MLLM might require different data mixtures or prompting strategies to achieve comparable cold-start quality.

This is not merely an academic concern about reproducibility. It means that a practitioner who wants to use a different MLLM—because of license restrictions, deployment constraints, or superior performance on their target domain—cannot assume that the ThinkAct recipe will transfer. Transferring to a new MLLM would require re-running the full three-stage pipeline (cold-start SFT, GRPO training, action adaptation) and retuning hyperparameters, which is computationally expensive (16 A100 GPUs, 20K + 6K + 120K iterations across stages).

What evidence exists in the paper. Only the Qwen2.5-VL-3B results (Table A5), which show the method works at a smaller scale within the same architecture family. The paper provides no results with InternVL, LLaVA, NVILA, or any other MLLM architecture despite comparing against several of these on embodied reasoning benchmarks (Table 2). The DiT-based action model is fixed throughout; there is no evaluation with alternative action model architectures (e.g., autoregressive action token prediction, energy-based models, or MLP-based policies).

Mitigation status. Not addressed. The paper does not discuss architecture dependence, recommend transfer procedures for new MLLMs, or characterize which MLLM properties are most important for ThinkAct's performance. The single-model-family validation is standard in the VLA literature (OpenVLA is validated on Prismatic; RT-2 on PaLI/PaLM) but limits the generality of the findings.

7. Implications and Future Directions

How This Work Changes the Landscape

ThinkAct introduces a reward representation shift for embodied reasoning that I believe will prove more consequential than the specific architectural choices or benchmark numbers in the paper. The dominant paradigm for eliciting reasoning in multimodal LLMs—exemplified by Video-R1, Cosmos-Reason1, Visual-RFT, and the broader DeepSeek-R1 lineage—treats reasoning as a text-generation problem evaluated by text-matching rewards: did the model output the correct multiple-choice letter? Does its free-form answer semantically match the ground truth? These rewards are purely linguistic. They cannot, by construction, distinguish between a reasoning trace that correctly analyzes a manipulation scene and one that is linguistically fluent but spatially nonsensical.

ThinkAct's central move is to recognize that the reward function for embodied reasoning must itself be embodied. The goal reward (Equation 1) and trajectory reward (Equation 2) are computed entirely in pixel space from 2D gripper trajectories. They ask: did the model's predicted path start at the right place? End at the right place? Follow a physically plausible shape relative to demonstrated expert trajectories? These questions cannot be answered by checking text tokens against a reference answer. They require comparing predicted coordinates—produced by the MLLM as part of its reasoning output—against coordinates extracted from actual manipulation videos using an off-the-shelf gripper detector.

This reframes RL-based reasoning for physical domains away from "verify the answer" toward "verify the action." The ablation study provides the decisive evidence that this reframing matters empirically, not just philosophically: removing both action-aligned rewards collapses SimplerEnv performance from 60.1 to 56.9—almost exactly the SFT cold-start baseline of 56.4 (Table 3). The GRPO framework itself is not the contributor; the specific form of the reward signal is. RL with QA-style rewards on embodied tasks produces minimal improvement because those rewards are structurally decoupled from the physical constraints the reasoning must respect.

This finding resolves a latent tension in the VLA reasoning literature that the paper does not explicitly name but that its results illuminate. Prior work on supervised CoT for VLAs (ECoT, RAD) demonstrated that adding reasoning traces as auxiliary supervision could improve action prediction—but only when the CoT annotations were carefully curated and domain-matched to the target tasks. This dependence on annotation quality was treated as a data engineering problem. ThinkAct's results suggest it is actually a reward specification problem: the CoT annotations in prior work were serving as a weak proxy for action-aligned feedback, and their brittleness reflected the mismatch between text-matching supervision and the physical objective of task completion. By replacing annotation-based supervision with trajectory-based RL rewards, ThinkAct achieves stronger performance (Table 1: 84.4% vs. DiT-Policy 76.8% on LIBERO) without requiring expertly curated CoT traces for each new task. This is not incremental—it changes what kind of data is needed to train reasoning VLAs from "annotated reasoning traces" to "demonstration videos with detectable trajectories," which is a substantially weaker requirement.

The paper also provides a diagnostic finding about the relationship between semantic planning and motor execution that extends beyond its own architecture. The few-shot adaptation results (Figures 5, A9) show that the reasoning MLLM—trained on diverse manipulation and human video data—transfers planning competence to new environments with only 5–10 action demonstrations, while an end-to-end model (Magma) adapts less effectively from the same limited data. This suggests that semantic planning and motor execution have fundamentally different sample complexity requirements: planning benefits from broad, embodiment-agnostic data (internet-scale vision-language, human activity videos, embodied QA), while motor execution requires dense, embodiment-specific demonstrations. A single end-to-end model trained on demonstration data will be bottlenecked by whichever competence requires more data—and in practice, that bottleneck is almost always semantic planning, because robot demonstration datasets are orders of magnitude smaller than internet vision-language data. The dual-system design makes this bottleneck explicit and resolves it by training each competence on data scaled to its needs.

This has direct implications for research prioritization. Less attractive: further work on scaling end-to-end VLA training on larger-but-still-narrow robot datasets, because the semantic planning bottleneck will persist regardless of how many additional demonstrations are collected. More attractive: research on (a) action-grounded reward design for embodied RL training of planning modules, (b) learned interfaces between planning and execution that transfer across embodiments, and (c) training data composition strategies that combine broad (internet-scale) semantic data with narrow (robot-specific) motor data. The dual-system architecture is one answer to these questions, but the paper's real contribution is establishing that these are the right questions to be asking.

Follow-Up Research This Work Enables

Grounded reward functions for 3D and force-domain reasoning. The paper's action-aligned visual reward operates entirely in 2D pixel space, using 2D trajectory waypoints as the action representation. This is sufficient for tabletop pick-and-place but fundamentally cannot represent depth precision (critical for stacking), occlusion (reaching into containers), or contact forces (grasping delicate vs. rigid objects). A direct extension would replace or augment the 2D trajectory reward with rewards computed from 3D quantities: point cloud registration between predicted and demonstrated end-effector trajectories, force-torque sensor alignment, or contact heatmap matching. The key question is whether off-the-shelf detectors exist for these modalities, as the 2D gripper detector does for pixel-space trajectories—if not, training such detectors becomes a prerequisite. A strong follow-up would evaluate on a benchmark that explicitly requires depth reasoning (e.g., RLBench tasks with stacking, peg insertion, or drawer opening) and compare 2D-only, 3D-only, and hybrid reward formulations, measuring both task success and the correlation between reward signal quality and downstream action model performance.

Cross-architecture transfer of the ThinkAct training recipe. Every experiment in the paper uses Qwen2.5-VL as the reasoning MLLM backbone. The visual plan latent extraction, the Q-Former projector, the GRPO hyperparameters, and the cold-start data mixture were all developed for this specific model. A necessary stress-test—which would determine whether ThinkAct is a method or merely a recipe—is replication with a substantially different MLLM architecture. The obvious candidates are InternVL3 (which outperforms ThinkAct on several OpenEQA subcategories in Table 2, including Attribute Recognition at 74.4 vs. 71.1) and LLaVA-OneVision (which has a different vision encoder and training distribution). The experiment would replicate the full three-stage pipeline (cold-start SFT, GRPO training, action adaptation) for each backbone, measuring: (a) whether the same hyperparameters transfer or require retuning, (b) whether the action-aligned reward produces similar gains over SFT baselines across architectures, and (c) whether the resulting visual plan latents are architecture-specific or converge to similar representations. A negative result—where the method works for Qwen2.5-VL but fails for InternVL or LLaVA—would indicate that the approach depends on specific properties of Qwen2.5-VL's pretraining (e.g., its spatial reasoning biases, its hidden state geometry, or its sensitivity to RL fine-tuning), refining our understanding from "ThinkAct works" to "ThinkAct works for VLMs with property X."

Quantitative evaluation of self-correction as a measurable capability. The paper claims self-correction as an emergent behavior but provides only three cherry-picked qualitative examples (Figures 6, A8). This is an existence proof, not a characterization. A rigorous follow-up would define self-correction operationally—detection rate (fraction of execution failures correctly identified), recovery rate (fraction of detected failures successfully resolved), and false positive rate (fraction of successful executions incorrectly flagged for replanning)—and measure these quantities systematically. The RoboFail dataset (0.5K samples used in ThinkAct's RL training, comprising 100 simulated and 30 real-world failure cases) provides a natural starting point: use it as a controlled failure-injection benchmark where the ground-truth failure state is known, and measure all three metrics. Additionally, instrument the LIBERO-Long evaluation (where ThinkAct achieves 70.9%) to automatically detect execution failures (gripper-object distance exceeding threshold, object dropping below table plane, task timeout) and classify each episode as (success, undetected failure, detected-and-recovered, detected-but-failed-recovery, false-positive-replan). This would transform self-correction from an anecdote to a characterized property with known reliability bounds, which is prerequisite for any deployment claim.

Replacing the visual plan latent with explicit textual or visual subgoal conditioning. ThinkAct compresses the MLLM's reasoning into a fixed-dimensional latent embedding $c_t$ and conditions the action model on this latent through a Q-Former projector. The paper argues this is superior to raw text conditioning but provides no direct ablation. A critical experiment would condition the same DiT-based action model on the raw text output of ThinkAct's MLLM—the reasoning text plus the predicted 2D trajectory coordinates—encoded through a frozen text encoder (the CLIP text encoder already in the action model's state encoder). This would isolate whether the latent compression through the Q-Former provides any benefit over simply letting the action model read the reasoning. A further comparison against CoT-VLA's visual subgoal approach—conditioning the action model on generated subgoal images rather than text or latents—would triangulate which reasoning-to-action interface works best. If text conditioning matches latent conditioning, the Q-Former is unnecessary complexity. If visual subgoal conditioning outperforms both, it suggests the action model benefits from spatially explicit rather than linguistically mediated planning information. This ablation has direct engineering implications for anyone building on the dual-system architecture.

Reinforcement learning with sim-to-real trajectory rewards. The paper's action-aligned reward depends on ground-truth trajectories extracted from demonstration videos using an off-the-shelf gripper detector. This works for offline training on curated datasets but does not provide a path for online RL in new environments where no demonstration trajectories exist. A natural extension would replace the trajectory-matching reward with a task-completion reward obtained from a simulator: the MLLM predicts a 2D trajectory, a simulated robot executes actions conditioned on that trajectory, and the reward is binary task success from the simulator. This would enable online RL where the MLLM improves its planning through interaction rather than through static trajectory matching. The key challenge is simulation fidelity: the 2D trajectory must correspond to actions that succeed in simulation, and the simulation must be sufficiently realistic that policies trained within it transfer to real robots. The SimplerEnv benchmark—which is already a simulation environment—provides a testbed: run online GRPO where each MLLM rollout produces a trajectory that conditions the action model, execute in simulation, and reward task success rather than trajectory similarity. Compare against the offline trajectory-matching approach on the same SimplerEnv tasks to quantify the benefit (or cost) of interactive training. If online RL with task-completion rewards outperforms offline trajectory matching, this opens a path toward continual improvement in deployed systems. If it underperforms (due to simulation bias or reward sparsity), it identifies a fundamental limitation of current sim-to-real transfer for reasoning policies.

Difficulty-stratified analysis to identify where reasoning helps and where it does not. The paper's aggregate results mask enormous variation in ThinkAct's effectiveness across tasks: 92.0% on SimplerEnv Pick Coke Can vs. 8.7% on Bridge Stack Blocks (Table 1). This 83.3-point spread within the same model and training pipeline is never analyzed. A diagnostic follow-up would categorize all evaluation tasks by measurable difficulty features—number of subgoals, required spatial precision (pixel-level tolerance for success), presence of distractors, degree of occlusion, and baseline DiT-Policy success rate—and then compute the reasoning benefit (ThinkAct success minus DiT-Policy success) as a function of each feature. The hypothesis from the reference paper on compute-optimal test-time scaling is that reasoning helps most on medium-difficulty tasks (where the base policy has non-trivial but imperfect success) and least on very easy tasks (already saturated) and very hard tasks (beyond the base policy's capability). If this pattern holds, it would mean ThinkAct is amplifying existing capability rather than creating new capability—a boundary condition with direct deployment implications. If the pattern does not hold (e.g., reasoning helps even on near-zero-baseline tasks), it would indicate the MLLM is contributing novel planning competence beyond what the demonstration data provides to the action model. Either outcome refines our understanding of how reasoning interacts with motor execution.

Practical Applications and Downstream Use Cases

Cost-efficient batch annotation of robot demonstration data with reasoning traces. ThinkAct's MLLM can generate visual trajectory plans and reasoning text for any manipulation video given only the first frame and a task instruction. This provides a path to automatically annotating large robot demonstration datasets (OXE, DROID, BridgeData) with structured reasoning traces—subgoal decompositions, spatial analyses, and trajectory predictions—without human annotation. The key cost is MLLM inference (17% overhead per the paper's measurement on LIBERO, amortized across $N$ actions), which is orders of magnitude cheaper than having human annotators write reasoning traces for each demonstration. These auto-generated traces could serve as cold-start training data for other VLA systems that want to incorporate reasoning, or as a filtering mechanism to identify demonstrations where the MLLM's plan and the actual executed trajectory diverge (potential annotation errors or unusual recovery behaviors worth human review). The paper's specific result that ThinkAct achieves 59.8 BLEU on RoboVQA and 48.2% on EgoPlan-Bench2 (Table 2) indicates the MLLM produces reasoning of sufficient quality to be useful for downstream tasks—these are strong absolute numbers for open-source 7B models on these benchmarks.

Fine-tuning data generation for few-shot robot skill acquisition. The few-shot adaptation results (Figures 5, A9) demonstrate that ThinkAct's reasoning MLLM transfers planning competence to new environments from only 5–10 action demonstrations, outperforming Magma by 7.3–9.5%. For a robotics lab or company deploying a robot to a new task, this means the process could be: (1) collect 5–10 human-teleoperated demonstrations of the new task, (2) run ThinkAct's frozen MLLM on the first frame of each demonstration to generate reasoning plans and trajectory predictions, (3) fine-tune the action model on the 5–10 demonstrations conditioned on these pre-computed plans, (4) deploy. The key engineering requirement is the action model pre-training on OXE—which provides the broad motor control prior—and the frozen MLLM—which provides environment-agnostic planning. This workflow potentially reduces the demonstration burden for new skills by a factor of 5–10× compared to training an end-to-end policy from scratch, based on the gap between ThinkAct's 5-shot performance and the full-dataset baseline (DiT-Policy at 76.8% on LIBERO requires full dataset training; ThinkAct approaches competitive performance with just 5–10 demos per task, though exact few-shot success rates are not reported in the text).

Failure detection and escalation in semi-autonomous robot fleets. The self-correction capability—even with the important caveat that it is only qualitatively demonstrated—suggests a deployment model for semi-autonomous robot fleets (warehouse picking, lab automation, assistive robotics) where the robot operates autonomously for routine execution but escalates to a human operator when the MLLM detects an execution failure and cannot recover autonomously. In this model, the MLLM's video-context reasoning (Section 4.5: input extended from single image to $o_{t-N:t}$ video segment) continuously monitors execution, comparing observed gripper trajectories against the planned trajectory. When a deviation is detected—dropped object, missed grasp, collision—the MLLM first attempts autonomous recovery (as in Figures 6, A8). If recovery fails (the revised plan also fails, or the MLLM determines the situation is unrecoverable), the system escalates to a human operator with the full reasoning context (what was attempted, what went wrong, what recovery was attempted). This is more informative than a bare "task failed" signal because the MLLM's reasoning trace provides a human-readable diagnosis. The paper's specific result that the MLLM was trained on RoboFail data (300 failure cases) provides some evidence that failure detection is not purely incidental, though the absence of quantitative detection rates means this application should be considered aspirational rather than validated by the current paper.

When to Prefer This Method

The paper does not explicitly position ThinkAct against specific named alternatives with decision rules, and I will not fabricate such a tradeoff matrix. However, the experimental results imply boundary conditions that a practitioner can use to decide whether ThinkAct's approach is appropriate for their setting:

ThinkAct's dual-system architecture with action-aligned RL training is likely preferable when:

  • The target domain involves long-horizon manipulation tasks with identifiable subgoals (the LIBERO-Long result of 70.9% vs. DiT-Policy 57.6% is the strongest evidence for this condition).
  • The deployment environment changes in visual characteristics (lighting, backgrounds, object appearances) relative to training—the SimplerEnv Variant Aggregation results show larger gaps between ThinkAct and DiT-Policy under distribution shift (16.9 points on Google-VA vs. 15.5 on Google-VM).
  • Only a small number of demonstrations per task are available for a new environment, but a pre-trained generalist action model (e.g., OXE-trained) exists—the few-shot adaptation results (Figures 5, A9) demonstrate this advantage.
  • The embodiment operates with a camera providing an unobstructed view of the manipulation workspace, and the tasks are predominantly tabletop or planar (the 2D trajectory representation is sufficient for these settings but insufficient for depth-critical or occluded manipulation).
  • Slightly increased inference latency (17% on LIBERO, Appendix B.7) is acceptable in exchange for improved success rates, particularly on long-horizon tasks where the performance gain (13.3 points on LIBERO-Long) dominates the cost.

ThinkAct's approach is likely less suitable when:

  • The task requires precise 3D spatial reasoning (stacking, peg insertion, reaching into occluded spaces)—the 8.7% success on Bridge Stack Blocks suggests fundamental difficulty with depth-critical tasks.
  • Real-time control latency is paramount and cannot tolerate any autoregressive generation overhead—the paper provides no evidence that the MLLM latency can be reduced to control-loop timescales without increasing $N$ (which degrades performance via stale reasoning).
  • The available compute budget for training is limited—the three-stage pipeline requires 16 A100 GPUs and substantial training iterations (20K + 6K + 120K + 75K across stages).
  • The target MLLM is not Qwen2.5-VL and the practitioner cannot afford to replicate the full hyperparameter tuning pipeline—the paper provides no evidence of cross-architecture transfer, and the specific GRPO hyperparameters, cold-start data mixture, and Q-Former design may not transfer without modification.