ArXiv: 2605.12481

🎯 Pitch

Giving computer agents access to both GUI clicks and API tools often backfires, causing them to either overuse tools and break workflows or ignore them entirely. ToolCUA solves this by training agents with synthesized interleaved trajectories and reinforcement learning that explicitly rewards smarter, shorter paths, achieving a 66% performance leap simply by teaching when to switch between action modes.

1. Executive Summary

This paper introduces ToolCUA, an end-to-end computer use agent trained through a staged paradigm to learn optimal GUI-Tool path selection—deciding when to continue with atomic GUI actions and when to switch to structured tool calls within a hybrid action space. The system scales interleaved GUI-Tool trajectories from existing pure-GUI corpora via tool synthesis, then applies Tool-Bootstrapped GUI RFT (warmup SFT followed by single-turn RL at critical GUI↔Tool switching boundaries) and Online Agentic RL with a Tool-Efficient Path Reward (combining a tool-appropriateness term that rewards tool use only on tool-beneficial tasks and a path-efficiency term that incentivizes shorter trajectories relative to the rollout group). On OSWorld-MCP, ToolCUA-8B achieves 46.85% accuracy—a ~66% relative improvement over the Qwen3-VL-8B-Instruct baseline and a +3.9% gain over pure GUI action settings—while reducing average completion steps from 19.34 to 14.93, establishing that hybrid GUI-Tool training yields more efficient and generalizable automation only when the agent is explicitly taught trajectory-level switching policies rather than merely exposed to both action spaces.

2. Context and Motivation

The Core Problem: Hybrid Action Spaces Confuse Computer Use Agents

The fundamental problem this paper tackles is that simply giving a computer use agent access to both GUI actions and tool calls does not make it use them intelligently—in fact, it often makes performance worse. A CUA exposed to a hybrid action space stands at what the authors call a "forked road" (Figure 2) at every step: should it continue with atomic GUI operations (clicking, typing, scrolling) or should it invoke a high-level structured tool call (an API that performs a complex multi-step operation in a single invocation)?

This decision is not trivial because GUI actions and tool calls have complementary but fundamentally different properties:

  • GUI actions (click, type, scroll, drag) are universally applicable—they can theoretically accomplish anything a human could do with a mouse and keyboard. But they are slow, brittle, and error-prone in long trajectories. Each GUI action is a single primitive operation, so complex tasks require dozens of sequential steps, each introducing a chance for visual grounding errors, timing failures, or cascading mistakes where an early error propagates through the entire remaining trajectory.

  • Tool calls (API-based operations like libreoffice_set_column_values or vscode_add_folder) can accomplish in one invocation what would require many GUI steps. They are precise, fast, and deterministic—when the tool interface is available and the agent invokes it correctly. But tools are constrained by coverage (not every operation has a corresponding tool), stability (tools can break or return unexpected errors), and context sensitivity (tools don't help when the agent needs to interact with a pop-up dialog or visually inspect screen state).

The authors formalize this as a trajectory-level policy learning problem (Section 2.1). Unlike step-level action selection—where the agent only needs to pick the best action for the current state—GUI-Tool orchestration requires reasoning about how each switching decision reshapes the entire subsequent trajectory. A premature switch to tools might skip necessary GUI grounding. An overcommitment to GUI actions might waste steps on operations that a single tool call could complete. This is fundamentally different from pure GUI decision-making because the agent must trade off between two qualitatively different modes of operation whose relative advantages depend on task context, tool availability, and current progress.

Why This Problem Matters

The paper motivates the importance of hybrid GUI-Tool orchestration along three dimensions: capability, efficiency, and generalizability.

Capability: Tool-augmented actions unlock tasks that are impractical with GUI alone. The motivating example in Figure 1(a) is concrete: modifying an entire column in LibreOffice can be done with a single API call (libreoffice_set_column_values), whereas a pure GUI solution requires a long sequence of clicks, selections, and typing that is fragile and time-consuming. This isn't just about convenience—it's about whether the task succeeds at all. Long GUI trajectories accumulate errors multiplicatively; if each step has a small probability of failure (incorrect grounding, misinterpreted screen state, timing issues), the probability of completing a 30-step task successfully can be extremely low. Tools collapse these long chains into single reliable operations, fundamentally changing what success rates are achievable.

Efficiency: Tools dramatically reduce execution steps. The paper's results bear this out quantitatively (Section 3.2): ToolCUA reduces average completion steps (ACS) from 19.34 to 14.93 compared to the baseline, and achieves the lowest ACS among all evaluated models. Even when tasks succeed using pure GUI approaches, they may require unnecessarily long trajectories that waste time and compute. For real-world deployment—where agents might automate thousands of repetitive tasks—reducing average steps by 23% represents substantial cost and latency savings.

Generalizability: Hybrid training produces better models even in GUI-only settings. A surprising finding (Section 3.3, Table 3) is that ToolCUA trained in the hybrid GUI-Tool action space achieves 42.9% accuracy even when evaluated in a pure GUI setting—compared to 42.05% for a model trained exclusively on GUI data. This suggests that learning to decide between GUI actions and tool calls develops transferable skills: the model becomes better at recognizing when a task is efficiently completable, even when tools aren't available. The hybrid action space serves as a richer training signal, forcing the model to learn about action utility and path efficiency rather than just action imitation.

Systemic significance: The gap between "access to tools" and "ability to use tools" is large and growing. As Table 1 demonstrates, this is not a hypothetical concern. Claude-4.5-Sonnet—a state-of-the-art proprietary model—drops from 61.9% to 48.4% accuracy when tools are made available, despite having the capability to use them. This gap exists because most large language models are trained primarily on text and GUI data, with limited exposure to the kind of interleaved GUI-Tool reasoning that real computer automation requires. As agencies increasingly deploy CUAs for production workflows, this gap between tool availability and tool utilization will become a critical bottleneck. The paper suggests that closing it requires not better prompting or better base models, but fundamentally different training paradigms that explicitly teach trajectory-level orchestration.

Where Existing Approaches Fall Short

The paper identifies two fundamental limitations in prior work: data scarcity and inadequate supervision.

Limitation 1: High-Quality Interleaved GUI-Tool Trajectories Are Scarce

Most current CUAs are undertrained on tool use because the training data simply doesn't exist. The reasons are practical and entrenched:

  • Real tool trajectories require environment instrumentation. Collecting trajectories where an agent actually calls tools in a live computer environment requires setting up sandboxes with working APIs, instrumenting those environments to record the results of tool calls alongside screenshots, and having human annotators or capable agents execute tasks while correctly switching between GUI and tools. This is expensive, slow, and hard to scale.

  • Usable tools are application-specific, incomplete, and unstable. Real desktop application APIs vary enormously in coverage and reliability. A tool that works for one version of LibreOffice may break in another. A tool that's available in one application may have no equivalent in another. Building and maintaining a library of verified, documented tools across multiple applications is itself a significant engineering challenge.

  • Existing GUI corpora are large but tool-free. The computer use community has invested heavily in collecting GUI-only trajectory datasets—OpenCUA, ScaleCUA, GUI-360, CUA-Suite—containing thousands of demonstrations of humans or agents performing tasks through atomic GUI actions. These datasets are valuable but don't help with tool training because they contain no tool invocations and no tool-Tool switching decisions.

The paper acknowledges prior attempts to address this limitation:

  • Code-based tool generation (e.g., UltraCUA, Step-GUI) uses LLMs to generate tool definitions from code repositories or API documentation. While this can produce tool definitions at scale, these tools are often not grounded in actual computer-use trajectories—they describe what an API could do in the abstract rather than how it would actually be used within a specific task context. This makes the resulting data unrealistic and potentially misleading, as the synthetic tools may not match the actual behavior of the desktop environment.

  • Manual tool construction (e.g., OSWorld-MCP) provides high-quality, verified tools but is inherently limited in scale—each tool must be manually specified, tested, and maintained, which is impractical for covering the diversity of tasks and applications that real CUAs encounter.

The synthesis pipeline proposed in this paper (Section 2.2) addresses data scarcity by taking a different approach: repurpose existing GUI-only trajectories into interleaved GUI-Tool trajectories by synthesizing tools that are grounded in observed GUI behavior. Instead of starting from code or API documentation, the pipeline watches what the GUI trajectory actually does and abstracts those observed procedures into tool signatures. This means every synthesized tool corresponds to something that demonstrably happens in a real computer-use task, making the resulting trajectories more realistic. And because it operates on existing GUI corpora, it can scale to thousands of trajectories without requiring new environment instrumentation.

Limitation 2: Existing Supervision Provides Limited Guidance for Trajectory-Level Orchestration

Even if an agent has basic tool-calling ability (e.g., from SFT on tool-augmented data), current training signals don't teach it when to use tools versus GUI actions. The paper identifies two forms of supervision that are individually insufficient:

Step-level imitation learning trains the agent to predict the correct action at each step given the current state. This is the standard SFT approach used by most CUA training pipelines (e.g., ScaleCUA, OpenCUA). However, imitation learning at the step level only captures local action plausibility—does this action look reasonable given the current screenshot? It provides no signal about whether choosing a tool call at this step leads to a more efficient or reliable overall trajectory than choosing a GUI action, because the training data simply shows one path (the one that happened to be recorded) without contrastive information about alternative paths. A model trained purely with step-level imitation may learn to use tools when they appear in the training data, but it won't develop a principled understanding of why a tool was chosen at that point rather than GUI actions.

Final task-completion rewards train the agent to maximize whether the task succeeds, typically through online RL in a sandbox environment. This is used by recent works like MobileRL and GUI-R1. Task-completion rewards address some of imitation learning's limitations by providing outcome-level feedback: the agent directly experiences whether its decisions led to success or failure. However, the paper argues that task-completion alone is an impoverished signal for learning GUI-Tool orchestration, for two reasons:

  1. Task completion doesn't decouple quality from path choice. A task can be completed successfully with a long, brittle GUI-only workaround, or with a short, efficient tool-call path. If the reward is binary (success/failure), both trajectories receive the same positive signal, so the agent has no incentive to prefer the more efficient path. Over time, it may learn that either path "works" without developing a preference for tool-appropriate strategies.

  2. It doesn't distinguish between appropriate and inappropriate tool use. An agent might successfully complete a task despite unnecessary tool invocations that don't contribute to progress (e.g., repeatedly calling an env_info tool to inspect state without acting on the information). Or it might complete a task using tools in a way that only works for this specific instance (e.g., hard-coding a file path that happens to exist) rather than developing a generalizable tool-use policy.

The paper's proposed Tool-Efficient Path Reward (Section 2.4) addresses these limitations by decomposing trajectory feedback into two explicit components:

  • RtoolR_{\text{tool}}: rewards tool use only when the task actually benefits from tools (and rewards tool abstention when tools aren't needed), based on task-level annotations of whether tools are beneficial.
  • RlengthR_{\text{length}}: rewards trajectories that are shorter than the average of their rollout group, encouraging the agent to discover tool-call opportunities that collapse redundant GUI operations.

Together, these signals provide the trajectory-level feedback that both step-level imitation and sparse task-completion rewards lack.

How This Paper Positions Itself

The paper situates itself at the intersection of three lines of work:

1. End-to-end CUA models (OpenCUA, ScaleCUA, UI-Tars, GUI-Owl, EvoCUA) have demonstrated that unified vision-language-action models can be trained to perform computer tasks through pure GUI interactions. These works establish the baseline capability—the ability to ground visual observations to actions—but operate entirely in the GUI action space. ToolCUA builds on this foundation by extending the action space to include tools and, more importantly, by introducing training mechanisms that teach the model to orchestrate between the two modes. The key insight is that this extension isn't trivial: you can't just add tool definitions to the system prompt and expect the model to use them wisely. Instead, it requires explicit trajectory-level optimization.

2. Tool-augmented LLMs (ToolLLM, Gorilla, ReTool) have shown that language models can learn to invoke APIs for coding, web search, and other tasks. However, these works typically operate in text-only or single-turn settings where the model calls a tool, receives a result, and then produces a final answer. Computer use presents a fundamentally different challenge because the agent must interleave tools with visual grounding over a long trajectory: it calls a tool, observes the resulting screenshot, decides whether to call another tool or switch to GUI actions, and so on. The visual state evolves continuously, and the model must integrate tool results with its visual understanding of the desktop.

3. RL for GUI agents (MobileRL, ARPO, GUI-R1, UI-R1) has shown that online reinforcement learning in sandbox environments can improve agent performance beyond what SFT alone achieves. These works demonstrate the promise of environment-driven optimization but operate in pure GUI action spaces. ToolCUA extends RL to the hybrid setting, where the action space is richer (the model can choose between qualitatively different types of actions) but also more challenging (the reward signal must distinguish between appropriate and inappropriate tool use). The paper's staged training paradigm—offline RFT to establish basic tool knowledge, then online RL with a shaped reward to optimize trajectory-level decisions—is designed specifically for this more complex setting.

The paper's core positioning claim is that training in a hybrid GUI-Tool action space is not just an extension of existing CUA work, but a paradigm shift. Pure GUI training teaches agents to be good at executing step-by-step procedures. Hybrid training teaches them to be good at deciding which procedure is worth executing. This meta-level skill—recognizing when a tool shortcut is available and whether it's actually beneficial given the current context—is what distinguishes ToolCUA from prior agents that either ignore tools or use them indiscriminately. The staged training paradigm (RFT → Online RL) is designed to teach this skill incrementally: first establish the basic capability of using tools at all (warmup SFT on synthesized interleaved data), then calibrate local switching decisions at critical boundaries (single-turn RL on GUI↔Tool transition points), and finally optimize global trajectory-level strategies through online interaction with shaped rewards.

The authors frame their contribution not as a single novel method but as a training methodology that addresses a newly identified failure mode: optimal path confusion under hybrid action spaces. Prior work implicitly assumed that if a model can use tools, it will use them appropriately. ToolCUA's experiments demonstrate that this assumption is false (Table 1)—models from 8B to 235B parameters, from open-source to proprietary, all suffer from this confusion to varying degrees. The paper's contribution is thus both diagnostic (identifying optimal path selection as the bottleneck) and prescriptive (providing a staged training recipe that systematically addresses it).

3. Technical Approach

3.1 Reader Orientation

This section explains the ToolCUA system—a complete training pipeline that produces an 8B-parameter vision-language model capable of acting as a computer use agent that intelligently switches between atomic GUI actions (clicks, typing, scrolling) and structured tool calls (API-based file operations, spreadsheet functions, code editor commands). The system solves the optimal GUI-Tool path selection problem: given a desktop task, the agent must dynamically decide at each step whether to continue with GUI operations or invoke a tool, such that the entire trajectory is both successful (task completed correctly) and efficient (minimum steps used). The "shape" of the solution is a staged training paradigm that first synthesizes interleaved GUI-Tool training data from existing GUI-only trajectories, then uses that data to bootstrap basic tool-calling capability through warmup SFT and single-turn RL at switching boundaries, and finally refines trajectory-level orchestration through online agentic RL with a carefully designed reward function that explicitly encourages appropriate tool use and path efficiency.

3.2 Big-Picture Architecture (Diagram in Words)

The ToolCUA system has five major components connected in a sequential pipeline:

  1. Interleaved GUI-Tool Trajectory Scaling Pipeline (offline data construction): Takes existing pure-GUI trajectories and converts them into interleaved GUI-Tool training data by (a) filtering and balancing source trajectories, (b) synthesizing a trajectory-aware library of tools from observed GUI procedures, (c) generating functionally equivalent tool-only trajectories with next-state grounding, and (d) constructing interleaved variants by randomly replacing tool calls with their corresponding GUI sequences. Output: two datasets—D_all containing full interleaved trajectories and D_critical containing only the GUI↔Tool switching boundary steps.

  2. Warmup Supervised Fine-Tuning (SFT): Trains the base model on D_all using standard cross-entropy loss over all action tokens. This establishes the model's basic multimodal tool-calling knowledge—understanding tool signatures, parameter formats, and how tool results appear in the environment. Output: model M_sft.

  3. Single-Turn RL on Critical Steps: Applies Group Relative Policy Optimization (GRPO) on D_critical to calibrate the model's decisions specifically at GUI↔Tool switching boundaries. At each critical step, the model samples multiple completions and receives direct feedback on whether it should continue with GUI actions or switch to a tool call given available tools. Output: model M_rft.

  4. Online Agentic RL in GUI-Tool Environment: Takes M_rft and runs multi-turn GRPO in a live computer-use sandbox with real tool APIs. The model generates complete trajectories, interacts with the environment, and receives a composite reward combining task accuracy, tool appropriateness, and path efficiency. The training infrastructure uses decoupled GPU clusters for policy optimization and distributed ECS servers for sandbox rollouts.

  5. Tool-Efficient Path Reward Function: Computed during online RL, this reward decomposes trajectory quality into three components—format correctness (R_fmt), task accuracy (R_acc), tool appropriateness (R_tool), and path efficiency (R_length)—with a weighted combination that shapes the agent toward tool-appropriate and efficient behavior.

Information flows as follows: source GUI trajectories → synthesis pipeline → D_all and D_critical → SFT warmup → M_sft → single-turn RL → M_rft → online RL with Tool-Efficient Path Reward → final ToolCUA model. The offline stages (synthesis, SFT, single-turn RL) establish foundational capabilities; the online stage refines them through real environment interaction.

3.3 Roadmap for the Deep Dive

  • First, the formal MDP definition (Section 2.1), which establishes the mathematical framework for the GUI-Tool orchestration problem—what states, actions, and rewards mean in this context, and why the hybrid action space changes the optimization landscape.
  • Second, the Interleaved GUI-Tool Trajectory Scaling Pipeline (Section 2.2), because it is the foundational data engine that enables all subsequent training. Understanding how tools are synthesized, how trajectories are converted, and what D_critical captures is essential before discussing how those datasets are used.
  • Third, Tool-Bootstrapped GUI RFT (Section 2.3), which covers the two-stage offline training that establishes basic tool-calling capability and calibrates switching decisions. This builds directly on the data described in step two.
  • Fourth, Online Agentic RL with Tool-Efficient Path Reward (Section 2.4), which covers the online training stage that refines trajectory-level orchestration. This includes the detailed reward function decomposition—why each term exists, what it computes, and how it shapes agent behavior.
  • Fifth, the training infrastructure and hyperparameter configurations distributed across the paper and Appendix C, which are essential for understanding the scale of the system and the practical choices that make it work.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily an empirical systems paper with a staged training methodology whose core idea is that the GUI-Tool orchestration problem cannot be solved by simply exposing agents to both action spaces—it requires (1) scalable data synthesis to create interleaved training trajectories from existing pure-GUI data, (2) targeted offline optimization at switching boundaries to calibrate local decisions, and (3) trajectory-level online RL with a shaped reward that explicitly incentivizes tool-appropriate and efficient path selection.

Formal Problem Definition as a Markov Decision Process

The paper formalizes the computer-use task as a Markov Decision Process (MDP, Section 2.1) to provide a precise language for describing states, actions, transitions, and rewards in the hybrid GUI-Tool setting. While MDPs are standard in reinforcement learning, the specific instantiation here matters because the hybrid action space creates decision points that don't exist in pure-GUI settings.

The MDP is defined as $\mathcal{M} = \langle \mathcal{S}, \mathcal{A}, \mathcal{P}, \mathcal{R}, \gamma \rangle$ where:

  • $\mathcal{S}$ is the state space, where each state $s_t \in \mathcal{S}$ is a multimodal observation that includes both the desktop screenshot at time $t$ and the results of previously invoked tool calls. This dual nature of state—visual observation plus structured tool feedback—is critical because the agent must integrate two qualitatively different information streams: pixel-level visual grounding (where are the buttons, what text is visible, what dialog is open) and structured semantic feedback (the tool returned a JSON object indicating success/failure with specific result data).

  • $\mathcal{A} = \mathcal{A}_{\text{GUI}} \cup \mathcal{A}_{\text{Tool}}$ is the hybrid action space. $\mathcal{A}_{\text{GUI}}$ contains atomic GUI interactions such as coordinate-based clicks (left_click, right_click, double_click), typing (type), key combinations (key), mouse movement (mouse_move), drag operations (left_click_drag), scrolling (scroll, hscroll), waiting (wait), task termination (terminate), and question answering (answer). $\mathcal{A}_{\text{Tool}}$ contains high-level structured tool invocations with tool-specific parameters, such as libreoffice_create_pivot_table(source_sheet, table_name, row_fields, value_fields, aggregation_function, target_cell) or osworld_mcp_code.add_folder(folder).

  • $\mathcal{P}$ is the state transition function, which depends on the action type: GUI actions modify the desktop state (a click opens a menu, a type enters text), producing a new screenshot; tool calls invoke APIs that may modify files, application state, or system configuration, with the result returning both as a textual JSON response and (optionally) as a visible change in the subsequent screenshot.

  • $\mathcal{R}$ is the reward function. During online RL, this is the Tool-Efficient Path Reward (Equation 2) that combines multiple components. During offline training, the reward structure depends on the stage (GRPO-based step-level signals for single-turn RL, cross-entropy loss for SFT).

  • $\gamma$ is the discount factor (standard in RL, but the paper focuses on episodic tasks with finite horizons of up to 30–50 steps, so the specific value is less critical than the reward shaping).

The objective is to learn a policy $\pi_\theta(a_t | s_t)$—a mapping from the current multimodal state to a probability distribution over both GUI actions and tool calls—that maximizes expected cumulative reward over a trajectory:

J(θ)=Eπθ[∑t=0TR(st,at)]J(\theta) = \mathbb{E}_{\pi_\theta}\left[\sum_{t=0}^{T} \mathcal{R}(s_t, a_t)\right]

where $\theta$ are the model parameters, $T$ is the trajectory length, and the expectation is over trajectories sampled by following the policy.

What this formalization captures that a simpler formulation would miss: The hybrid action space means the policy must solve two qualitatively different sub-problems at each step: (1) action type selection—should I use GUI or tool?—and (2) action parameterization—if GUI, which coordinates and operation? If tool, which tool and what parameters? These sub-problems interact: a tool call might be correct in type but fail if parameterized incorrectly, while a GUI action might be correctly parameterized but type-incorrect (leading to an unnecessarily long trajectory). The MDP framing makes explicit that both decisions affect the trajectory's future states and ultimate success, so optimization must consider them jointly rather than as independent modules.

Why the state definition matters: Including previous tool results in $s_t$ means the agent has memory of what tools have been called and what they returned. This is essential because tool calls can produce side effects (a file was saved, a column was formatted) that may not be immediately visible in the screenshot but change what subsequent actions are appropriate. For example, after calling libreoffice_set_column_values, the screenshot might look unchanged until the user scrolls to the affected column, but the agent should know from the tool's return value that the operation succeeded and plan accordingly rather than re-executing it.

Interleaved GUI-Tool Trajectory Scaling Pipeline

The data synthesis pipeline (Section 2.2, Figure 3a) is the foundation of ToolCUA's training because it solves the core data scarcity problem: there exist large corpora of pure-GUI trajectories (thousands of recorded human or agent demonstrations performing computer tasks through atomic GUI actions), but virtually no interleaved GUI-Tool trajectories. The pipeline's key insight is that tools can be synthesized by analyzing what GUI trajectories actually do, then used to convert those trajectories into hybrid-action training data. This grounds every synthesized tool in observable behavior, avoiding the realism problems of tools generated purely from code or documentation.

The pipeline operates in four sequential stages, each building on the output of the previous one.

Stage 1: Trajectory Filtering and Balancing

The pipeline begins with raw GUI trajectories aggregated from multiple sources: OpenCUA (8,500 trajectories, 110k steps), ScaleCUA (300 trajectories, 20k steps), and internally collected sandbox rollouts using powerful MLLMs (1,200 trajectories, 62k steps after filtering), totaling approximately 10,000 trajectories and 192,000 raw GUI steps (Appendix C.2, Table 5).

These trajectories are filtered along three axes to ensure quality and diversity of the source distribution for tool synthesis:

  • Execution quality: Only successful trajectories are retained—those where the task was completed correctly according to the environment's evaluator. Failed trajectories would produce synthetic tools that describe incorrect or incomplete procedures, which would be harmful training data regardless of how well the tools capture the observed behavior.

  • Trajectory length: Extremely short trajectories (1-2 steps) are removed because they provide insufficient context for synthesizing meaningful tools—a tool that abstracts a single click is just a renamed GUI action, not a useful abstraction. Extremely long trajectories may also be filtered (though the paper doesn't specify a maximum threshold) because they often contain redundant or exploratory actions that don't reflect efficient task completion.

  • Application domain coverage: The remaining trajectories are balanced across application domains (LibreOffice, Chrome, VSCode, Thunderbird, GIMP, system operations, file management, etc.) to ensure the synthesized tool library covers diverse software ecosystems rather than specializing in a single application.

The balance operation matters because tool synthesis quality depends on the diversity of observed GUI procedures. If 90% of source trajectories were from Chrome, the resulting tool library would be Chrome-heavy, and models trained on it would be poorly equipped to handle LibreOffice or VSCode tasks where different tool abstractions are appropriate.

Stage 2: Trajectory-Aware Synthetic Tool Library Construction

This is the core synthesis step. For each filtered GUI trajectory, the pipeline uses an MLLM (the paper mentions Kimi-K2.5 or Claude-4.5-Sonnet as synthesis models in Section 2.2, and notes in Appendix A that replacing stronger proprietary models with Qwen3.5-Plus led to "noticeably lower generation efficiency and trajectory quality") to analyze the task and synthesize a candidate library of tools.

The MLLM receives as input:

  • The task goal (e.g., "Set the red sections of the pie chart to gold, the purple sections to blue and the orange to shadow lines")
  • The complete sequence of GUI actions from the original trajectory
  • Dense screenshot descriptions generated by a separate SCREENSHOT DESCRIPTION PROMPT (Appendix E), which produces 2-4 sentence English descriptions of each screenshot including visible applications, UI elements, and primary content

From this input, the MLLM produces tool definitions using the TOOL GENERATION PROMPT (Appendix E), which specifies detailed constraints on tool design. Each synthesized tool is specified by:

  • A functional signature with parameter names, types, and descriptions inferred from the trajectory context
  • A natural language description of what the tool accomplishes
  • A returns schema that always includes success (boolean), result (object/string/null), and error_message (string/null)
  • A category from the set {navigation, interaction, extraction, filesystem, system, terminate}
  • A granularity label of either fine or coarse

The key design property that makes these tools "trajectory-aware" rather than generic is the grounding rule in the TOOL GENERATION PROMPT: "Prioritize tools that explain real state transitions visible in the recorded trajectory frames. The main tool set should be sufficient for the observed workflow in this trajectory." This means every synthesized tool corresponds to something that demonstrably happens in the recorded GUI trajectory—the MLLM is not imagining what APIs could exist for an application, but rather abstracting what it observes the GUI trajectory doing into callable operations.

To increase diversity along the tool granularity dimension, the pipeline synthesizes tools at multiple levels of specificity (Section 2.2, Figure 7):

  • Fine-grained tools capture a focused semantic intent corresponding to one local sub-goal. Example: chrome_open_settings (opens the Chrome settings page), which abstracts a few clicks in the browser menu.
  • Mid-grained tools cover several adjacent UI operations that together form a coherent subroutine. Example: chrome_open_language_settings (navigates through settings to the language configuration section), which abstracts a longer sequence of menu navigation.
  • Coarse-grained tools capture a broader intent spanning multiple sub-goals. Example: libreoffice_create_pivot_table (creates a complete pivot table from source data to final placement), which might abstract dozens of GUI steps involving range selection, menu navigation, dialog configuration, and confirmation.

Appendix C.2 reports that the synthesized tool inventory contains 4,350 unique tools across all trajectories, comprising approximately 2,000 fine-grained, 1,900 mid-grained, and 450 coarse-grained tools. On average, each trajectory has access to 19.75 candidate tools and executes 7.89 of them (these are the tools that would be used in the tool-only version of the trajectory, not necessarily all available tools).

The diversity of granularity is important for two reasons: (1) it forces the model to learn tool selection at different levels of abstraction—sometimes the right choice is a coarse tool that accomplishes a lot, sometimes a fine tool that provides precise control; (2) it creates realistic "tool library" contexts where multiple tools could plausibly accomplish the same subgoal, requiring the agent to discriminate between them based on context.

Naming convention: The TOOL GENERATION PROMPT enforces a specific naming rule: if a tool is tied to a specific application, its name must start with that application name using lowercase_with_underscores (e.g., libreoffice_export_pdf, vscode_open_user_settings, chrome_download_file). If no specific application can be determined, the name must start with general_. This convention ensures that tool names are informative about their domain, which helps the model learn to associate tools with applications.

Validation: After synthesis, a rule-based format verification (FIX TOOL PROMPT in Appendix E) checks that each tool definition conforms to the required schema—correct JSON structure, valid parameter types, presence of required fields (success, result, error_message in the returns schema), and adherence to the granularity and category constraints. Invalid tools are repaired or discarded.

Stage 3: Tool Trajectory Generation with Next-State Grounding

Given the synthesized tool library for a trajectory and the original GUI trajectory, this stage produces a functionally equivalent tool-only trajectory—a sequence of tool calls that accomplishes the same task, grounded to the same state transitions visible in the original GUI screenshots.

The generation uses the JOINT GENERATION PROMPT (Appendix E), which instructs an MLLM to simulate a smart agent completing the task using only tool calls. For each step, the MLLM:

  1. Receives the task goal, trajectory history (previous tool calls and their results), a description of the current screenshot, the current "world state" (a structured representation of what's known about the environment), and the list of available tools.
  2. Produces an observation (grounded in the screenshot description), a thought (reasoning about the next tool call), an action (natural language summary), a tool_call (the actual function invocation with parameters), and a predicted tool_response (the expected return value).

The prompt includes several important constraints designed to keep the generated trajectories realistic:

  • "Do not generate multiple steps at once" (forces step-by-step reasoning)
  • "Do not call wait-like tools more than once in a row" (prevents degenerate stalling behavior)
  • "Every step must make progress toward the task goal"
  • "Choose only a tool action that can plausibly move the current state to a later real state in the recorded trajectory" (the grounding constraint)
  • "Do not invent a tool usage whose effect would not be visibly groundable to a later recorded screenshot"

Next-state grounding is the critical mechanism that ensures each synthetic tool step corresponds to an actual state in the original trajectory. After generating a tool trajectory step, the pipeline uses the PREDICT SCREENSHOT PROMPT (Appendix E) to predict the UI state after the tool call, then the DESCRIBE AND LOCATE PROMPT to match this predicted state against candidate screenshots from the original trajectory. This produces a mapping: tool step $i$ produces a state that matches screenshot $j$ from the original GUI trajectory, where $j > i$ (the tool step advances the task state to a later point).

The grounding serves two purposes: (1) it verifies that the synthesized tool call is plausible—if the predicted post-tool state doesn't match any actual screenshot, the tool call likely doesn't correspond to real application behavior; (2) it provides the alignment needed for the next stage, where tool steps are replaced with their corresponding GUI sequences.

Bottom-up merging is applied at this stage to create multi-granularity variants. The MERGE TREE PLANNING PROMPT and BOTTOM UP MERGE PROMPT (Appendix E) identify contiguous chunks of fine-grained tool steps that serve a shared sub-goal and merge them into higher-level composite tool calls. For example, three fine-grained steps—libreoffice_select_data_range, libreoffice_open_insert_menu, libreoffice_click_pivot_table—might be merged into the coarse-grained libreoffice_create_pivot_table(...). The merge preserves the original step order and is constrained to merge only adjacent steps that together form a coherent higher-level operation. By applying this merging at multiple levels (up to a specified max_coarse_levels), the pipeline produces trajectories at different granularities from the same source, increasing the diversity of the training data.

Stage 4: Interleaved GUI-Tool Trajectory Generation

This final stage converts a grounded tool-only trajectory into multiple interleaved GUI-Tool variants by selectively replacing tool calls with their corresponding GUI action sequences.

The process works as follows (Section 2.2):

  1. Start with a complete tool-only trajectory that has been grounded to original GUI screenshots.
  2. Randomly sample a subset of the tool calls to replace with GUI actions. The selection is stochastic, producing different variants from the same base trajectory on different runs.
  3. For each selected tool call, replace it with the GUI action sequence from the original trajectory that corresponds to the same state transition. The grounding from Stage 3 provides the mapping: if tool step $k$ was grounded to screenshots $j_1$ through $j_m$, the replacement is the GUI actions that occur between those screenshots in the original GUI trajectory.
  4. Remove the replaced tools from the available tool library for that trajectory variant. This constructs a partial tool-availability context where the agent has access to some tools but not others—in particular, it lacks the tools that were replaced. This means the agent must fall back to GUI operations for those specific sub-tasks while still having tool access for others.

This partial-availability design is important because it creates realistic deployment scenarios. In real computer use, tool availability is inconsistent—some applications have well-maintained MCP servers with many tools, others have few or none, and some tools may fail or be unavailable in certain contexts. By training on trajectories with varying tool availability, the model learns to be robust to these inconsistencies rather than assuming all tools are always available.

Critical switching steps: Each replacement naturally creates two types of boundary transitions:

  • GUI → Tool: The agent completes a sequence of GUI actions and then switches to a tool call.
  • Tool → GUI: The agent receives a tool result and then switches to GUI actions.

These transition points are extracted and collected into a dedicated dataset D_critical (approximately 5,000 steps, Appendix C.2, Table 5). They are called "critical" because these are precisely the decision points where the agent must determine whether the current state calls for continued GUI operations or a switch to tools—the "forked road" from Figure 2. The single-turn RL stage (Section 2.3) specifically targets these points to calibrate the model's switching decisions.

Scale of the synthesized data: From approximately 10,000 source GUI trajectories with 192,000 raw steps, the pipeline produces 10,000 interleaved GUI-Tool trajectories comprising approximately 180,000 high-quality steps for the warmup SFT stage, plus 5,000 critical switching steps for single-turn RL (Appendix C.2, Table 5). The average tool pool size per trajectory is 19.75, and the average number of executed tools per trajectory is 7.89 (Table 6).

Why this pipeline design over alternatives: The paper's approach differs from prior tool synthesis methods in three key ways:

  1. Grounded in observed behavior, not code or documentation: Tools synthesized from API documentation or code repositories (e.g., UltraCUA) describe what an API supports in principle, but may generate tools that don't correspond to actual workflows or that behave differently in practice than in documentation. By extracting tools from observed GUI trajectories, the pipeline ensures every tool describes something that demonstrably happens in a real desktop environment—the grounding step explicitly verifies this by matching predicted tool effects to actual screenshots.

  2. Scalable through repurposing, not new collection: Collecting real GUI-Tool trajectories requires instrumented environments with working tool APIs, which is expensive and slow. The pipeline operates on existing pure-GUI corpora, which are already large and diverse, and converts them without requiring new data collection. This makes the approach scalable to new applications and tasks as GUI trajectory datasets grow.

  3. Systematic coverage of switching contexts: By randomly sampling which tools to replace with GUI sequences, the pipeline generates interleaved trajectories with diverse switching patterns—sometimes tools are used early and GUI later, sometimes the reverse, sometimes tools are interspersed throughout. This variety ensures the model sees many different GUI↔Tool transition contexts during training rather than only the patterns that a human demonstrator or scripted agent happened to produce.

The primary limitation of this approach, acknowledged in Appendix A, is that "the diversity and quality of the synthesized hybrid trajectories are coupled with the breadth, fidelity, and task distribution of the source demonstrations." If the source GUI trajectories don't cover certain applications or task types, the resulting tool library won't either. Additionally, the synthesis quality depends on the capability of the MLLM used for generation—replacing Claude-4.5-Sonnet with Qwen3.5-Plus led to noticeably lower quality in the authors' internal trials.

Tool-Bootstrapped GUI RFT

With the synthetic interleaved data D_all and critical switching steps D_critical, Tool-Bootstrapped GUI RFT (Section 2.3) trains the base model through two sequential offline stages: warmup SFT to establish basic hybrid-action capabilities, and single-turn RL to calibrate decisions at switching boundaries. The "tool-bootstrapped" label reflects that the model's tool knowledge is bootstrapped entirely from synthesized data rather than from real tool interaction—the online RL phase later grounds this bootstrapped knowledge in actual environment feedback.

Warmup Supervised Fine-Tuning (SFT)

The warmup SFT stage trains the base model M_base (Qwen3-VL-8B-Instruct) on the full interleaved GUI-Tool dataset D_all using standard next-token prediction:

LSFT=−∑log⁡πθ(at∣st)\mathcal{L}_{\text{SFT}} = -\sum \log \pi_\theta(a_t | s_t)

where the sum is over all action tokens in the trajectory, $\pi_\theta(a_t | s_t)$ is the model's predicted probability of the correct action token $a_t$ given the current state $s_t$, and $\theta$ are all model parameters (both vision tower and LLM backbone).

What it computes: the standard cross-entropy loss for autoregressive language modeling applied to action prediction. For each step in each training trajectory, the model receives the multimodal state (screenshot + text context including task goal, action history, and tool results) and must predict the next action tokens—which encode either a GUI operation with coordinates/parameters or a tool call with function name and arguments. The loss is high when the model assigns low probability to the correct action tokens and low when it assigns high probability.

Why this form: Cross-entropy is the maximum-likelihood objective for autoregressive token prediction, which is the standard fine-tuning objective for transformer-based language models. The key property in this context is that it trains the model to imitate the entire action distribution present in the training data—it learns not just which actions are correct, but also the syntactic format of tool calls (JSON structure, parameter naming, tool name conventions), the contextual cues that precede different action types, and the relationship between visual state and action choice.

Training configuration (Appendix C.3): The SFT runs for 3 epochs using full-parameter fine-tuning of both the vision tower and the LLM backbone. The training hardware is a cluster of 8 × 8 GPUs. This is a substantial compute investment for an 8B-parameter model, reflecting that both the vision encoder and language model need to adapt to the multimodal tool-calling domain—the vision tower must learn to recognize application-specific UI elements relevant to tool decisions (e.g., distinguishing between a LibreOffice spreadsheet view and a VSCode editor view), while the LLM backbone must learn the syntax and semantics of tool calls alongside GUI actions.

What the model learns during SFT: The warmup SFT establishes several foundational capabilities that are prerequisites for the later RL stages:

  • Tool signature knowledge: The model learns what tools exist, what parameters they take, and what names they have (e.g., osworld_mcp_libreoffice_calc.create_pivot_table takes source_sheet, table_name, row_fields, etc.). Without this, the model couldn't even formulate valid tool calls.

  • Tool parameter grounding: The model learns to map visual observations to tool parameters. For example, seeing a spreadsheet with columns labeled "Product" and "Revenue" should suggest using "F" (the column index for "Product") as a row_fields parameter for a pivot table tool. This requires integrating visual understanding with structured parameter specification.

  • Tool result interpretation: By seeing tool responses (<tool_response> blocks containing JSON with success, result, and error_message fields) paired with subsequent screenshots, the model learns what tool results mean and how to continue after receiving them. A success: true with a specific result might mean the task is partially complete, while success: false with an error message might require a different approach.

  • Interleaving patterns: By observing trajectories that switch between GUI actions and tool calls, the model learns the syntactic and contextual patterns of switching—the <tool_call> XML block format, the tool response format, how the screenshot changes after a tool call versus a GUI action.

  • Partial tool availability handling: Because the interleaved trajectories include variants where some tools are unavailable (replaced by GUI sequences), the model learns that tools are not always present and that GUI fallback is sometimes necessary.

After SFT, the resulting model M_sft has basic tool-calling competence but has not been explicitly trained to make optimal switching decisions. It imitates the switching patterns in the training data, which were constructed by random replacement rather than by optimal decision-making. The subsequent RL stages address this limitation.

Single-Turn RL on Critical Steps

Building on M_sft, the single-turn RL stage applies Group Relative Policy Optimization (GRPO) specifically to the critical switching steps in D_critical. GRPO is a variant of policy gradient RL that operates on groups of sampled completions for the same input, computing advantages relative to the group mean rather than requiring a separate value function. This is well-suited to the single-turn setting because the model needs to learn a local decision—switch to tool or continue with GUI?—without the complications of multi-turn credit assignment.

Why critical steps are targeted: The 5,000 steps in D_critical are exactly the GUI↔Tool boundaries extracted during the interleaved trajectory generation (the yellow stars in Figure 3a). At a GUI→Tool boundary, the model must decide: given the current GUI state and the available tools, should I invoke a tool now, or continue with another GUI action? At a Tool→GUI boundary, the model must decide: given the tool result and the new screenshot, should I invoke another tool, or switch back to GUI?

These decisions are what the paper identifies as the "optimal path selection" bottleneck (Section 1, Figure 2). The baseline model M_sft learned to imitate these decisions from randomly-constructed training data, but the imitation signal is weak because the training data doesn't encode why a switch happened at a particular point—it just shows that it happened. Single-turn RL provides a stronger signal: at each critical step, the model samples multiple completions (some choosing to switch, some not), receives feedback on which choice is appropriate (derived from the ground-truth trajectory), and updates its policy to favor the correct choice.

GRPO mechanism: For each critical step with state $s_t$, the model samples a group of completions (group size = 32, Appendix C.3). Each completion is an action—either a GUI action or a tool call. The completions are scored, and the model is updated to increase the probability of above-average completions and decrease the probability of below-average completions, with the advantage computed as:

Ai=ri−mean({r1,…,rG})std({r1,…,rG})A_i = \frac{r_i - \text{mean}(\{r_1, \ldots, r_G\})}{\text{std}(\{r_1, \ldots, r_G\})}

where $r_i$ is the reward for the $i$-th completion in the group of size $G$, and the mean and standard deviation are computed over the group. The reward $r_i$ reflects whether the chosen action matches the ground-truth action from the trajectory (i.e., whether the model correctly decides to switch or not switch).

What this form does: Group-relative advantage normalizes rewards within each group, which has two important properties: (1) it makes the learning signal invariant to the absolute scale of rewards—what matters is whether a completion is better than alternatives for the same state, not whether its absolute reward is high or low; (2) it naturally handles the exploration-exploitation tradeoff—completions that are worse than average are suppressed regardless of their absolute quality, so the model is pushed to distinguish between good and better choices rather than just between good and bad ones.

Training configuration (Appendix C.3): The single-turn RL uses a group size of 32 (32 sampled completions per critical step), a learning rate of 1 × 10^{-6}, and a training batch size of 128. The learning rate is an order of magnitude lower than typical SFT learning rates, which is standard for RL fine-tuning to avoid catastrophic forgetting of the capabilities learned during SFT. The smaller batch size (128 compared to the larger SFT batch) reflects that each training example requires 32 forward passes (for the group of completions), making the effective computation per example much higher.

After single-turn RL, the resulting model M_rft has calibrated local switching decisions—it is better than M_sft at recognizing when a GUI→Tool or Tool→GUI switch is appropriate. However, it has still only been trained on individual steps, not on complete trajectories. It doesn't know how switching decisions interact over multiple steps to produce globally efficient trajectories. The online RL phase addresses this gap.

Online Agentic RL with Tool-Efficient Path Reward

The online RL stage (Section 2.4) takes M_rft and further optimizes it through multi-turn interaction with a real computer-use sandbox environment. This is where the model transitions from knowing how to use tools (from SFT) and when to switch locally (from single-turn RL) to learning why certain GUI-Tool orchestration strategies lead to better overall trajectories.

Why online RL is necessary despite offline training: The offline stages provide strong initialization, but they have fundamental limitations that only online interaction can overcome:

  • Distribution shift: The synthetic trajectories in D_all and D_critical were generated by powerful MLLMs (Claude-4.5-Sonnet, Kimi-K2.5) with different capabilities and biases than the 8B Qwen model being trained. The model might learn patterns from this data that don't transfer well to its own action distribution. Online RL lets the model learn from its own trajectories—the states it visits, the actions it chooses, and the outcomes it experiences.

  • Credit assignment over long horizons: Single-turn RL at critical steps optimizes local decisions, but the quality of a switching decision depends on what happens many steps later. A tool call that seems appropriate at step 5 might lead to a dead end at step 12 that GUI actions would have avoided. Online RL with trajectory-level rewards provides the long-horizon credit assignment that single-turn RL cannot.

  • Exploration of alternative paths: The synthetic trajectories show one path to task completion (the one generated by the synthesis MLLM). Online RL allows the agent to discover alternative—potentially more efficient—paths through exploration, guided by the path efficiency reward.

Environment and Infrastructure

The online RL is conducted in a sandbox built on OSWorld QEMU virtual machine images, extended with MCP tool implementations from OSWorld-MCP and AutoGLM (Appendix C.3). The environment supports over 150 tools across multiple desktop applications (LibreOffice suite, Chrome, VSCode, Thunderbird, GIMP, terminal, file manager, system settings). The training tasks are drawn from OSWorld, excluding the multi_apps domain (which is held out for out-of-distribution evaluation), and augmented with scaled tasks from RLAnything and paraphrased goal instructions.

The training infrastructure uses a decoupled design (Appendix C.3) built on the verl framework:

  • Policy optimization runs on a GPU cluster (8 × 8 GPUs for training, 4 × 8 GPUs for dedicated inference serving)
  • Environment rollouts execute on approximately 250 independent Docker instances running on distributed ECS (Elastic Compute Service) servers
  • Rollout results (trajectories with rewards) are aggregated and fed back to the policy optimizer

This decoupling is important because the GPU requirements for LLM inference during rollouts (generating actions autoregressively) and the CPU/memory requirements for running QEMU-based desktop sandboxes are fundamentally different workloads. Running them on the same hardware would create resource contention and limit scaling.

Training configuration (Appendix C.3): The online RL uses a rollout size of 32 per group (32 complete trajectories sampled for each training prompt), a learning rate of 1 × 10^{-6}, a training batch size of 32, and a maximum execution horizon S_max = 30 steps per trajectory. Training runs for approximately 25 optimization steps, with dynamic filtering (inspired by DAPO) that retains only rollout groups containing both successful and failed trajectories.

Tool-Efficient Path Reward

The core innovation of the online RL stage is the Tool-Efficient Path Reward, which decomposes the trajectory-level reward into four components:

R=Rfmt+Racc+λ⋅Rtool+β⋅RlengthR = R_{\text{fmt}} + R_{\text{acc}} + \lambda \cdot R_{\text{tool}} + \beta \cdot R_{\text{length}}

where $R_{\text{fmt}}$ is a format reward, $R_{\text{acc}}$ is an accuracy (task success) reward, $R_{\text{tool}}$ is a tool appropriateness reward, and $R_{\text{length}}$ is a path efficiency reward. The hyperparameters are set to $\lambda = 0.4$ and $\beta = 0.2$ (Appendix C.3).

Why these weights: $\lambda > \beta$ means tool appropriateness is weighted twice as heavily as path efficiency. This prioritization reflects that using tools when they're genuinely needed (or avoiding them when they're not) is more important than shaving a few steps off an already-efficient trajectory. A trajectory that uses tools appropriately but takes 20 steps is better than one that takes 15 steps but uses tools inappropriately (e.g., calling tools that don't contribute to progress). The absolute values of $\lambda$ and $\beta$ are small enough that $R_{\text{acc}}$ (task success) remains the dominant term—the shaping rewards provide guidance on how to succeed, but success itself is the primary objective.

Format reward $R_{\text{fmt}}$: This is a standard reward component in LLM RL that penalizes malformed outputs—actions that don't parse as valid JSON tool calls, actions with missing required parameters, or actions that violate the output format specification. The paper doesn't specify the exact value or computation of $R_{\text{fmt}}$ (it's a standard implementation detail), but it serves the critical function of keeping the model's outputs syntactically valid during exploration, preventing degenerate behavior where the model generates unparseable actions that would break the environment interaction loop.

Accuracy reward $R_{\text{acc}}$: This is the task completion reward, provided by the OSWorld environment evaluator. It is a binary or sparse reward that indicates whether the final state of the environment matches the task specification (e.g., whether the pivot table was actually created with the correct data). The paper notes in Appendix C.3 that the training tasks use the task-level tool-beneficial annotations from OSWorld-MCP, with secondary manual verification to ensure label quality. $R_{\text{acc}}$ is the base signal that drives task completion; the other reward components shape how the task is completed.

Tool Appropriateness Reward Term $R_{\text{tool}}$: This component explicitly shapes the agent's tool usage decisions by rewarding tool calls only when they are genuinely beneficial for the task:

Rtool=Isucc⋅I[(tb>0∧c>0)∨(tb<0∧c=0)]R_{\text{tool}} = \mathbb{I}_{\text{succ}} \cdot \mathbb{I}\left[(t_b > 0 \land c > 0) \lor (t_b < 0 \land c = 0)\right]

where $\mathbb{I}_{\text{succ}}$ is an indicator that is 1 only if the trajectory succeeded (the task was completed correctly), $t_b \in \{1, -1\}$ is a task-level tool-beneficial label assigned during data construction (1 = tools are beneficial for this task, -1 = tools are unnecessary), and $c$ is the cumulative number of tool calls in the trajectory. The indicator inside the brackets is 1 when either: (a) the task benefits from tools AND the agent used at least one tool call, OR (b) the task does not benefit from tools AND the agent used zero tool calls. Otherwise it is 0.

What this computes: $R_{\text{tool}}$ is a binary reward (0 or 1) applied only when the trajectory succeeds ($\mathbb{I}_{\text{succ}} = 1$). It rewards tool use on tool-beneficial tasks and tool abstention on non-tool-beneficial tasks. It does not penalize tool non-use on tool-beneficial tasks or tool use on non-tool-beneficial tasks provided the trajectory still succeeds—it provides a bonus for getting the tool decision "right" according to the task labels.

Why this form and not a simpler alternative: A naive alternative would be to always reward tool calls (treating them as inherently good) or never reward them (treating them as neutral). The first alternative would produce models that overuse tools (the failure mode of Qwen3VL-235B in Table 1), calling tools even when they don't help or actively harm performance. The second would produce models that underuse tools (the failure mode of Qwen3VL-8B), never learning to take advantage of tool shortcuts. The task-conditional design—rewarding tool use only when the task annotation says tools are beneficial—provides a signal that is aligned with actual task structure. However, this design has a limitation the paper doesn't fully address: it assumes the task-level annotation $t_b$ is correct and comprehensive. If a task is labeled as tool-beneficial but the specific tool call the agent makes is inappropriate (wrong tool, wrong parameters), the agent still gets the reward as long as the task succeeds and at least one tool was called. This could theoretically incentivize spurious tool calls—calling any tool just to satisfy the $c > 0$ condition—but the path efficiency reward ($R_{\text{length}}$) likely mitigates this by penalizing unnecessary tool calls that don't reduce trajectory length.

The $\mathbb{I}_{\text{succ}}$ gating: $R_{\text{tool}}$ is only applied when the trajectory succeeds. This means the model never receives tool-appropriateness feedback for failed trajectories—it doesn't learn that calling a tool was a bad idea if the trajectory failed, because $R_{\text{tool}} = 0$ regardless of whether it called tools or not. This design choice prioritizes the signal-to-noise ratio of the reward: for failed trajectories, it's often unclear whether the tool decision was wrong or whether some other aspect of the trajectory caused the failure, so the reward stays silent rather than potentially providing misleading feedback.

Path Efficiency Reward Term $R_{\text{length}}$: This component encourages the agent to find shorter execution paths, particularly by using tool calls to collapse redundant GUI operations:

Rlength=Isucc⋅{(1+sˉ−ssˉ)if s<sˉexp⁡(−s−sˉSmax⁡−sˉ)if s≥sˉR_{\text{length}} = \mathbb{I}_{\text{succ}} \cdot \begin{cases} \left(1 + \frac{\bar{s} - s}{\bar{s}}\right) & \text{if } s < \bar{s} \\ \exp\left(-\frac{s - \bar{s}}{S_{\max} - \bar{s}}\right) & \text{if } s \geq \bar{s} \end{cases}

where $\mathbb{I}_{\text{succ}}$ is the success indicator (same gating as $R_{\text{tool}}$), $s$ is the current trajectory's step count, $\bar{s}$ is the group average step length across all rollouts for the same task, and $S_{\max} = 30$ is the maximum execution horizon.

What it computes: For trajectories shorter than the group average ($s < \bar{s}$), the reward is $1 + (\bar{s} - s)/\bar{s}$, which ranges from slightly above 1 (when $s$ is just below $\bar{s}$) to at most 2 (when $s = 0$, which never occurs in practice). This provides a linear bonus proportional to the relative step reduction—a trajectory that is 25% shorter than average gets a bonus of 0.25, one that is 50% shorter gets a bonus of 0.5. For trajectories at or above the group average ($s \geq \bar{s}$), the reward is $\exp(-(s - \bar{s})/(S_{\max} - \bar{s}))$, which decays exponentially from 1 (when $s = \bar{s}$) toward 0 as the trajectory becomes increasingly longer than average. The exponential decay is steeper for trajectories that are much longer than average, reflecting that excessively long trajectories are strongly dispreferred—a 30-step trajectory when the average is 15 gets a very small reward, while a 17-step trajectory when the average is 15 gets only a modest penalty.

Why group-relative not absolute: The group-relative design—comparing each trajectory against the average of its rollout group rather than against a fixed threshold—has two key properties. First, it is self-normalizing: as the agent improves and all trajectories get shorter, $\bar{s}$ decreases accordingly, so the reward continues to provide meaningful differentiation between trajectories within each group. If a fixed threshold were used (e.g., "reward trajectories shorter than 20 steps"), the reward would become meaningless once the agent consistently achieves shorter trajectories, providing no further incentive for improvement. Second, it provides a natural curriculum: early in training, when trajectories are mostly long, even modestly shorter trajectories get rewarded, encouraging initial exploration of efficiency. As training progresses and average trajectory length decreases, the bar for receiving a bonus rises, pushing the agent toward increasingly efficient paths.

Why this form for $R_{\text{length}}$: The piecewise definition with linear bonus for shorter trajectories and exponential penalty for longer ones creates an asymmetric incentive structure. The linear bonus for shorter trajectories means the marginal benefit of reducing steps is constant—saving one more step always provides the same additional reward, regardless of how short the trajectory already is. This encourages continued efficiency improvements even when the agent is already performing well. The exponential penalty for longer trajectories means the marginal cost increases as the trajectory gets longer—going from 20 to 25 steps (when $\bar{s} = 15$) costs more than going from 15 to 20 steps. This strongly penalizes the degenerate behavior of extremely long, inefficient trajectories while being relatively forgiving of trajectories that are slightly above average.

Interaction between $R_{\text{tool}}$ and $R_{\text{length}}$: These two reward terms are designed to work together to produce tool-appropriate and efficient behavior. $R_{\text{tool}}$ provides a binary signal about whether tools were used when they should be; $R_{\text{length}}$ provides a continuous signal about execution efficiency. A trajectory that uses tools appropriately but is still long will get $R_{\text{tool}} = 1$ (bonus) but $R_{\text{length}} < 1$ (penalty), encouraging the agent to find tool call opportunities that actually shorten the path. A trajectory that is short but uses tools inappropriately will get $R_{\text{tool}} = 0$ (no bonus) but potentially $R_{\text{length}} > 1$ (efficiency bonus), showing that while it was efficient, it missed an opportunity to be even more efficient or more robust through appropriate tool use. The optimal trajectory under this reward is one that is both tool-appropriate and efficient—using tools exactly when the task benefits from them and resulting in a shorter path than the group average.

Multi-Turn GRPO with Dynamic Filtering

The online RL uses multi-turn GRPO, extending the group-relative policy optimization from the single-turn setting to complete trajectories. The model generates full trajectories of up to $S_{\max} = 30$ steps, each step involving an action, environment execution, observation (screenshot + tool response), and the cycle continuing. After all rollouts in a group are complete, the composite reward $R$ is computed for each trajectory, advantages are computed as group-relative normalized rewards, and the policy is updated.

Dynamic filtering (DAPO-inspired): Following the DAPO approach, rollout groups are filtered to retain only those containing both successful and failed trajectories. This is important because groups where all trajectories succeed or all fail provide no relative signal for GRPO's advantage computation—if all rollouts have the same outcome, the group-relative advantage is zero for all trajectories, and no meaningful update occurs. By discarding homogeneous groups, the training focuses computation on the informative cases where the model can learn what distinguishes successful trajectories from failed ones.

Each training step uses approximately 1,200 effective training samples (Appendix C.5) after dynamic filtering, with training running for about 25 optimization steps. Each ablation run consumes approximately 8 × 8 GPUs plus distributed ECS sandbox workers for about six days, reflecting the substantial computational cost of running hundreds of parallel desktop environment instances alongside large-scale model training.

Tool-calling interface optimization (Appendix C.3): The paper notes that the tool-calling interface is optimized with "an agent-readable return format that provides concise, semantically dense feedback to reduce token overhead and improve grounding accuracy." This is a practical engineering detail that matters for RL training efficiency: if tool responses are verbose or poorly structured, they consume many tokens in the model's context window, increasing inference cost during rollouts and making it harder for the model to extract the relevant signal (success/failure, specific result values, error messages). Concise, structured feedback reduces the number of tokens the model must process per step, enabling more rollouts within the same compute budget.

Message construction for training and inference (Appendix F): The agent receives a system prompt composed of three parts: (a) the predefined GUI action schema for computer_use (always present), (b) optional MCP tool definitions appended from the environment's tool list (only when tools are available for the current task), and (c) an important-reminder section that varies based on the action space. For the hybrid GUI-Tool setting, the reminder includes specific guidance about tool behavior: "Some MCP Tool actions may NOT cause any visible change in the screenshot, so rely on the JSON tool result when appropriate" and "Do NOT repeat the same MCP Tool call if it keeps failing or produces no useful progress."

The historical context for each step is constructed with a sliding window of at most 5 previous screenshots (image history), each paired with its tool-calling result in a <tool_response> block. Steps before the window are compressed into a text-only action history embedded in the instruction prompt. This windowing is a practical necessity for online RL: retaining all previous screenshots would cause the context length to grow unboundedly over 30-step trajectories, making inference prohibitively expensive and potentially exceeding the model's context window. Limiting to 5 recent images balances between providing sufficient visual context for the model to understand the current state and keeping inference costs manageable.

Summary of Key Design Choices

The staged training paradigm (SFT → Single-Turn RL → Online RL) is not an arbitrary decomposition but follows a deliberate progression of learning objectives:

  • SFT on D_all teaches the model the building blocks of hybrid action behavior—what tools exist, how to format tool calls, how to interpret tool responses, and what interleaved trajectories look like. This is pure imitation learning; the model learns to reproduce patterns from the training data without understanding why certain decisions are better than others.

  • Single-turn RL on D_critical targets the specific decisions where the model is most likely to fail—the "forked road" moments where it must choose between GUI and tool paths. By focusing RL on these boundary points, the training efficiently allocates RL computation to the hardest decisions rather than spreading it across all steps uniformly. The single-turn setting is computationally cheaper than multi-turn RL (no need to simulate full trajectories, no credit assignment over long horizons), making it a cost-effective intermediate step.

  • Online RL with Tool-Efficient Path Reward provides the trajectory-level feedback that neither SFT nor single-turn RL can offer. The model experiences the consequences of its decisions over complete trajectories, learns through exploration of alternative paths, and is shaped by rewards that explicitly encode what makes a trajectory efficient and tool-appropriate. This stage is computationally expensive (requiring sandbox infrastructure and multi-turn rollouts) but addresses the limitations of the offline stages.

This progressive design means that the online RL stage starts from a strong initialization (M_rft already has basic tool knowledge and calibrated local switching), allowing it to focus on trajectory-level optimization rather than having to learn tool syntax and basic switching from scratch through sparse environment rewards—which would be extremely sample-inefficient, as the ablation results confirm (Section 3.3, "w/o Interleaved data" in Figure 6).

4. Key Insights and Innovations

Innovation 1: Optimal Path Confusion as a Diagnostic Concept

The paper's most conceptually significant contribution is not a method but a diagnostic reframing: it identifies that the core barrier to effective hybrid GUI-Tool agents is not capability (the model can't call tools) or knowledge (the model doesn't know what tools exist), but rather optimal path confusion—a trajectory-level decision problem where agents systematically make poor choices at the branching points between GUI and tool execution paths, even when they possess both the GUI grounding ability and the tool-calling knowledge individually.

This reframing matters because it explains a pattern that was visible but undiagnosed in prior work. The field implicitly operated under the assumption that exposing models to both action spaces would be sufficient—if a model can click and type competently and can format tool calls correctly, then giving it access to both should improve performance. The counterintuitive empirical reality documented in Table 1 systematically demolishes this assumption: Claude-4.5-Sonnet drops from 61.9% to 48.4%, EvoCUA-32B drops 12.0%, Qwen3VL-235B drops 2.0%—all from merely adding tool access. These aren't marginal degradations; they represent fundamental failures of the "add tools and prompt" paradigm. Prior work (UI-Tars, OpenCUA, GUI-Owl) either ignored tools entirely, treated them as always-available extensions of GUI action spaces, or delegated tool decisions to separate routing modules without framing the switching decision itself as the learning bottleneck.

What makes this a genuine diagnostic concept rather than an observation is that it specifies the structure of the failure. The paper identifies two symmetric failure modes: tool underuse (Qwen3VL-8B, averaging 0.003 tool calls and never exploiting available shortcuts) and tool overuse (Qwen3VL-235B, averaging 6.10 tool calls and degrading through inappropriate invocation). These aren't random errors—they are systematic biases reflecting that the model has no learned policy for the GUI↔Tool decision boundary. A model that underuses tools is effectively blind to the possibility of switching; a model that overuses tools fails to recognize when GUI grounding is necessary before tool invocation. Both emerge from the same underlying deficit: the absence of trajectory-level training that teaches when each mode is appropriate.

The significance extends beyond the specific domain. The optimal path confusion concept applies to any agent operating in a heterogeneous action space where different action types have qualitatively different cost-reliability profiles. This includes web agents choosing between DOM-based actions and visual clicking, coding agents choosing between file edits and shell commands, or multi-modal agents choosing between vision-based reasoning and structured API calls. The paper demonstrates that simply providing access is not enough—the selection policy itself must be a first-class learning objective.

The evidence for this diagnostic claim is concentrated in Table 1 and Figure 2. Table 1 shows that across model scales (8B to 235B, open-source to proprietary), the pattern holds: hybrid action spaces degrade performance without explicit training for path selection. ToolCUA is the sole counterexample in the table—the only model that improves with tool access (+3.9%)—which directly validates the claim that the confusion is trainable away with the right paradigm. The ablation in Figure 6 ("w/o Interleaved data") provides further evidence: even with online RL and the Tool-Efficient Path Reward, a model that skips the offline interleaved data bootstrapping fails to acquire reliable tool-calling behavior, confirming that the confusion cannot be resolved by environment rewards alone without foundational exposure to switching patterns.

Innovation 2: The Trajectory-Aware Tool Synthesis Paradigm

Prior work on tool-augmented agents has pursued two strategies for obtaining tool definitions, each with fundamental limitations. API-based tool generation (ToolLLM, Gorilla, UltraCUA) extracts tools from code repositories, documentation, or API specifications. This produces tools that are syntactically valid but disconnected from actual usage contexts—a libreoffice_create_pivot_table function generated from documentation tells the model what parameters exist but not when or how it would actually be invoked during a spreadsheet task. Manual tool construction (OSWorld-MCP) provides high-quality, verified tools but is fundamentally unscalable—each tool must be specified, tested across application versions, and maintained, limiting coverage to a small set of well-supported applications.

ToolCUA introduces a third paradigm: synthesize tools from observed GUI behavior rather than from code or documentation. The key idea is that every tool in the library is grounded in a concrete state transition visible in a recorded GUI trajectory—the MLLM watches what the human or agent did through GUI actions and abstracts that procedure into a callable tool signature. This inverts the direction of tool design: instead of starting from "what APIs does this application expose?" and hoping they match real workflows, it starts from "what workflows actually occur in computer use tasks?" and creates tools that capture them.

This paradigm shift has three conceptual implications that make it more than an engineering convenience:

First, it resolves the grounding problem inherent in API-based tools. Tools synthesized from documentation describe what an API could do in the abstract. But whether that abstract capability maps to actual task-relevant behavior in a specific desktop environment is an empirical question that documentation-based approaches don't answer. A tool for libreoffice_set_cell_format might exist in the API but behave subtly differently than expected when invoked in a real QEMU sandbox—timing issues, dialog popups, or application-specific quirks might make the documented behavior inaccurate. By grounding every synthesized tool to an observed screenshot transition, the pipeline verifies that the tool corresponds to something that demonstrably happens. The next-state grounding procedure (Section 2.2, Stage 3) is not just a quality check—it's the mechanism that ensures tools are behaviorally valid rather than merely syntactically plausible.

Second, it makes tool synthesis scalable because it reuses existing data infrastructure. The computer use community has invested heavily in GUI trajectory datasets (OpenCUA's 8,500 trajectories, ScaleCUA's collection, CUA-Suite's video demonstrations). These datasets were created for training GUI-only agents and would be tremendously expensive to recollect with tool annotations. ToolCUA's pipeline converts this existing investment into hybrid-action training data without requiring new environment instrumentation. This is conceptually significant because it suggests a general principle: static trajectory data can be retroactively enriched with structured action abstractions, not by re-executing the trajectories but by analyzing what patterns they contain.

Third, it generates diverse tool granularities that create realistic decision complexity. The paper's bottom-up merging strategy produces tools at fine, mid, and coarse granularities, with 4,350 unique tools across the full library (Appendix C.2, Table 6). This matters because real tool-use decisions involve choosing not just whether to use a tool but which granularity of tool is appropriate. Should the agent call a fine-grained libreoffice_select_data_range followed by libreoffice_open_insert_menu, or the coarse-grained libreoffice_create_pivot_table that encompasses both? The correct answer depends on context—what other tools are available, how reliable each is, whether the agent needs intermediate visual confirmation. By training on trajectories with multi-granularity tool libraries, the model learns to make these meta-decisions rather than simply defaulting to the most powerful tool available.

The evidence for this innovation's importance appears in the ablation results. Figure 6 ("w/o Interleaved data") shows that skipping the entire synthesis pipeline—and performing online RL directly from the base model—produces an agent that almost never calls tools (tool calls stay near zero throughout training, TIR reaches only ~15%). This demonstrates that the synthesized interleaved data provides something that online RL alone cannot: the foundational exposure to tool signatures, switching patterns, and interleaved trajectory structures that enables subsequent learning. The synthesis pipeline is not just a data augmentation trick but the enabling condition for hybrid-action training.

Innovation 3: Decomposed Trajectory-Level Shaping for Heterogeneous Action Spaces

The paper's Tool-Efficient Path Reward represents a conceptual advance in how to provide reinforcement signals for agents operating in hybrid action spaces. The standard approach in prior CUA RL work (MobileRL, ARPO, GUI-R1, ZeroGUI) uses task-completion as the primary or sole reward, sometimes supplemented with a format reward for output validity. The implicit assumption is that if the agent can explore enough trajectories in the environment, it will naturally discover which action types lead to success and which don't.

ToolCUA challenges this assumption by arguing that sparse task-completion rewards are insufficient for learning in heterogeneous action spaces because they conflate two orthogonal dimensions of trajectory quality: was the task completed? and was the path taken efficient and appropriate? A trajectory can succeed through a 30-step pure-GUI workaround or through a 10-step sequence with well-placed tool calls—both receive the same task-completion reward, so the agent has no signal to prefer the latter. Over many rollouts, the agent may converge to the path that is statistically easier to discover (often the GUI-heavy path, since it requires no tool-calling knowledge) rather than the path that is genuinely more effective.

The decomposition into R_tool and R_length addresses this by providing orthogonal shaping signals that can be optimized independently:

  • R_tool addresses the tool selection dimension: it rewards tool use on tasks where tools are beneficial and tool abstention on tasks where they aren't. This signal is binary and task-conditional—it doesn't care whether the tool call was optimally parameterized or whether the trajectory was short, only whether the agent's tool-usage pattern aligned with task structure. By decoupling tool-appropriateness from task success (it only fires on successful trajectories, but provides additional reward beyond success), it creates a gradient toward tool-appropriate behavior that pure success rewards cannot provide.

  • R_length addresses the path efficiency dimension: it rewards trajectories that are shorter than the group average, independent of whether tools were used. This captures the intuition that tools are valuable primarily because they shorten trajectories, not because they are inherently good. By making efficiency a group-relative metric, it adapts to the agent's current capability level—as training progresses and average trajectory length decreases, the bar for receiving an efficiency bonus rises, preventing reward saturation.

The conceptual significance of this decomposition is that it makes explicit what prior work left implicit. In pure GUI settings, the path efficiency dimension largely collapses into task success—the only way to complete a task is through GUI actions, so shorter trajectories are naturally better but there's no qualitatively different path to discover. In hybrid settings, the agent faces genuine tradeoffs: take the reliable but long GUI path, or take the risky but short tool path? The decomposed reward provides separate gradients for each aspect of this tradeoff, allowing the agent to learn both simultaneously rather than having to infer tool value indirectly from the noise of task-completion outcomes.

The group-relative design of R_length is a subtle but important choice. An absolute threshold for "efficient" (e.g., reward trajectories shorter than 20 steps) would be brittle—it would need to be tuned per task, would become meaningless as the agent improves, and would provide no signal for tasks where even optimal trajectories are long. The group-relative formulation ensures that the efficiency signal is always meaningful: on hard tasks where all trajectories are long, the relatively shorter ones still get rewarded; on easy tasks where all trajectories are short, only the exceptionally efficient ones get a bonus. This adaptivity is what makes the reward function work across the diverse difficulty distribution of OSWorld tasks without task-specific tuning.

The evidence for this innovation's effectiveness is in Figure 6 ("w/o Our path reward"). When R_tool and R_length are removed and the agent trains with only accuracy and format rewards, accuracy becomes unstable (showing a clear drop mid-training and ending ~7 percentage points lower), TIR and tool-calls fluctuate without a consistent upward trend, and trajectory length lacks a stable downward trend. This demonstrates that the shaped reward is not merely an incremental improvement—it qualitatively changes what the agent learns from online exploration, enabling it to discover tool-appropriate and efficient paths that vanilla GRPO with task rewards cannot reliably find.

Innovation 4: The Staged Training Paradigm as a Solution to Credit Assignment in Heterogeneous Sequences

While each component of ToolCUA's training pipeline (data synthesis, SFT, single-turn RL, online RL) has precedents in prior work, the paper's intellectual contribution lies in the architecture of the staging itself—the specific sequence of learning objectives and how each stage addresses a distinct aspect of the credit assignment problem in hybrid action spaces.

The central challenge in training a hybrid GUI-Tool agent is that the learning signal is sparse along multiple dimensions simultaneously. The model needs to learn: (a) what tools exist and how to format them (knowledge), (b) when to switch between GUI and tool modes (local decision-making), and (c) how switching decisions cascade to affect trajectory-level outcomes (global credit assignment). A single-stage approach—whether pure SFT (which can teach (a) but not (b) or (c)), pure online RL from scratch (which must learn (a), (b), and (c) simultaneously from sparse rewards), or single-stage RL from SFT initialization (which lacks intermediate calibration of switching decisions)—will inevitably struggle with at least one of these dimensions.

ToolCUA's staged design maps each learning objective to a stage where the signal-to-noise ratio for that objective is maximized:

Stage 1 (SFT on synthesized interleaved data) maximizes the signal for tool knowledge acquisition. The dense token-level supervision of SFT is ideally suited for learning tool signatures, parameter formats, and basic interleaving patterns—these are exactly the kind of syntactic and semantic regularities that next-token prediction excels at capturing. The SFT stage doesn't try to teach optimal switching; it simply exposes the model to the distribution of switching patterns, giving it a prior over when switches tend to occur.

Stage 2 (single-turn RL on critical steps) maximizes the signal for local switching calibration. By isolating the exact GUI↔Tool boundary steps and applying RL only at those points, the training focuses computation on the hardest decisions. The single-turn setting removes the credit assignment problem—the model receives immediate feedback on whether its switching choice was correct, without having to infer it from downstream outcomes 10 steps later. This stage is conceptually analogous to shaping in classical RL: provide intermediate rewards at sub-goal points to accelerate learning of the full policy.

Stage 3 (online RL with decomposed rewards) maximizes the signal for trajectory-level orchestration. By this stage, the model already knows how to use tools (from SFT) and when to switch locally (from single-turn RL). The online RL can therefore focus on the global dimension—how switching decisions interact over multi-step trajectories—without wasting samples on basic tool syntax or obvious switching errors. The decomposed reward (R_tool + R_length) provides gradient signals for both tool-appropriateness and path-efficiency that the earlier stages couldn't provide.

The staged design is not merely an engineering convenience—it's a solution to a multi-scale credit assignment problem. Different aspects of hybrid-action behavior operate at different temporal scales (single actions, local switching decisions, full trajectories), and the optimal supervision signal varies by scale (dense token-level for actions, immediate binary for local switches, shaped trajectory-level for global orchestration). The staging maps scale to signal quality, ensuring that each learning objective is trained with the supervision density that best matches its difficulty.

The evidence for the staging's necessity is distributed across the ablation results. Table 3 shows that pure GUI SFT + RL (skipping the hybrid data and switching calibration entirely) reaches 42.05%, while the full staged pipeline reaches 46.85%—a nearly 5-point gap that reflects the value of the hybrid staging. Figure 6 ("w/o Interleaved data") shows that online RL from the base model without SFT initialization fails to acquire tool-calling, confirming that Stage 1 is necessary. And "w/o Our path reward" shows that online RL from M_rft without the shaped reward underperforms, confirming that Stage 3's reward design is necessary. The staging is not an arbitrary decomposition—it's a principled mapping of the three learning sub-problems to the three training regimes where they can be most effectively solved.

5. Experimental Analysis

Evaluation Methodology

  • Dataset. The primary benchmark is OSWorld-MCP (Jia et al., 2025), which extends OSWorld (Xie et al., 2024) with 150+ MCP-based tools and covers mainstream desktop applications (LibreOffice, Chrome, VSCode, Thunderbird, GIMP, terminal, file manager, system settings). Evaluation is conducted on the feasible tasks subset (333 tasks total), following the benchmark's protocol, where "feasible" means tasks that are executable in the sandbox environment without environmental failures. The authors also evaluate cross-task generalization on held-out Linux multi_apps tasks and cross-platform transfer on WindowsAgentArena (Bonatti et al., 2024). All training tasks are drawn from OSWorld excluding the multi_apps domain, which is reserved for out-of-distribution verification (Appendix C.3).

  • Base model(s). All ToolCUA variants are built on Qwen3-VL-8B-Instruct (Bai et al., 2025), an 8-billion parameter vision-language model with multimodal understanding capabilities. The authors argue this model is "representative of the capabilities of many contemporary LLMs" (Section 4) and sits in a regime where it has non-trivial GUI grounding ability (~29% accuracy on OSWorld in the pure GUI setting, Table 1) but far from saturation, leaving substantial room for hybrid-action training to make a difference. The model is chosen over alternatives because ToolCUA targets the sub-10B parameter class where open-source CUA development has been most active (UI-Tars-1.5-7B, GUI-Owl-1.5-8B, EvoCUA-8B).

  • Metrics. The paper reports three primary metrics following the OSWorld-MCP protocol (Appendix C.4):

    • Task Accuracy: Whether the agent completes the target instruction according to the benchmark evaluator. This is the primary success metric and reflects both visual grounding and tool-calling ability. Reported as average@3 to mitigate sandbox stochasticity.
    • Tool Invocation Rate (TIR): Defined as (n_t + n_g) / (N_t + N_g), where N_t is the total number of Tool-Beneficial Tasks, n_t is the number of such tasks where the agent invoked a tool and succeeded; N_g is the total number of Non-Tool-Beneficial Tasks, n_g is the number where the agent did not invoke a tool and succeeded. TIR measures whether the agent aligns tool usage with task-level tool utility—using tools when beneficial, abstaining when unnecessary.
    • Average Completion Steps (ACS): The mean number of environment interaction steps across all tasks. Lower ACS indicates more efficient execution, typically reflecting the agent's ability to use tool calls to replace redundant GUI operations.

  • Baselines. The paper compares ToolCUA against two categories:

    • General-purpose foundation models: Gemini-2.5-Pro, OpenAI o3, Seed1.5-VL, Claude-4-Sonnet, Gemini-3.1-Pro, Claude-4.5-Sonnet, Qwen3-VL-235B-A22B, and Qwen3.5-397B-A17B. These represent state-of-the-art proprietary and large open-weight models evaluated with tool access but without specialized CUA training.
    • Specialized CUA models: UI-Tars-1.5-7B (Qin et al., 2025), EvoCUA-8B and EvoCUA-32B (Xue et al., 2026), and GUI-Owl-1.5-8B and GUI-Owl-1.5-32B (Xu et al., 2026). These are purpose-built computer use agents trained with various data scaling and RL strategies, operating at comparable or larger scales to ToolCUA-8B. The Qwen3-VL-8B-Instruct base model is also reported as a direct baseline to measure relative improvement.

  • Generation budget / compute accounting. The paper does not measure compute in FLOPs or tokens. Instead, the primary unit is environment steps—the number of actions executed before the agent terminates or reaches the maximum horizon. The maximum steps per task is set to 50 for evaluation and 30 for online RL training (to keep rollouts computationally feasible). For the online RL stage, training runs for approximately 25 optimization steps with a rollout size of 32 per group, using ~1,200 effective training samples after dynamic filtering (Appendix C.5). Compute infrastructure is substantial: 8 × 8 GPUs for policy training, 4 × 8 GPUs for dedicated inference serving, and approximately 250 independent Docker instances for sandbox rollouts (Appendix C.3).

  • Cross-validation / statistical protocol. To mitigate environmental stochasticity in the sandbox, all primary OSWorld-MCP metrics are reported as average@3—each task is evaluated three times and the average accuracy is reported. For the online RL training, dynamic filtering (inspired by DAPO) retains only rollout groups containing both successful and failed trajectories, improving the informativeness of group-relative policy updates. The multi_apps domain is explicitly held out from training and used exclusively for out-of-distribution evaluation. No test-set cross-validation is reported for hyperparameter selection (the paper uses fixed hyperparameters rather than sweeping on test data).

Main Quantitative Results

Overall Accuracy on OSWorld-MCP

The headline result in Table 2: ToolCUA-8B achieves 46.85% accuracy on OSWorld-MCP, establishing what the paper claims as state-of-the-art among models of comparable scale. This represents a +18.62 percentage point absolute improvement over the Qwen3-VL-8B-Instruct baseline (28.23%), corresponding to approximately 66% relative improvement. Among 8B-class specialized CUAs, ToolCUA surpasses the previous best GUI-Owl-1.5-8B (43.84%) by 3.01 points. Among general models, ToolCUA outperforms Gemini-3.1-Pro (41.14%) by 5.71 points and Claude-4-Sonnet (43.54%) by 3.31 points, while trailing only Claude-4.5-Sonnet (48.35%) by 1.50 points.

The performance is disaggregated by task type in Table 2:

  • Tool-Beneficial Tasks (238 tasks): ToolCUA achieves 45.80% accuracy, a +18.07 point gain over the baseline (27.73%). On these tasks where tools should help, ToolCUA substantially closes the gap to leading proprietary models (Claude-4.5-Sonnet: 50.00%).
  • Non-Tool-Beneficial Tasks (95 tasks): ToolCUA achieves 49.47% accuracy, a +20.00 point gain over the baseline (29.47%). Notably, ToolCUA's performance on these tasks exceeds its performance on tool-beneficial tasks, suggesting the model has learned to avoid tool overuse—it does not degrade on tasks where tools are unnecessary, unlike several baselines (EvoCUA-32B drops from 38.95% to 47.37% on non-tool-beneficial tasks but only achieves 37.82% on tool-beneficial ones, indicating inappropriate tool usage patterns).

Tool Invocation and Execution Efficiency

Beyond raw accuracy, ToolCUA demonstrates substantially improved GUI-Tool orchestration as measured by TIR and ACS (Table 2):

  • TIR increases from 8.41% to 24.32% (+15.91 percentage points), indicating that ToolCUA learns to invoke tools when beneficial and abstain when unnecessary. While this TIR is lower than several baselines (GUI-Owl-1.5-32B: 41.14%, Claude-4.5-Sonnet: 40.24%), ToolCUA achieves its accuracy with far fewer tool calls per trajectory.
  • ACS decreases from 19.34 to 14.93 (−4.41 steps), the lowest among all evaluated models. This is a critical efficiency signal: ToolCUA completes tasks in 23% fewer steps than the baseline while also being 66% more accurate. The next most efficient model is GUI-Owl-1.5-8B at 21.19 steps. ToolCUA's combination of high accuracy with low ACS suggests it is not merely succeeding through aggressive exploration but finding genuinely shorter execution paths through selective tool use.

The efficiency-accuracy tradeoff is further illuminated by the per-model comparison in Table 1. ToolCUA averages only 0.74 tool calls per trajectory while reducing steps from 19.4 to 14.9 and improving accuracy from 42.9% to 46.8% over its GUI-only counterpart. In contrast, Qwen3VL-235B averages 6.10 tool calls yet sees accuracy degrade from 41.1% to 38.1%, and EvoCUA-32B averages 7.49 tool calls with a 12.0% accuracy drop. This comparison makes concrete the paper's central claim: effective hybrid execution is about selective, well-timed tool use, not about using tools more frequently. ToolCUA achieves the best result with the fewest tool calls, demonstrating that the staged training paradigm teaches discrimination rather than indiscriminate tool invocation.

Hybrid vs. Pure GUI Training Effectiveness

Table 3 compares training in the hybrid GUI-Tool action space against pure GUI training at equivalent data scales:

  • Pure GUI pipeline: Baseline (29.03%) → SFT on GUI-only data (34.93%) → Agentic RL in GUI-only environment (42.05%). Total improvement: +13.02 points.
  • Hybrid GUI-Tool pipeline: Baseline (28.23% on OSWorld-MCP) → RFT on interleaved data (38.13%) → Full ToolCUA after online RL (46.85%). Total improvement: +18.62 points.

The hybrid pipeline achieves a larger absolute improvement (+18.62 vs. +13.02) and a higher final accuracy (46.85% vs. 42.05%). The advantage is present at both stages: RFT alone (38.13%) already outperforms GUI SFT + RL (42.05%) in the pure-GUI setting evaluated on OSWorld-MCP, suggesting that the interleaved training data provides benefits even before online optimization. The online RL stage then widens the gap further, likely because the GUI-Tool environment provides richer reward signals (the Tool-Efficient Path Reward includes R_tool, which has no equivalent in pure GUI settings) that enable more effective policy optimization.

A particularly notable finding: ToolCUA achieves 42.9% accuracy in pure GUI settings (Table 1, GUI row for ToolCUA), which is comparable to the 42.05% achieved by the GUI-only trained model. This means hybrid GUI-Tool training does not compromise GUI-only capability—the model retains its GUI grounding ability even though it was trained primarily on interleaved trajectories. This cross-mode transfer supports the paper's claim that training in a richer action space develops generalizable skills rather than overfitting to tool-specific patterns.

Difficulty-Dependent and Domain-Specific Performance

Figure 5 breaks down performance across the 11 application domains in OSWorld-MCP, revealing substantial heterogeneity in where ToolCUA's gains concentrate:

  • Highest absolute accuracy: vs_code (94.4%), chrome (85.7%), vlc (81.2%). These applications have well-structured tool interfaces where tool calls can reliably replace GUI operations.
  • Largest improvements over RFT alone: libreoffice_calculation (23.9% → 34.8%, +10.9 points), thunderbird (64.3% → 85.7%, +21.4 points), os (47.4% → 68.4%, +21.0 points). These domains benefit substantially from the online RL stage, suggesting that the Tool-Efficient Path Reward helps the agent discover efficient tool-use strategies that SFT alone did not capture.
  • Smallest improvements or plateaus: vlc remains flat between RFT and ToolCUA (46.7% → 53.3%) and gimp shows marginal change. These are applications where tool interfaces may be less comprehensive or where tasks are inherently visual (image editing), making GUI actions more necessary regardless of tool availability.
  • Baseline comparison: The baseline (Qwen3-VL-8B-Instruct) achieves non-zero accuracy in only some domains (e.g., 77.8% on vs_code, 66.7% on vlc, 57.1% on chrome) but near-zero on others (19.6% on calculation, 9.8% on multi_apps). ToolCUA improves across all domains, with particularly large relative gains on domains where the baseline struggled (e.g., calculation from 19.6% to 34.8%, multi_apps from 9.8% to 23.9%).

The domain breakdown reveals that ToolCUA's gains are not uniform—they are largest where tools can substitute for the most GUI steps (structured data operations in LibreOffice, email management in Thunderbird, system operations) and smallest where visual grounding is inherently dominant (image editing in GIMP). This aligns with the paper's theoretical framing: the value of hybrid action training scales with how much efficiency tools can provide over pure GUI approaches, and that value varies substantially by application.

Cross-Task and Cross-Platform Generalization

The paper presents generalization results on two held-out settings not seen during online RL training:

Cross-task generalization (Figure 5, Table in Section 3.2). ToolCUA achieves 23.9% on the held-out multi_apps domain, improving from 18.5% after RFT alone and 9.8% for the baseline. This is a 14.1 percentage point absolute improvement over the baseline, despite the multi_apps domain being explicitly excluded from online RL training. The gain is particularly meaningful because multi-app tasks require the agent to coordinate across different applications—exactly the kind of scenario where GUI-Tool orchestration matters most, as different applications may have different tool availability. The result suggests that ToolCUA learns a generalizable switching policy that transfers to unseen task compositions, not merely memorizing application-specific patterns.

Cross-platform generalization (Table 4). ToolCUA achieves 33.8% accuracy on WindowsAgentArena, a Windows-based desktop benchmark, compared to 26.4% for the Qwen3-VL-8B-Instruct baseline (+7.4 percentage points). This is notable because ToolCUA was trained entirely on Linux-based trajectories and sandboxes (OSWorld QEMU images). The model transfers its GUI-Tool orchestration capability to a different operating system with different visual appearance, different application layouts, and different tool interfaces. ToolCUA also outperforms the larger Qwen3-VL-235B-A22B (32.1%), suggesting that cross-platform transferability is not simply a function of model scale but reflects learned orchestration skills that abstract beyond the training platform's specifics.

The cross-platform result, while positive, shows a smaller relative gain (+28% over baseline) compared to the in-distribution improvement (+66% on OSWorld-MCP). This is expected—the model cannot leverage platform-specific tool knowledge on an unseen OS, and the visual distribution shift likely degrades GUI grounding to some extent. The fact that gains do transfer, even partially, supports the claim that hybrid GUI-Tool training produces more generalizable agents rather than environment-specific specialists.

Online RL Training Dynamics

Figure 6 tracks four metrics across the ~25 steps of online RL training for ToolCUA and two ablations:

  • ToolCUA (full method): Test accuracy rises from approximately 38% (the RFT initialization) to ~47% by the end of training, with the steepest gains in the first ~8 steps. TIR increases from roughly 18% to ~24%, showing consistent improvement in appropriate tool use. The number of MCP tool calls per trajectory rises modestly from near zero to roughly 0.7, consistent with the final average of 0.74 in Table 1. Trajectory completion steps decline from ~17 to ~15, reflecting the efficiency gains from the R_length reward.
  • w/o Interleaved data (online RL directly from base model): Test accuracy improves from the baseline ~28% to ~40%, but tool-calling behavior remains near zero throughout (MCP tool calls stay close to 0, TIR peaks around 15%). This confirms that online RL alone, even with the Tool-Efficient Path Reward, cannot overcome the base model's GUI-centric bias without first being bootstrapped on interleaved training data.
  • w/o Our path reward (RFT-initiated model trained with only R_acc + R_fmt): Accuracy initially improves but then shows a clear drop around steps 8–11, recovering partially but ending ~7 percentage points below ToolCUA. TIR and tool-calls fluctuate without a consistent upward trend, and trajectory length lacks a stable downward trend. This demonstrates that the decomposed reward is necessary for stable, efficient learning—vanilla multi-turn GRPO with only task-completion signals does not reliably learn GUI-Tool orchestration.

The training dynamics validate the paper's staged design: the offline stages (data synthesis + RFT) provide a necessary foundation that online RL builds upon, and the shaped reward components (R_tool, R_length) provide learning signals that pure task-completion rewards cannot substitute for.

Ablation Studies and Robustness Checks

Interleaved GUI-Tool data bootstrapping (w/o Interleaved data, Figure 6): Removing the entire offline interleaved data pipeline—both warmup SFT and single-turn RL—and directly performing online RL from the base model causes the agent to remain GUI-centric. Tool calls stay near zero throughout training, TIR plateaus at ~15%, and final accuracy (~40%) substantially underperforms ToolCUA (~47%). This is the strongest evidence that the synthesis pipeline provides something irreplaceable: the base model has no prior over tool syntax, switching patterns, or tool↔screenshot relationships, and online RL's sparse rewards cannot efficiently teach these from scratch. The ~40% accuracy achieved is itself notable—it suggests that even without tool use, online RL with the Tool-Efficient Path Reward (minus R_tool, which would never fire) improves GUI grounding from the baseline's 28.23%—but the gap to ToolCUA shows this is a lower ceiling.

Tool-Efficient Path Reward components (w/o Our path reward, Figure 6): Training the RFT-initialized model with only R_acc + R_fmt (standard GRPO without tool-appropriateness or path-efficiency shaping) yields unstable learning dynamics and a ~7 percentage point accuracy gap compared to the full ToolCUA reward. The instability (accuracy drop mid-training) suggests that without explicit shaping, the agent oscillates between exploration strategies—sometimes discovering tool use, sometimes reverting to GUI-only behavior—without converging to a stable policy. The absence of stable downward trend in trajectory length confirms that R_length is necessary to drive efficiency improvements; task-completion rewards alone do not create pressure toward shorter paths.

Pure GUI vs. hybrid GUI-Tool training (Table 3): The pure GUI pipeline (GUI SFT + GUI agentic RL) reaches 42.05% accuracy; the hybrid pipeline reaches 46.85%. The 4.80 percentage point gap demonstrates that training in a hybrid action space provides benefits beyond what GUI-only training can achieve with comparable data and compute. The gap exists at both the SFT stage (GUI SFT: 34.93% vs. hybrid RFT: 38.13%) and widens after RL (GUI RL: 42.05% vs. ToolCUA: 46.85%), suggesting the hybrid environment provides both better initialization and better RL signal quality.

Cross-task generalization (Figure 5): ToolCUA's accuracy on the held-out multi_apps domain (23.9%) consistently exceeds the RFT-only model (18.5%) and the baseline (9.8%). This is not a controlled ablation (the multi_apps domain is simply excluded from training), but it serves as an robustness check on whether the model overfits to the training domains. The 5.4-point improvement over RFT alone on multi_apps—despite no multi_apps tasks in the online RL training—suggests that the online RL stage learns generalizable orchestration strategies rather than domain-specific tool invocation patterns.

Cross-platform transfer (Table 4): ToolCUA's 33.8% on WindowsAgentArena vs. the baseline's 26.4% (+7.4 points) demonstrates that training on Linux trajectories transfers partially to Windows. The authors present this as evidence of generalization, though it is not a controlled ablation—it shows that the full ToolCUA pipeline outperforms the base model, but does not isolate which components drive the transfer (is it the SFT data, the tool-calling knowledge, or the orchestration policy?).

Base model scale comparison (Table 2, multiple model rows): While not a direct ablation, the performance of Qwen3-VL-235B-A22B (38.14%) and Qwen3.5-397B-A17B (40.84%) compared to ToolCUA-8B (46.85%) demonstrates that scale alone does not solve the optimal path confusion problem. Much larger models, when evaluated with tool access but without ToolCUA's staged training, perform substantially worse than the 8B ToolCUA. This serves as a robustness check on the claim that training methodology, not model capacity, is the bottleneck.

Dynamic filtering during online RL (Section 2.4, Appendix C.5): The paper notes that DAPO-inspired filtering (retaining only groups with both successes and failures) is applied during online RL training. While no explicit ablation of this choice is reported, the training dynamics in Figure 6 suggest stable learning, which is non-trivial for multi-turn GRPO in stochastic sandbox environments. The filtering likely contributes to training stability by removing homogeneous rollouts that provide zero-gradient updates, but its quantitative impact is not isolated.

Critical Assessment

Claim 1: ToolCUA achieves 46.85% accuracy on OSWorld-MCP, a ~66% relative improvement over the baseline, establishing SOTA among similar-scale models. This claim is directly supported by Table 2, where ToolCUA-8B's 46.85% exceeds all other 7-8B class models (GUI-Owl-1.5-8B: 43.84%, EvoCUA-8B: 35.74%, UI-Tars-1.5-7B: 12.31%) and matches or exceeds several larger models (EvoCUA-32B: 40.54%, Qwen3-VL-235B: 38.14%). The ~66% relative improvement figure is computed as (46.85 - 28.23) / 28.23, which is an accurate reflection of the absolute gains. However, the "SOTA among similar-scale models" claim is constrained by the paper's model selection—very few 8B-class models have been evaluated on OSWorld-MCP with tool access, and other strong CUAs (e.g., UI-Tars-2, OpenCUA variants at 8B) are not present in the comparison table. The claim holds against the baselines that are evaluated but may be making a stronger statement than the evidence base supports.

Claim 2: ToolCUA demonstrates +3.9% improvement over pure GUI settings, showing successful GUI-Tool orchestration. This claim is supported by Table 1, where ToolCUA's hybrid accuracy (46.8%) exceeds its GUI-only accuracy (42.9%). This is methodologically strong because it uses the same model evaluated in both settings, isolating the effect of tool access. However, the claim's persuasive force depends on the 42.9% GUI-only figure. The paper reports this as ToolCUA's GUI-only performance (Table 1, "GUI" row for ToolCUA), but the experimental setup for obtaining this number deserves scrutiny: it requires evaluating the model trained on interleaved GUI-Tool data in an environment where tools are disabled, which the model was not explicitly prepared for. The fact that it achieves 42.9%—comparable to the GUI-only trained model's 42.05% (Table 3)—is presented as evidence that hybrid training doesn't degrade GUI capability, but the experimental design for measuring this is not fully detailed (were tools simply removed from the system prompt? Was the model fine-tuned for this setting?).

Claim 3: The staged training paradigm (synthesis pipeline → RFT → online RL) is necessary for learning optimal GUI-Tool path selection. The ablation evidence for this claim is strong in demonstrating that each component contributes, but less strong in establishing necessity:

  • The "w/o Interleaved data" ablation (Figure 6) shows that online RL from scratch fails to acquire tool-calling behavior. This supports the necessity of offline bootstrapping but does not distinguish whether the critical ingredient is the synthesized interleaved data, the SFT training on it, or the single-turn RL at critical steps—the ablation removes all three simultaneously.
  • The "w/o Our path reward" ablation (Figure 6) shows that R_tool + R_length substantially improves online RL. This supports the value of decomposed rewards but does not test whether alternative reward designs (e.g., reward based on absolute rather than relative efficiency, or reward for tool use proportional to step reduction) would work equally well.
  • Missing ablation: single-turn RL on critical steps is never ablated in isolation. The paper cannot distinguish whether the improvement from RFT comes primarily from the warmup SFT on D_all, from the single-turn RL on D_critical, or from their combination. This is a significant gap because the paper claims (Section 2.3) that calibrating "critical switching steps" is important, but provides no experimental evidence that targeting these steps specifically matters more than simply doing more SFT or RL on all steps.
  • Missing ablation: tool granularity diversity. The synthesis pipeline deliberately produces fine, mid, and coarse tools, but the paper does not test whether multi-granularity tool libraries outperform single-granularity ones. The claim that this diversity matters (Section 2.2) is asserted but unverified.

Claim 4: The Tool-Efficient Path Reward enables appropriate tool use and shorter execution paths. Figure 6 provides strong support: removing R_tool and R_length causes unstable accuracy, lack of TIR improvement, and no downward trend in trajectory length. However, the reward design has several unexamined assumptions:

  • The task-level tool-beneficial labels t_b are taken as ground truth. Appendix C.3 mentions "secondary manual verification" of these labels, but the paper provides no analysis of label quality, inter-annotator agreement, or edge cases where tool beneficence is ambiguous. If a task labeled t_b = 1 is actually completable efficiently without tools, R_tool could incentivize unnecessary tool calls. The relatively low TIR of ToolCUA (24.32%, Table 2) compared to its accuracy (46.85%) suggests that the model often succeeds on tool-beneficial tasks without calling tools—does this mean it's not learning to use tools where it should, or that the t_b labels overestimate tool beneficence?
  • The group-relative design of R_length is presented as adaptively self-normalizing, but the paper doesn't test alternatives (absolute threshold, task-specific thresholds, percentile-based). The ablation removes R_length entirely, which tests its value, but doesn't test whether the specific functional form (piecewise linear bonus + exponential penalty) matters.
  • The reward weights (λ = 0.4, β = 0.2) are set heuristically without sensitivity analysis. The paper doesn't explore whether different weightings change the learned behavior, which limits the generalizability of the reward design to other settings.

Claim 5: Hybrid GUI-Tool training produces more generalizable agents, as evidenced by cross-task and cross-platform transfer. The cross-task result on multi_apps (23.9% vs. baseline 9.8%) and the cross-platform result on WindowsAgentArena (33.8% vs. baseline 26.4%) are positive but contextually limited:

  • The multi_apps result (Figure 5) compares ToolCUA against the baseline and RFT-only model, showing improvement. However, no comparison is provided against the pure-GUI trained agent (Table 3, 42.05%) on multi_apps tasks specifically. If the pure-GUI model also transfers well (which is plausible since it was trained on diverse GUI trajectories), the claim that hybrid training specifically improves generalization would be weakened.
  • The WindowsAgentArena result (Table 4) compares ToolCUA against the base model and two larger Qwen3-VL variants, but doesn't include the GUI-only trained agent or the RFT-only model. Without these comparisons, it's unclear whether the Windows transfer gain comes from the full ToolCUA pipeline or from any training on desktop tasks (even pure GUI training). The 33.8% accuracy, while exceeding the 32B model's 30.9%, is a relatively modest absolute number on a 50-step task benchmark—it's plausible that this reflects improved GUI grounding from additional training data rather than learned tool orchestration transfer.
  • Neither the multi_apps nor the WindowsAgentArena result isolates the contribution of tool-use skills specifically. The model could be benefiting from general visual grounding improvements from training on diverse screenshots, rather than from learning transferable tool-switching policies. An ablation where the model is trained on the same data but with tools disabled would help disentangle these effects.

Overall experimental limitations:

  • Single benchmark for primary evaluation: All main results are on OSWorld-MCP's 333 feasible tasks. While this is a reasonable scope for a paper introducing a new training paradigm, the claim that hybrid GUI-Tool training is "a promising paradigm for real-world digital agents" (abstract) would be strengthened by evaluation on additional hybrid-action benchmarks (e.g., MCPWorld, which is cited as related work but not used for evaluation). The 333-task test set, split into 11 application domains, means some domains have very few tasks (~10–20), making per-domain accuracy estimates noisy.
  • Uncontrolled sandbox stochasticity: The paper uses average@3 to mitigate sandbox randomness, but desktop environments are inherently non-deterministic. A tool that works in 2 out of 3 runs due to timing issues could still produce a "correct" average@3 result if the evaluator is lenient. The paper doesn't report variance across the three runs or discuss how many tasks had inconsistent outcomes.
  • Reliance on strong MLLMs for data synthesis: The synthesis pipeline uses Kimi-K2.5 or Claude-4.5-Sonnet (Section 2.2), and Appendix A acknowledges that replacing these with Qwen3.5-Plus "led to noticeably lower generation efficiency and trajectory quality." This means ToolCUA's training data quality is bounded by proprietary model capability, creating a reproducibility challenge for researchers without access to these models. The paper also doesn't report how much the synthesis pipeline costs in API calls or compute.
  • No comparison to retrieval-augmented or planning-based methods: The baselines are all end-to-end models. Methods that decompose the problem—using separate modules for tool selection and GUI grounding, or using planning over tool sequences—are not compared. This makes it difficult to assess whether ToolCUA's end-to-end approach outperforms modular alternatives or simply hasn't been compared to them.
  • Computational cost of the full pipeline: Each online RL ablation run consumes 8 × 8 GPUs plus ~250 Docker sandbox instances for approximately six days (Appendix C.5). This is a substantial compute investment for an 8B model, and the paper doesn't provide FLOP counts or dollar-cost estimates that would help practitioners assess whether the 4×4\times efficiency gains at inference time (fewer steps per task) offset the training cost.

6. Limitations and Trade-offs

1. Difficulty Estimation Cost Is Not Accounted for in the Pipeline

The assumption or constraint. The interleaved GUI-Tool trajectory scaling pipeline depends on powerful proprietary MLLMs (Kimi-K2.5 or Claude-4.5-Sonnet) to synthesize tools, generate tool-only trajectories, perform next-state grounding, and construct interleaved variants. The paper acknowledges this dependence in Appendix A (Limitations and Future Works):

"Our tool-scaling process also depends on the capability of the general model used for synthesis; in our internal trials, replacing stronger proprietary models with Qwen3.5-Plus led to noticeably lower generation efficiency and trajectory quality."

The pipeline also requires processing approximately 10,000 GUI trajectories through multiple MLLM calls per trajectory (screenshot description, tool generation, tool trajectory generation, state grounding, bottom-up merging, interleaved variant construction), representing millions of API calls to proprietary models.

The consequence. This creates two practical problems. First, the data synthesis pipeline has a substantial but unreported financial and computational cost that is not amortized into any of the paper's efficiency or accuracy claims. A practitioner seeking to replicate ToolCUA for a new application domain would need either (a) access to state-of-the-art proprietary models and a large API budget, or (b) a comparably capable open-weight model—and the paper provides negative evidence for option (b) by noting that Qwen3.5-Plus produced noticeably worse synthesis quality. Second, the quality ceiling of the entire training pipeline is bounded by the synthesis model's capability. If the synthesis MLLM produces suboptimal tools (wrong granularity, missing parameters, inaccurate grounding), the interleaved trajectories will encode those suboptimalities, and the trained agent will inherit them. The paper provides no systematic analysis of synthesis quality or how synthesis errors propagate to downstream agent performance.

What evidence exists in the paper. Appendix A explicitly acknowledges the dependence but provides no quantification: no synthesis error rates, no comparison of agent performance when trained on data synthesized by different MLLMs, no cost estimates. Table 6 reports aggregate statistics (4,350 unique tools, 19.75 tools per trajectory pool) but does not report how many synthesized tools were discarded due to validation failures or what fraction of generated trajectories required manual correction.

Mitigation status. The paper acknowledges this as a limitation in Appendix A and suggests future work on "reducing the dependence of agentic RL on heavy sandbox infrastructure by building lighter, more diverse, and more robust environments." It does not propose any concrete method for reducing the synthesis pipeline's dependence on proprietary models, nor does it report costs that would allow practitioners to budget for replication. This limitation is entirely unaddressed in the current work.

2. Single Benchmark and Single Model Family Constrain Generality Claims

The assumption or constraint. All primary experimental results are on a single benchmark (OSWorld-MCP, 333 feasible tasks) with a single base model family (Qwen3-VL, specifically the 8B-Instruct variant). The paper acknowledges this narrow evaluation scope in Appendix A:

"due to the scarcity of open-source GUI-Tool coordination benchmarks for computer-use agents, our main performance evaluation is primarily conducted on OSWorld-MCP, leaving broader benchmark coverage as an important limitation."

The cross-platform evaluation on WindowsAgentArena (Table 4) uses only 5 models (Qwen3-VL-8B-Instruct, Qwen3-VL-32B-Instruct, Qwen3-VL-235B-A22B, and ToolCUA-8B) and does not include the RFT-only model or a GUI-only trained model, so it cannot isolate whether the transfer gain comes specifically from hybrid GUI-Tool training or from any additional training on desktop tasks.

The consequence. Several key claims in the paper lack cross-benchmark or cross-model-family validation. The claim that "training in a hybrid GUI-Tool action space is a promising paradigm for real-world digital agents" (abstract) rests entirely on OSWorld-MCP results with Qwen3-VL-8B. It is unclear whether the findings generalize in three critical dimensions:

  • To other model families: Qwen3-VL may have specific architectural properties (vision encoder design, multimodal fusion strategy, pretraining data distribution) that make it particularly amenable or resistant to hybrid-action training. The confusion patterns documented in Table 1 (tool underuse by 8B, tool overuse by 235B) might be specific to Qwen's training rather than universal.
  • To other benchmarks: OSWorld-MCP covers primarily Linux desktop productivity applications (LibreOffice, Thunderbird, Chrome). It is unclear whether the optimal path confusion problem and ToolCUA's solution generalize to web-based tasks (WebArena, Mind2Web), mobile tasks (AndroidWorld, MobileEnv), or tasks requiring different tool abstractions.
  • To tasks with ambiguous tool beneficence: OSWorld-MCP provides binary t_b labels for whether tools are beneficial. In real deployments, tool beneficence is often continuous and context-dependent—a tool might be beneficial only if the agent has correctly identified the target element, or only if the tool's API version matches the application version. The binary-label formulation may not capture this nuance.

What evidence exists in the paper. The cross-task generalization to multi_apps (Figure 5) and cross-platform transfer to WindowsAgentArena (Table 4) are positive signals but limited. The multi_apps result (23.9% vs. baseline 9.8%) shows improvement but the absolute accuracy is low—most multi-app tasks still fail. The WindowsAgentArena result (33.8% vs. baseline 26.4%) shows a +7.4 point gain, but without the RFT-only model as a comparison point, it's impossible to know whether this gain comes from the full ToolCUA pipeline (including tool training) or simply from additional GUI grounding training. The paper does not report results on any non-OSWorld hybrid-action benchmark (e.g., MCPWorld, which is cited in the related work but not used for evaluation).

Mitigation status. The paper explicitly identifies this as a limitation in Appendix A and suggests future work to "further explore hybrid GUI-Tool action spaces across broader platforms, including desktop, mobile, and web environments." No additional experiments or analyses are provided to bound the expected generalization. This limitation is acknowledged but entirely unaddressed in the current work.

3. The Single-Turn RL Stage on Critical Steps Is Never Ablated

The assumption or constraint. The Tool-Bootstrapped GUI RFT stage (Section 2.3) consists of two sequential components: warmup SFT on D_all and single-turn RL on D_critical. The paper claims that targeting "critical switching steps" is important:

"By sampling multiple completions at these critical switching steps, the model receives direct feedback on whether to continue with GUI actions or switch to tool calls when appropriate tools are available. This targeted optimization calibrates the model's discernment at decision boundaries."

However, no ablation isolates the contribution of single-turn RL from the contribution of warmup SFT. The "w/o Interleaved data" ablation (Figure 6) removes both SFT and single-turn RL simultaneously. The "w/o Our path reward" ablation removes R_tool and R_length from the online RL stage but keeps the full RFT initialization. There is no experiment comparing: (a) SFT only vs. (b) SFT + single-turn RL, to determine whether the 5,000 critical steps in D_critical provide value beyond what is already learned from the 180,000 steps in D_all.

The consequence. The paper's claim that "targeted optimization calibrates the model's discernment at decision boundaries" is unverified. It is equally plausible that the warmup SFT on 180,000 interleaved steps already provides sufficient exposure to GUI↔Tool switching patterns, and the single-turn RL stage adds negligible marginal benefit while consuming additional compute (group size 32, learning rate 1e-6, batch size 128). The paper cannot distinguish between these possibilities because the responsible ablation is missing. This matters for two practical reasons:

  1. Compute efficiency: If single-turn RL contributes little, practitioners could skip this stage and proceed directly to online RL, saving significant training compute.
  2. Conceptual validity: The concept of "critical switching steps" as distinct from other trajectory steps is central to the paper's framing (Figure 2, the "forked road" analogy). If the model learns switching equally well from full-trajectory SFT as from targeted RL at boundary points, the conceptual distinction between "local switching decisions" and "global trajectory optimization" would be empirically weaker than the paper presents.

What evidence exists in the paper. The paper reports the performance of M_rft (after SFT + single-turn RL) at 38.13% (Table 3) and the baseline at 28.23%, but does not report the performance of a model trained with only warmup SFT (without single-turn RL). The training dynamics in Figure 6 start from the RFT initialization (approximately 38% accuracy for ToolCUA), but since the initialization includes both SFT and single-turn RL, the curve does not reveal their independent contributions. The per-domain breakdown in Figure 5 shows the RFT-only model achieving 38.1% overall, but again without decomposing SFT vs. single-turn RL effects.

Mitigation status. This limitation is entirely unaddressed. The paper does not acknowledge the missing ablation, does not provide the SFT-only model's performance, and does not discuss the independent contribution of the single-turn RL stage. This is arguably the most significant experimental gap in the paper, as it leaves a core component of the claimed staged training paradigm empirically unvalidated.

4. Verifier Over-Optimization and Reward Hacking Are Not Analyzed

The assumption or constraint. The online RL stage optimizes the agent against a composite reward function R = R_fmt + R_acc + λ·R_tool + β·R_length in a sandbox environment. The paper assumes that optimizing this reward produces genuinely better GUI-Tool orchestration rather than reward hacking—finding trajectories that score highly under the reward function without actually exhibiting the desired behavior. This is a well-documented failure mode in RLHF and RL-based agent training, where agents learn to exploit reward function loopholes (e.g., taking inefficient actions that happen to satisfy format constraints, or terminating early to avoid accumulating errors while reporting spurious success).

The consequence. Several aspects of ToolCUA's reward design create potential for reward hacking:

  • R_tool rewards any tool call on tool-beneficial tasks: The reward fires whenever c > 0 (at least one tool call) on a t_b = 1 task, regardless of whether the specific tool call was helpful. An agent could learn to insert a single low-cost but useless tool call (e.g., env_info with no follow-up) early in the trajectory, collect the R_tool bonus, and then complete the task through GUI actions. This would satisfy the reward function but violate the intended behavior of using tools when genuinely beneficial.

  • R_length is group-relative and success-gated: The efficiency bonus only applies to successful trajectories (I_succ gating). This means the agent receives no efficiency feedback on failed trajectories, even if the failure was caused by taking an unnecessarily long path. An agent that succeeds on a task through a long GUI trajectory gets a positive (though sub-1.0) R_length, but an agent that fails while attempting a shorter tool-based path gets R_length = 0 (due to I_succ = 0). Over training, this could bias the agent toward safer but longer GUI-only strategies, since failed efficient attempts receive no efficiency signal.

  • R_acc is a sparse binary reward from the environment evaluator: The environment's task-completion check may have edge cases where the agent's trajectory state passes the evaluator despite not genuinely completing the task as a human would judge it. If such edge cases exist, the agent could learn to exploit them—taking actions that satisfy the evaluator function rather than the task intent.

What evidence exists in the paper. The paper provides no analysis of reward hacking, no qualitative examples of trajectories that score highly under the reward function but exhibit unintended behavior, and no comparison of learned policies against human judgments of trajectory quality. Figure 6 shows that removing R_tool and R_length (leaving only R_acc + R_fmt) produces less stable learning and lower final accuracy, which suggests the shaped rewards are providing useful signal. However, this does not rule out the possibility that the shaped rewards also introduce exploitable loopholes that the model is partially exploiting. The case study in Table 8 shows ToolCUA appropriately switching between tool calls (add_folder) and GUI actions (click on trust dialog), but this is a single cherry-picked example, not a systematic analysis of reward alignment.

Mitigation status. This limitation is entirely unaddressed. The paper does not discuss reward hacking as a potential concern, does not report analyses that would detect it (e.g., correlation between R_tool and actual tool usefulness, comparison of tool-call quality in high-reward vs. low-reward trajectories), and does not propose mechanisms to prevent it. The DAPO-inspired dynamic filtering removes homogeneous rollout groups, which might incidentally filter some reward hacking cases (if all trajectories in a group exploit the same loophole, the group is homogeneous and discarded), but this is not its intended purpose and would not catch cases where different trajectories exploit different loopholes.

5. The Hardest Tasks Remain Effectively Unsolved

The assumption or constraint. ToolCUA's training pipeline assumes that the base model has sufficient GUI grounding and reasoning capability to complete tasks with appropriate tool use, and that the staged training paradigm can teach the model to discover more efficient GUI-Tool paths. However, this assumption breaks down for tasks that are fundamentally outside the base model's capability range—where even the optimal GUI-Tool path cannot be discovered because the model cannot correctly execute the constituent steps.

The consequence. The paper's per-domain results in Figure 5 reveal substantial performance heterogeneity that maps onto task difficulty:

  • High-performing domains: vs_code (94.4%), chrome (85.7%), vlc (81.2%)—these applications have well-structured tool interfaces and tasks that align with common pretraining patterns.
  • Low-performing domains: libreoffice_calculation (34.8%), impress (44.7%), multi_apps (23.9%)—these involve complex multi-step reasoning, precise parameter specification, or cross-application coordination that likely exceeds the 8B model's capability.

The multi_apps domain is particularly revealing: despite being held out from online RL training, the absolute accuracy of 23.9% means more than three-quarters of cross-application tasks still fail. The libreoffice_calculation domain shows improvement from 19.6% (baseline) to 34.8% (ToolCUA), but 65% of tasks still fail. These failure rates suggest that ToolCUA's approach amplifies existing capability (it improves where the base model already has some competence) but does not create new capability—tasks requiring reasoning or grounding beyond the 8B model's capacity remain largely unsolved regardless of how effectively the model orchestrates GUI and Tool actions.

This is not merely an "unsolved by ToolCUA" problem but a fundamental bound on the hybrid GUI-Tool training paradigm. Tool calls can replace GUI operations that the model knows how to perform, but they cannot compensate for failures in visual grounding, instruction understanding, or multi-step planning. If the model cannot correctly identify which spreadsheet column contains the "Revenue" data, no amount of tool-training will let it create the correct pivot table—it will call create_pivot_table with incorrect parameters and fail.

What evidence exists in the paper. Figure 5 provides the per-domain accuracy breakdown, which shows the performance range from 23.9% (multi_apps) to 94.4% (vs_code). Table 3 shows that even with the full ToolCUA pipeline, the hybrid-trained model achieves 42.9% in pure GUI settings—implying that 57% of tasks are beyond the model's capability even in its strongest configuration. The paper does not analyze what types of tasks fall into the "still failing" category (e.g., are they visually complex? Do they require long-horizon planning? Do they involve novel application interfaces?), nor does it discuss whether the remaining failures are addressable through more training or require fundamentally different approaches.

Mitigation status. The paper does not explicitly address the capability ceiling as a limitation. The training paradigm is presented as generally applicable, but the results suggest it is most effective when the base model already has non-trivial competence on the target tasks. The abstract's claim that "training in a hybrid action space is a promising paradigm for real-world digital agents" should be qualified by the observation that this promise is currently demonstrated only on the subset of tasks within an 8B model's reach. The paper does not propose strategies for extending the approach to harder tasks (larger base models, curriculum learning, task decomposition). This limitation is unaddressed and unacknowledged.

6. Online RL Infrastructure Cost Makes the Full Pipeline Inaccessible to Most Practitioners

The assumption or constraint. The online RL stage requires substantial distributed infrastructure: 8 × 8 GPUs for policy training, 4 × 8 GPUs for dedicated inference serving, and approximately 250 independent Docker instances running on distributed ECS servers for sandbox rollouts (Appendix C.3). Each online RL ablation run takes approximately six days with this infrastructure (Appendix C.5). The paper assumes that practitioners interested in training hybrid GUI-Tool agents will have access to comparable compute resources, or that the offline stages alone provide sufficient capability.

The consequence. This creates a severe practical barrier to adoption and replication. Even setting aside the data synthesis pipeline's dependence on proprietary MLLMs (Limitation 1), the online RL stage requires:

  • GPU cluster access: 96 GPUs (8×8 + 4×8) for approximately six days per run. At current cloud GPU pricing, this represents a substantial cost that the paper does not estimate. For context, 96 A100 GPUs for 144 hours at typical cloud rates would cost tens of thousands of dollars per training run.

  • Sandbox orchestration infrastructure: Managing 250 parallel Docker instances running full Linux desktop environments (QEMU-based OSWorld images) is an engineering challenge in itself—sandbox startup time, reliability (QEMU instances can hang or crash), state management, and result collection all require infrastructure that is not described in the paper beyond noting that the verl framework is used.

  • Multi-run requirements: The paper reports at least three online RL runs (full ToolCUA, w/o Interleaved data, w/o Our path reward), plus any hyperparameter tuning runs not reported. The total compute investment for the paper's experiments likely exceeds 3 × 6 × 96 = 1,728 GPU-days, not including the offline training stages or the data synthesis pipeline.

The consequence for the field is that ToolCUA's strongest results cannot be replicated by researchers without access to large-scale industrial compute infrastructure. The offline stages (SFT + single-turn RL on synthesized data) are more accessible, but the offline-only model M_rft achieves only 38.13% accuracy (Table 3)—a substantial improvement over the baseline (28.23%) but far below the full ToolCUA (46.85%). This means the paper's headline result is substantially gated by compute access, limiting its impact to well-resourced industrial labs.

What evidence exists in the paper. Appendix C.3 and C.5 describe the infrastructure requirements in sufficient detail to understand their scale. Table 3 shows the performance gap between the offline-only model (38.13%) and the full ToolCUA (46.85%), quantifying how much of the improvement comes from the compute-intensive online RL stage. The paper does not report the cost of the online RL stage, does not discuss how training cost scales with model size or task diversity, and does not explore whether cheaper alternatives (e.g., shorter online RL, fewer parallel environments, smaller rollout groups) could achieve comparable results.

Mitigation status. The paper does not acknowledge infrastructure cost as a limitation. Appendix A suggests future work on "reducing the dependence of agentic RL on heavy sandbox infrastructure by building lighter, more diverse, and more robust environments with hybrid GUI-Tool actions" and "asynchronous RL frameworks that decouple training and inference-time rollout," but these are future directions rather than current mitigations. The paper provides no cost estimates, no scaling analysis of how performance varies with infrastructure scale, and no guidance for practitioners on how to trade off between infrastructure investment and agent performance. This limitation is acknowledged only indirectly through future work suggestions.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper introduces a fundamental reframing of the computer use agent training problem—not as "teaching agents to use tools" but as "teaching agents to decide when tools are the right choice in a trajectory." The distinction is subtle but consequential: prior work implicitly assumed that tool-calling capability and tool-calling wisdom were the same thing, that if you gave a model tool definitions and a few examples it would naturally learn to invoke them appropriately. ToolCUA demolishes this assumption with systematic evidence (Table 1) showing that models from 8B to 235B parameters, from open-source to proprietary, all suffer from optimal path confusion—some underusing tools to the point of never leaving the GUI branch (Qwen3VL-8B averaging 0.003 tool calls), others overusing tools so aggressively that accuracy degrades (Claude-4.5-Sonnet dropping 13.5 percentage points with tool access). The counterintuitive implication is that exposing models to more capabilities can make them less capable unless the selection policy itself is a first-class training objective.

This is not an incremental improvement in CUA training methodology but a diagnostic reframing that changes what the field should be optimizing. Before ToolCUA, the CUA research agenda was largely: scale up GUI trajectory data (OpenCUA, ScaleCUA, CUA-Suite), improve visual grounding through larger models or better architectures (UI-Tars, GUI-Owl), and apply RL to improve task completion rates (MobileRL, GUI-R1, ARPO). Tools, when considered at all, were treated as an additional action type to be appended to the action space—a straightforward extension of existing paradigms. ToolCUA's core insight is that hybrid action spaces introduce a qualitatively new learning problem that existing approaches don't address: the trajectory-level credit assignment of whether a tool call at step t leads to a globally better outcome than continuing with GUI operations. This problem cannot be solved by scaling up GUI data (which contains no tools), by improving visual grounding (which doesn't distinguish between action types), or by applying RL with task-completion rewards (which doesn't differentiate between a successful short tool path and a successful long GUI workaround).

The paper reconciles the apparent contradiction between tool optimism (API agents can dramatically outperform GUI agents in efficiency, as argued by Zhang et al., 2025; UFO3) and tool pessimism (giving agents tools often hurts performance, as shown in Table 1 across multiple model families). The resolution is that both perspectives are partially correct: tools can provide dramatic efficiency gains, but only when the agent has learned a principled policy for when to invoke them. The contradiction arose because prior work conflated "tool availability" with "tool orchestration capability." ToolCUA demonstrates that these are independent variables—availability is a property of the environment, orchestration is a learned behavior—and that the gap between them is the binding constraint on hybrid agent performance. This has direct implications for how the community should allocate research effort: building more tools or better tool definitions is less leveraged than building better training methodologies for tool orchestration, because the bottleneck is not what tools exist but whether agents can use them discriminatively.

The paper also shifts the narrative around what constitutes "efficient" CUA training. The finding that hybrid GUI-Tool training produces a model that is also better at pure GUI tasks (42.9% for ToolCUA in GUI-only evaluation vs. 42.05% for a GUI-only trained model, Table 3) suggests that the hybrid action space provides a richer training signal even for single-mode deployment. This is a non-obvious result: one might expect that training on interleaved trajectories would dilute the model's GUI grounding ability, since it spends some of its training budget on tool-specific patterns. Instead, the hybrid training appears to produce transferable meta-skills—recognizing task structure, evaluating action efficiency, planning multi-step sequences—that improve GUI-only performance beyond what dedicated GUI training achieves. This implies that hybrid action training is not just a way to enable tool use but a generally superior training paradigm for computer use agents, even when tools are unavailable at deployment time. The practical consequence is that CUA training pipelines should incorporate tool data—whether synthetic or real—regardless of whether the target deployment environment supports tools, because the hybrid training signal produces more capable agents in any action space.

Perhaps the most significant landscape shift is the paper's implication that trajectory-level optimization is the next frontier for CUA research, superseding the current focus on step-level action prediction. The dominant paradigm in end-to-end CUA training has been imitation learning at the action level: given a screenshot and task context, predict the next action. This paradigm achieved substantial progress (ScaleCUA, OpenCUA, UI-Tars) but is fundamentally bounded by the quality of the demonstration data—models learn to reproduce observed actions, not to discover better action sequences that weren't demonstrated. ToolCUA's staged training paradigm (RFT → online RL with Tool-Efficient Path Reward) represents a concrete architecture for moving beyond imitation to trajectory-level optimization, where the model learns through environment interaction that certain action sequences are more efficient or reliable than others, even if those sequences never appeared in the training demonstrations. The R_length reward component is particularly significant in this regard: it provides a signal that is fundamentally unavailable in pure imitation learning, because demonstration data shows one path per task with no contrastive information about whether that path was efficient. By making efficiency a group-relative metric during online RL, ToolCUA creates pressure toward path optimization that no amount of SFT on static data can provide.

This trajectory-level optimization framing has implications beyond the CUA domain. Any agent operating in a heterogeneous action space—web agents choosing between DOM actions and visual clicking, coding agents choosing between file edits and shell commands, robotics agents choosing between fine motor control and high-level skill primitives—faces a version of the same problem: the selection between qualitatively different action modes is a trajectory-level decision that sparse task-completion rewards cannot efficiently teach. ToolCUA's solution—decomposed rewards that separately shape mode-selection and path-efficiency, combined with staged training that first establishes basic multi-mode competence before optimizing global orchestration—provides a template that could generalize across these domains.

Follow-Up Research This Work Enables

Quantifying and reducing the synthesis pipeline's dependence on proprietary MLLMs. The paper's data synthesis pipeline relies on Kimi-K2.5 or Claude-4.5-Sonnet for tool generation, trajectory conversion, and state grounding, with Appendix A noting that substituting Qwen3.5-Plus "led to noticeably lower generation efficiency and trajectory quality." This creates a reproducibility bottleneck: researchers without access to top-tier proprietary models cannot replicate the full pipeline. A concrete follow-up would systematically compare synthesis quality (and downstream agent performance) across a spectrum of synthesis MLLMs—from open-weight models at various scales (Qwen2.5-VL-7B, Qwen2.5-VL-72B, InternVL2-76B) to proprietary models at different capability tiers—and measure the correlation between synthesis model benchmark scores (MMMU, MathVista) and resulting agent accuracy on OSWorld-MCP. The key question is whether there exists an "good enough" threshold below which synthesis quality degrades gracefully rather than catastrophically. If open-weight 72B models produce trajectories that yield agent performance within 5% of proprietary-synthesized trajectories, the pipeline becomes meaningfully reproducible. A negative result—showing that only the strongest proprietary models produce usable trajectories—would reveal a more fundamental limitation of the synthesis paradigm and motivate research into weaker-model-compatible synthesis strategies (iterative refinement, self-consistency checks, human-in-the-loop correction for the hardest cases).

Isolating the contribution of single-turn RL on critical switching steps. The paper's staged training includes a single-turn RL phase targeting 5,000 critical steps from D_critical, but this stage is never ablated independently of the warmup SFT (Section 6, Limitation 3). A clean experiment would train three model variants: (a) SFT only on D_all (180k steps, no critical-step RL), (b) SFT + single-turn RL on D_critical (the full RFT pipeline), and (c) SFT + single-turn RL on an equal-sized random sample of non-critical steps from trajectories (to test whether targeting boundaries matters specifically or whether any additional RL helps equally). All three would then undergo identical online RL with the Tool-Efficient Path Reward, and the final accuracies would reveal: (1) whether single-turn RL provides any benefit beyond SFT alone, (2) whether the benefit is specific to critical switching steps or general to additional RL training. If (a) matches (b), the paper's conceptual emphasis on "critical switching steps" as distinct from other steps would be empirically weakened, and the single-turn RL stage could be dropped to reduce training cost. If (c) underperforms (b), it would validate the claim that boundary decisions are indeed the hardest decisions and deserve targeted optimization. The experiment design is straightforward and requires no new data or infrastructure beyond what the paper already uses.

Tool granularity ablation: does multi-granularity tool diversity matter? The synthesis pipeline deliberately produces fine, mid, and coarse tools (4,350 total, with 2,000 fine, 1,900 mid, 450 coarse per Table 6), and the paper claims this diversity is important for realistic decision complexity. However, no experiment tests whether training with single-granularity tools (e.g., only fine or only coarse) produces different agent behavior. A concrete experiment would train three ToolCUA variants using only fine-grained tools, only coarse-grained tools, and the full multi-granularity library, then evaluate on OSWorld-MCP with metrics that are sensitive to granularity choice: accuracy on tasks where the "correct" granularity is ambiguous (tasks solvable by both a single coarse tool and a sequence of fine tools), TIR on tool-beneficial vs. non-tool-beneficial tasks (does coarse-only training lead to tool overuse because every tool does a lot?), and ACS (does fine-only training produce longer trajectories because the model chains many small tool calls?). A positive result—multi-granularity training outperforms both extremes—would validate the design choice and suggest that tool diversity should be an explicit objective in future synthesis pipelines. A negative result—e.g., coarse-only training matches or exceeds multi-granularity—would simplify future work by removing the need for granularity management.

Reward design sensitivity: how much does the specific functional form of R_length matter? The paper introduces a piecewise path-efficiency reward with a linear bonus for shorter-than-average trajectories and an exponential penalty for longer-than-average ones. This functional form is presented without justification beyond its components (group-relative, success-gated), but the specific mathematical shape could influence what kinds of efficiency improvements the agent discovers. A sensitivity study would compare the paper's formulation against alternatives: (a) a simpler absolute threshold (R_length = 1 if s < threshold, else 0), (b) a continuous linear penalty (R_length = 1 - s/S_max regardless of group average), (c) a percentile-based reward (reward trajectories in the bottom 25% of the group by length), and (d) an inverse-length reward (R_length = 1/s). The key metric would be not just final accuracy but the distribution of trajectory lengths—does the exponential penalty produce a different length distribution (e.g., fewer extremely long trajectories) than a linear penalty? Does the group-relative normalization lead to more stable training across tasks of varying difficulty compared to absolute thresholds? If the specific functional form matters little (all reasonable efficiency rewards produce similar improvements), the finding would be that any length signal helps, simplifying reward design for practitioners. If the form matters a lot, it would motivate theoretical work on optimal reward shaping for trajectory efficiency.

Combining PRM-based search with ToolCUA's hybrid action policy. This direction connects ToolCUA to the test-time compute literature. The paper's approach optimizes the policy through training, but does not explore whether additional test-time computation—specifically, search against a process reward model (PRM) that scores partial trajectories—could further improve GUI-Tool orchestration at deployment. The idea: at each step, rather than greedily selecting the highest-probability action, the agent could sample multiple candidate actions (some GUI, some tool), use a PRM to estimate the value of each partial trajectory, and pursue the most promising branch. This would be particularly valuable at the "forked road" moments (GUI↔Tool boundaries) where the paper shows even trained models can make errors. The PRM could be trained on the same synthesized interleaved trajectories, using Monte Carlo rollout supervision (following the approach in Lightman et al., 2023, or the compute-optimal scaling literature) to provide step-level value estimates. The experiment would compare greedy ToolCUA against ToolCUA + beam search with a fixed inference budget (e.g., best-of-4 or beam width 4 at critical steps), measuring accuracy, tool invocation appropriateness, and trajectory length. A positive result—test-time search further improves orchestration, particularly on medium-difficulty tasks—would suggest that training and inference are complementary levers for hybrid action optimization, analogous to how pretraining and test-time compute are complementary in reasoning domains. A negative result—search doesn't help or actively hurts due to PRM over-optimization (the paper doesn't train a verifier, and over-optimization is a known failure mode)—would reveal a fundamental difference between reasoning tasks and computer-use tasks: in CUAs, the state is visual and tool results are structured, so a PRM trained on offline data may not transfer well to online states, making test-time search less reliable than in text-only reasoning.

Stress-testing cross-platform generalization: does the orchestration policy transfer, or just the GUI grounding? The paper reports that ToolCUA achieves 33.8% on WindowsAgentArena vs. the baseline's 26.4%, but cannot isolate whether this gain comes from learned tool-switching policies or simply from improved GUI grounding due to additional training data. A clean stress-test would evaluate ToolCUA on WindowsAgentArena in two configurations: with tools available (the MCP tools that exist for Windows applications, if any) and with tools entirely disabled (pure GUI mode). If the tool-disabled performance still shows gains over the baseline, the improvement is primarily from better GUI grounding. If the gain appears only or mostly in the tool-enabled setting, it reflects transferred orchestration capability. A stronger test would train a ToolCUA variant where all tool-specific training data is replaced with pure GUI data of equivalent scale, then compare its cross-platform transfer to the full ToolCUA. If both transfer equally well, the orchestration policy doesn't generalize across platforms—the cross-platform gain is just more GUI data. If ToolCUA transfers better, it suggests the model learned platform-agnostic heuristics for when tools are likely to help (e.g., structured data operations, file management) that apply even on unseen operating systems. This experiment would clarify whether the paper's claim of "generalizable automation" is specifically about tool orchestration or more broadly about training data diversity.

Practical Applications and Downstream Use Cases

Cost-efficient batch desktop automation for enterprise workflows. Organizations running large-scale repetitive desktop tasks—formatting spreadsheets, updating presentation decks, configuring software settings across many machines—currently rely on brittle GUI automation scripts (AutoHotkey, Selenium-based desktop automation) or manual human effort. ToolCUA's 46.85% accuracy on OSWorld-MCP with an average of only 14.93 steps per task (Table 2) suggests a deployment architecture where an 8B-parameter model, trained with the ToolCUA pipeline on company-specific desktop workflows, could automate a substantial fraction of routine tasks while requiring minimal steps. The key economic metric is not just accuracy but task completion time × cost per inference. ToolCUA's low ACS implies faster task completion: at 14.93 steps with each step involving one screenshot analysis and action generation (perhaps 2-3 seconds of inference on commodity hardware with an 8B model), total task time might be 30-45 seconds, compared to several minutes for human execution or brittle script debugging. For organizations processing thousands of such tasks daily, the 23% step reduction over the baseline (19.34 → 14.93) translates directly to throughput and cost savings. The trained model could be deployed on-premises, avoiding the data privacy concerns of sending desktop screenshots to cloud APIs—an important consideration for enterprise adoption.

Data generation for self-improving CUA training loops. A bottleneck in CUA research is the acquisition of high-quality training trajectories: human demonstrations are expensive, and scripted trajectories lack diversity. ToolCUA's offline + online training paradigm suggests a self-improvement loop: (1) train a ToolCUA agent on existing data, (2) deploy the agent to collect new trajectories on a broader set of tasks, (3) use the successful trajectories (filtered by the environment evaluator) as additional training data for the next iteration, potentially applying the synthesis pipeline to extract new tools from these trajectories. The paper's finding that the offline RFT model alone reaches 38.13% (Table 3) while online RL pushes to 46.85% suggests there is substantial headroom for iterative improvement: better training data (from a stronger agent) would improve the RFT initialization, which would improve the online RL starting point, which would produce an even stronger agent for the next data collection round. This is directly analogous to the STaR/ReST^EM self-improvement paradigm (Zelikman et al., 2022; Singh et al., 2024) but applied to computer use rather than reasoning, and with the added dimension of tool synthesis—each iteration could synthesize new tools from newly discovered efficient action sequences, gradually expanding the tool library without manual engineering. The key risk (flagged by the paper's Appendix K finding that ReST^EM-based optimization degraded revision model performance) is that on-policy data collection can amplify spurious correlations, so trajectory filtering and data quality checks would be critical.

On-device or edge deployment of lightweight automation agents. ToolCUA's core result—an 8B model achieving accuracy competitive with 32B+ models (46.85% vs. EvoCUA-32B's 40.54%, Table 2) through trained orchestration rather than scale—has direct implications for on-device deployment. An 8B vision-language model can run on consumer GPUs (RTX 4090, M2 Ultra) or even quantized on high-end mobile devices, unlike 70B+ models that require datacenter infrastructure. The paper's finding that hybrid training improves pure-GUI performance (42.9% vs. 42.05% for GUI-only training, Table 3) means a single on-device model could handle both tool-enabled tasks (when MCP servers or local APIs are available) and pure GUI tasks (when they're not) without performance degradation in either mode. The practical deployment scenario: a user's laptop runs a quantized ToolCUA-8B locally; the model automates file management, email sorting, and document formatting using available tool APIs when connected to the appropriate MCP servers, and falls back to GUI operations for applications without tool support—all without sending screenshots to the cloud. The 0.74 average tool calls per trajectory (Table 1) means the model is conservative about tool invocation, reducing the risk of inappropriate automated actions that could modify user files.

When to Prefer This Method

The paper positions ToolCUA's staged training paradigm against two alternatives implicitly, based on the experimental comparisons: (1) pure GUI training without tool orchestration (Table 3), and (2) giving models tool access without hybrid-action-specific training (the baseline and general model rows in Table 2). From these comparisons, a decision rule emerges:

  • Prefer ToolCUA's hybrid GUI-Tool staged training when the deployment environment has a heterogeneous action space where high-level tools can replace sequences of low-level GUI operations on at least a subset of tasks, and the base model has non-trivial GUI grounding ability (29%+ accuracy on the target domain in pure GUI settings, as Qwen3-VL-8B-Instruct had on OSWorld). Under these conditions, ToolCUA's pipeline delivers approximately 66% relative improvement over the baseline (28.23% → 46.85%, Table 2) and 4.8 percentage points absolute improvement over pure GUI training at equivalent scale (42.05% → 46.85%, Table 3).

  • Prefer pure GUI training (GUI SFT + GUI agentic RL) when tools are unavailable or unreliable in the target deployment environment, or when the cost of synthesizing interleaved training data (proprietary MLLM API calls, sandbox infrastructure) outweighs the expected accuracy gain. The paper shows that GUI-only training with comparable compute reaches 42.05% (Table 3), which may be sufficient for applications where tools don't exist or where maintaining MCP server compatibility across application versions is impractical.

  • Do not expect "just add tools to the prompt" to work, regardless of model scale. Table 1 shows that models from 8B (Qwen3VL-8B, −0.8% with tools) to Claude-4.5-Sonnet (−13.5%) degrade when given tool access without hybrid-action-specific training. ToolCUA is the only model in Table 1 that improves with tool access (+3.9%), and this improvement requires the full staged training pipeline—skipping the offline interleaved data bootstrapping (Figure 6, "w/o Interleaved data") results in near-zero tool usage even with online RL.

The paper does not articulate a clear tradeoff between ToolCUA and alternative hybrid-action training methods (UltraCUA, Step-GUI, OS-Symphony's tool routing), because none of these are evaluated as baselines on OSWorld-MCP. The comparison is against general models and pure-GUI CUAs, not against other hybrid training paradigms. A practitioner choosing between ToolCUA and a multi-agent orchestration approach (where separate modules handle GUI and tool decisions) would need to weigh ToolCUA's end-to-end simplicity (one model, one training pipeline) against multi-agent systems' modularity and debuggability, but the paper provides no evidence for this tradeoff.