CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agents

ArXiv: 2605.25624

🎯 Pitch

Training data scarcity for computer-use agents is not a collection problem but a structural one—every RLVR task needs a matching environment state and a deterministic reward function that can be checked programmatically. The authors solve this with an agentic pipeline that generates all three jointly, then trains a 3B model to outperform its untrained 17B counterpart while discovering that environment diversity is a scaling axis as important as data volume.

1. Executive Summary

This paper introduces CUA-GYM, an agentic pipeline that co-generates task instructions, executable environment states, and programmatic reward functions for reinforcement learning with verifiable rewards (RLVR) training of computer-use agents, resolving the structural bottleneck that has kept CUA RLVR data orders of magnitude smaller than its counterparts in math and code. Instantiating the pipeline on a pool of 110 environments—combining 16 desktop applications with 94 synthesized mock web applications from the companion CUA-GYM-HUB suite—produces 32,112 verified training tuples, and training Qwen3.5-35B-A3B and Qwen3.5-397B-A17B with GSPO on this data yields 62.1% and 72.6% on OSWorld-Verified, with the smaller model matching the untrained A17B base at roughly 10× fewer total parameters. Performance scales smoothly with both data volume and environment diversity, with the latter identified as a complementary scaling axis, establishing that verified task and environment scale were a substantive bottleneck for current GUI agents.

2. Context and Motivation

The Core Problem: RLVR's Missing Domain — Computer-Use Agents

The paper addresses a specific, structural gap in the post-training landscape. Reinforcement learning with verifiable rewards (RLVR) has become the dominant paradigm for improving foundation models after pretraining, with a now-standard recipe: procedurally synthesize thousands of training tasks, each paired with a deterministic reward signal, then optimize a policy against these rewards using algorithms like GRPO or GSPO. This recipe has driven state-of-the-art results in mathematical reasoning (DeepSeek-R1, DeepSeekMath), software engineering (SWE-smith, SWE-Gym, SWE-Universe), and terminal operation (Endless Terminals, Terminal-Task-Gen).

Yet computer-use agents (CUAs)—models that click, type, scroll, and navigate through desktop and web interfaces to accomplish user goals—have not benefited from this recipe at comparable scale. The paper quantifies this gap in Table 6c (Section 4): existing CUA RLVR datasets are 1–2 orders of magnitude smaller than their counterparts in code and math, and those that achieve scale typically sacrifice reward verifiability by using VLM-as-a-judge scoring.

This is not because CUAs are less important. The paper argues that the potential impact is arguably larger: CUAs can automate the vast landscape of digital knowledge work that currently consumes billions of hours of human attention annually. Unlike math or code, where the task space is relatively constrained to formal symbolic manipulation, GUI interaction spans every software application ever built. The economic and productivity implications of capable CUAs are commensurately broader.

The Structural Bottleneck: Why CUA RLVR Data Doesn't Scale

The gap is not algorithmic—the same GRPO/GSPO algorithms that work in math and code should, in principle, work for CUAs—but structural. A CUA RLVR training instance is a 3-tuple (t, s, r):

• t: a natural-language task instruction (e.g., "Send an email to each client listed in the Notion database using the Gmail template").
• s: an executable, reproducible initial environment state—the desktop or browser state the agent observes when it begins.
• r: a programmatic reward function that deterministically scores the agent's final state as a value in [0, 1], evaluating whether the task was completed.

Each of these three components is a non-trivial engineering artifact that must be internally consistent with the other two. The environment state must contain exactly the files, application windows, and data fixtures needed to make the task possible and unambiguous. The reward function must check exactly the right properties—the final file contents, UI element states, or database entries that indicate task completion—without rewarding spurious correlations or penalizing valid alternative approaches.

In math, a training instance reduces to a problem statement and a checkable answer. In code, it reduces to a buggy codebase and a unit test. But in GUI interaction, hand-authoring a single tuple takes hours of expert effort, because the author must:

1. Set up a specific application state (create the right spreadsheets with the right data, populate email inboxes with the right messages, configure the right system settings).
2. Write a reward function that programmatically inspects the post-task state—parsing document formats, reading filesystem hierarchies, checking database entries, or introspecting application-specific data structures.
3. Ensure the reward function correctly distinguishes the intended outcome from plausible near-misses, assigns partial credit where appropriate, and does not collapse to all-or-nothing.

The cost compounds with application diversity, since each new application (Slack, Jira, Salesforce, LibreOffice Calc, GIMP, VS Code) brings its own file formats, APIs, state representations, and verification primitives. As the authors state in Section 1:

"Hand-authoring a single such tuple takes hours of expert effort, and the cost compounds with application diversity since each new application brings its own setup procedures and verification interfaces."

This explains why CUA RLVR datasets have remained at the scale of hundreds or low thousands of tasks—three orders of magnitude behind the tens to hundreds of thousands of tasks available in math and code.

Why This Gap Matters: Scaling Curves Across Domains

The paper situates this gap within a broader empirical pattern that has emerged across digital agent domains. The introduction (Section 1) and related work (Section 5) cite converging evidence that RLVR data-performance scaling curves remain unsaturated at current volumes, meaning additional verified tasks continue to yield proportional performance improvements:

• SWE-smith (Yang et al., 2025) demonstrates log-linear performance scaling with synthesized task volume that remains unsaturated at tens of thousands of tasks.
• SWE-Gym (Pan et al., 2025) scales the paradigm to thousands of repositories and reports the same pattern.
• Endless Terminals (Gandhi et al., 2026) observes identical scaling behavior on terminal operation tasks.

The consistency of this pattern across heterogeneous domains suggests a general principle: RLVR data synthesis is not yet a saturating regime, and the ceiling on agent capability is gated more by the supply of verified training data than by algorithmic innovation. Under this view, the scarcity of CUA RLVR data is not a minor inconvenience—it is the primary bottleneck preventing CUAs from following the same capability trajectory as math and code agents.

The paper's central motivation is therefore to build the data-generation infrastructure that removes this bottleneck. CUA-GYM is not primarily a new training method or a new model architecture; it is a data engineering contribution—a pipeline that automates the engineering work that has previously required human experts, enabling verified CUA training tuples to scale by orders of magnitude.

Prior Approaches and Their Shortcomings

The paper identifies three existing approaches to generating training data for GUI agents, none of which simultaneously satisfy the three requirements of scalable CUA RLVR: deterministic verifiable rewards, broad application coverage, and scalable task diversity.

Approach 1: Supervised Datasets (Coverage Without Rewards)

Several large-scale datasets provide trajectory-level demonstrations for GUI tasks (OpenCUA, AGUVIS, EvoCUA, AgentTrek). These datasets cover diverse applications and provide rich imitation targets—sequences of screenshots paired with correct actions. However, they provide trajectory-level supervision rather than outcome rewards. An RLVR policy needs a scalar reward signal that measures whether the task was completed, not just a demonstration of how to complete it. Supervised trajectories can be used for SFT warm-up (as CUA-GYM does with its 3,578 curated teacher demonstrations in Section 3.1), but they cannot drive the RL optimization loop, which is where the largest capability gains have been observed in math and code.

Approach 2: VLM-as-a-Judge (Scale Without Reliability)

To achieve broad application coverage without per-application instrumentation, several works (ZeroGUI, Gym-Anything) propose using vision-language models as reward models: show the VLM a screenshot of the final state, ask it to judge whether the task was completed, and use that judgment as the reward signal. This approaches the coverage of supervised datasets and the scalability of automated evaluation, but introduces a fatal problem for RL: reward noise destabilizes policy optimization. The paper explicitly cites ZeroGUI's own ablation: "ZeroGUI itself confirms through ablation that the resulting false-positive rewards destabilize RL training" (Section 5). When the reward signal is unreliable—sometimes rewarding incorrect completions, sometimes penalizing correct ones—the RL optimization landscape becomes a noisy, shifting surface where the policy cannot converge to a stable solution.

This is the fundamental tension the paper identifies: VLM judges trade off verifiability for coverage, but RLVR training requires both. The CUA-GYM pipeline resolves this tension by generating programmatic rewards automatically, achieving the coverage and scale of automated synthesis while retaining the deterministic reliability of hand-coded verifiers.

Approach 3: Code-Native Browser Environments (Verifiability Without Coverage)

A third line of work synthesizes controllable browser environments with embedded programmatic reward oracles. GUI-Genesis (Cao et al., 2026) generates per-task Flask applications with built-in verification. InfiniteWeb (Zhang et al., 2026) uses task-centric test-driven development to produce browser environments with code-native rewards. AutoWebWorld (Wu et al., 2026) models web applications as finite-state machines and generates verifiable task instances from state-transition models. These approaches achieve the reward reliability that VLM judges lack, but at a steep cost: they are confined to browser-based interactions. They cannot cover:

• OS-level desktop tasks (file management, system configuration, application launching).
• Desktop application workflows (LibreOffice Calc spreadsheets, VS Code editor configurations, GIMP image editing, Thunderbird email management, VLC media playback).
• Cross-application workflows that span both desktop and web applications (e.g., extracting data from a Gmail mock into a LibreOffice Calc spreadsheet, then generating a PDF report).

Real-world knowledge work routinely involves all three—a fact the paper grounds in external data sources: O*NET occupational taxonomies (U.S. Department of Labor) show that administrative, managerial, and technical occupations extensively use both desktop productivity suites and web-based tools, and the Anthropic Economic Index shows that web-native and desktop-native software each account for substantial fractions of actual worker hours. A browser-only training pipeline necessarily produces agents that cannot handle the full spectrum of computer-use tasks.

Additionally, even within the browser domain, code-native approaches have so far produced environments at small scale (GUI-Genesis: 969 tasks in 1 environment; InfiniteWeb: 600 tasks). The paper's contribution—110 environments across desktop and web, 32K verified tuples—represents a step change in scale for programmatically-verified CUA training data.

Approach 4: Hybrid but Narrow (Recent Attempts to Bridge Desktop and Verifiability)

The paper acknowledges recent work that attempts to bridge the gap between desktop coverage and programmatic verification. UltraCUA (Yang et al., 2025) produces 17,000 desktop tasks with programmatic rewards but covers only 9 desktop applications—a narrow slice of the software landscape. SeAgent (Sun et al., 2025) and AgentTrek (Xu et al., 2025) achieve broader coverage but fall back to checklist-based or VLM-based evaluation, reintroducing the reliability problem. The paper's position is that none of these approaches simultaneously achieves all three requirements: they either trade off coverage, reward reliability, or scale.

The Environment as a Missing Scaling Axis

Beyond per-task data synthesis, the paper identifies a second, subtler bottleneck: existing benchmarks and training environments cover only a narrow slice of the software landscape that knowledge workers actually use. The paper analyzes this through two external grounding sources (Section 4):

1. O*NET SOC Major Groups: The Standard Occupational Classification system maintained by the U.S. Department of Labor defines 23 major occupational groups. Knowledge-work-heavy groups (Management, Business & Financial Operations, Computer & Mathematical, Sales, Office & Administrative Support) each involves distinct clusters of software applications—CRM systems for sales, project management tools for operations, code hosting and CI/CD platforms for software engineers, document collaboration suites for administrative staff.

2. Anthropic Economic Index: This dataset provides empirical software-usage frequencies estimated from large-scale agent-traffic logs. It reveals that application usage is highly skewed: a small number of applications (Slack, Gmail, Google Docs, Jira, GitHub) account for a large fraction of total interaction time, but a long tail of domain-specific tools (Salesforce, Stripe Dashboard, AWS Console, Canvas LMS, Epic Health) are essential for specific occupational categories.

The implication is that training CUAs on a narrow set of applications produces agents that overfit to those specific interfaces and cannot generalize to the long tail. However, real websites and applications are fundamentally unsuitable as RL training environments because they have authentication gates, rate limits, deterministic state is impossible to inject or reset programmatically, and third-party integrations introduce non-reproducible behaviors. The paper therefore argues that environment synthesis itself is a necessary scaling investment: the CUA-GYM-HUB mock web application suite is not a byproduct but a first-class contribution, designed to make environment coverage a reusable substrate that supports many downstream training recipes beyond the particular CUA-GYM task corpus.

How This Paper Positions Itself

The paper frames its contribution as infrastructure rather than algorithmic innovation. The introduction (Section 1) and conclusion (Section 7) are explicit about this:

"We read these results as evidence that task and reward diversity were a substantive bottleneck for current GUI agents rather than as claims about the pipeline's ultimate ceiling."

The contribution is not a better RL algorithm (the paper uses off-the-shelf GSPO) or a better base model (it uses existing Qwen3.5 checkpoints) or a better agent architecture (it uses standard screenshot-to-action tool-calling). The contribution is the data pipeline that makes RLVR training possible for CUAs at scale for the first time.

This positioning is important because it implies a different standard of evaluation. A methods paper would be evaluated on whether it proposes a novel technique that outperforms baselines. A data-engineering paper is evaluated on whether the pipeline produces data that drives downstream performance, and whether the pipeline is reproducible and reusable by the community. The paper's claims—$4\times$ efficiency gains from data scaling, identification of environment diversity as a complementary scaling axis, 62–72% on OSWorld-Verified—are all claims about the value of the generated data, not about any specific algorithmic innovation within the pipeline.

The paper also explicitly connects to the broader trend of synthesizing training data from executable environments (Section 5, Related Work), positioning CUA-GYM as the GUI-domain analog of SWE-smith, SWE-Gym, and Endless Terminals. The consistency of the scaling behavior across domains—log-linear improvement with task volume, unsaturated at current scales—is cited as evidence that the approach is sound and that the gap the paper fills is real and consequential.

3. Technical Approach

3.1 Reader Orientation

CUA-GYM is an agentic pipeline—a system of coordinated, specialized LLM agents—that automatically produces complete, verified training instances for reinforcement learning of computer-use agents, where each instance contains a task instruction, an executable environment setup, and a programmatic reward function that deterministically scores task completion. The system solves the structural bottleneck that has kept CUA RLVR data scarce by delegating the expert engineering work of writing environment setup scripts and reward verification code to coding agents, then enforcing correctness through an adversarial verification loop and a multi-stage filtering process, enabling the generation of verified training tuples at scale without human intervention beyond pipeline design and prompt engineering.

3.2 Big-Picture Architecture (Diagram in Words)

The pipeline operates at two levels: per-task co-generation (producing individual (task, setup, reward) tuples) and environment provisioning (providing the diverse virtual desktops and mock web applications that tasks execute within).

At the per-task level, five components interact sequentially and iteratively:

1. Task Generator: Given a topic specification and coverage targets, synthesizes a natural-language task instruction paired with a detailed context describing the exact entities, files, and states that must exist in the environment. This is a one-shot generation pass, not part of the adversarial loop.

2. Orchestrator: The central coordinator. For each task, it provisions two isolated virtual machines (one for the initial pre-task state, one for the golden post-task state), spawns the Generator and Discriminator subagents, monitors their outputs against five agreement conditions, manages iterative rounds of revision, and routes accepted tuples to the dataset-level filter.

3. Generator Subagent: Reads the task instruction and context, then produces two Python scripts: initial_setup.py (executed on the initial VM to create the pre-task world state) and golden_patch.py (executed on the golden VM to create the expected post-task world state). It operates under an information barrier: it cannot see the Discriminator's reward script.

4. Discriminator Subagent: Reads only the task instruction and context (not the Generator's setup scripts), inspects the initial and golden VM states through a restricted API, and writes reward.py—a programmatic reward function that produces a score in [0, 1] by verifying properties of the final state. It also produces a structured review verdict (REVIEW.md) recording whether the five agreement conditions are met.

5. Dataset-Level Filter: After the adversarial loop accepts a tuple, two independent stages—LLM majority voting across multiple critic models, followed by teacher-model rollouts with VLM-as-a-judge verification—catch failures that the per-task loop cannot detect (ambiguous instructions, infeasible tasks, trivial tasks, reward hacking that passes per-task checks but fails under realistic agent behavior).

At the environment level, a separate multi-agent pipeline (the CUA-GYM-HUB synthesis pipeline) produces self-contained mock web applications:

• Plan Agent: Researches the target real-world application, produces design specifications and feature inventories.
• Dev Agent: Implements a single-page application matching the specification.
• Web Agent: Exercises every interactive element via Playwright, comparing live behavior against the spec and feeding discrepancies back to the Dev agent.

The two levels connect through a unified state API: every mock application exposes endpoints for state injection (/post), state inspection (/go and /state), and file upload (/upload), scoped by session identifier. This API is what the Generator uses to materialize task-specific initial states in web environments, and what reward.py calls to inspect the final state for verification.

3.3 Roadmap for the Deep Dive

• First: The task generation subsystem—how task instructions and their grounding contexts are synthesized, including the feature taxonomy tree, the five-dimensional sampling matrix, and the three-pass generation strategy that enforces coverage constraints. This establishes what the rest of the pipeline operates on.

• Second: The adversarial Generator-Discriminator loop—the core mechanism that produces internally consistent (task, setup, reward) tuples. This covers the five agreement conditions, the information barrier and access matrix, the forbidden-pattern static scan, and the feedback protocol that drives iterative convergence across up to five rounds.

• Third: The dataset-level filter (LLM majority voting and teacher-model rollouts)—the quality-control stage that catches failures invisible at the per-task level. Understanding this is essential because the filter is what closes the gap between "the reward distinguishes initial from golden" and "the tuple is useful for RL training."

• Fourth: The CUA-GYM-HUB environment synthesis pipeline—the Plan-Dev-Web agent loop that produces the 94 mock web applications, including the unified state API specification, the session-isolation mechanism, and the fidelity-vs-practicality tradeoffs built into the mock design. This section establishes why environments are a first-class scaling axis, not merely a substrate for task generation.

• Fifth: The integration of both levels into the complete synthesis pipeline, covering the per-task output bundle format, the OSWorld evaluator compatibility contract, and the domain skill files (SKILL.md) that encode application-specific knowledge for both Generator and Discriminator.

• Sixth: A walkthrough of the pipeline's operational parameters—loop budgets, filter thresholds, anti-repetition rules, per-stage yield statistics—that make the abstract architecture concrete and reproducible.

3.4 Detailed, Sentence-Based Technical Breakdown

This is a data-engineering paper whose core idea is that automated coding agents, when structured as adversarially coupled sub-systems with an information barrier and iterative verification, can replace human experts in constructing the three interdependent artifacts—task instruction, environment setup, reward function—that constitute a CUA RLVR training instance. The paper is not proposing a new RL algorithm or model architecture; it is building the data-generation infrastructure that makes RLVR training possible for computer-use agents at the 32K-tuple scale.

Task Generation: Producing Instruction-Context Pairs

The pipeline's first stage synthesizes task instructions that are simultaneously natural, specific, grounded in executable environments, and diverse across multiple coverage dimensions. A task is not merely a sentence like "Format the spreadsheet"—that would be ambiguous, since the agent needs to know which spreadsheet, which cells, with what formatting, and the reward function needs to know what to check. CUA-GYM resolves this by generating each task as an instruction-context pair (t, c), where c enumerates the entities, files, and application states that must exist in the initial environment.

Feature Taxonomy Tree Construction

For each application in the environment pool (both desktop applications and mock web applications), the Task Generator constructs a feature taxonomy tree—a hierarchical decomposition of the application's capabilities into atomic, groundable UI operations. The tree is bootstrapped from two sources: official application documentation (manuals, API references, tutorial pages) and an LLM-driven web research pass that crawls community forums, user guides, and real-world usage examples to ground the inventory in how people actually use the software.

The tree is constrained to depth ≤ 4 with branching factor ≤ 12 at any internal node. For example, libreoffice_calc > formatting > cell_appearance > number_format is a path of depth 4, where the leaf node number_format represents the atomic capability "format a cell or range with a specific number format pattern." The constraint on depth and branching factor balances coverage against sampler tractability: a tree that is too deep or too wide would make it difficult to guarantee that every leaf receives sufficient task coverage in the final dataset.

Each leaf node is annotated with three pieces of metadata:

1. A one-line capability description (e.g., "Format column D as currency with the EUR symbol, two decimals, and a thousands separator")—this is what the instruction generator draws from when composing task instructions.
2. A list of UI elements involved (menus, dialogs, ribbon tabs, keyboard shortcuts, drag handles)—this ensures that tasks sampled from different leaves exercise genuinely different parts of the UI rather than surface-level variations on the same action.
3. Difficulty indicators—leaves are tagged with whether their typical use involves single steps, multi-step sequences, or conditional branching, which feeds into the difficulty labeling scheme described below.

The paper provides a concrete worked example in Appendix A.2.6: the libreoffice_calc tree contains 147 leaves spread across 9 top-level subtrees (formatting, data_import_export, formulas, charts, pivot_tables, sheet_structure, protection, references, scripting), with the formatting subtree alone accounting for ~21 leaves covering cell appearance (number formats, date/time formats, conditional formatting, borders and fills, alignment, typography) and styles/themes (named styles, table styles).

Five-Dimensional Sampling Matrix and Coverage Constraints

Tasks are sampled across a five-dimensional matrix that ensures the final dataset is balanced along axes that matter for generalization:

The sampler maintains running per-cell counts and applies inverse-frequency weighting: cells with fewer completed tasks receive higher sampling probability in subsequent draws. This biases generation toward under-covered combinations and prevents the sampler from over-producing easy single-application spreadsheet tasks while starving hard cross-application web-desktop combinations.

Additionally, three hard coverage constraints are enforced:

• No domain category exceeds 21% of the final dataset. This prevents any single application paradigm (e.g., spreadsheets) from dominating.
• Hard tasks comprise at least 40% of the corpus. This skews the distribution toward the regime where RL training has the most room for improvement (easy tasks are already close to saturation).
• Cross-application tasks comprise at least 35% of the corpus. This ensures the agent encounters multi-app coordination scenarios frequently enough to learn the corresponding skills.

Three-Pass Generation Strategy: Breadth, Gap-Fill, Edge Cases

Task synthesis proceeds in three sequential passes over the matrix, each with a distinct objective:

Pass 1 (Breadth) samples uniformly across the feature taxonomy to maximize leaf-level coverage. The output volume is calibrated so that every leaf node receives at least 25 task instructions. This ensures that no feature of any application is completely absent from the training data—the agent will encounter at least some training signal for every UI capability.

Pass 2 (Gap-Fill) re-samples under-represented matrix cells identified by post-hoc coverage analysis on the Pass-1 output. This is where the cross-application and hard-difficulty distributions are deliberately upweighted to meet the quantitative targets (≥ 40% hard, ≥ 35% cross-app). The gap-fill pass does not introduce new leaves; it increases the density of tasks in existing leaves that are under-represented relative to the target distribution.

Pass 3 (Edge Cases) targets boundary conditions identified during the loop-failure analysis of the subsequent pipeline stages. Specifically, it samples:

• Tasks at the limit of single-shot solvability (where the agent must coordinate many sub-steps but the task is still completable within the step budget).
• Tasks that combine multiple applications in unusual orderings (e.g., starting in a web app, then spawning a desktop tool, then returning to the web app).
• Tasks that stress-test rare action primitives (drag-and-drop, hotkey chords, multi-window navigation, context-menu interactions).

This pass is empirically motivated: early pipeline runs revealed that certain task categories (particularly cross-application workflows with non-standard action sequences) had high loop-failure rates because the Discriminator could not write robust rewards for them. Adding these edge cases to the generation pass forces the pipeline to handle them, improving the robustness of both the Generator and Discriminator over time.

Anti-Repetition Rules and Instruction Quality

Within and across passes, generated instructions are de-duplicated via three independent checks:

1. Sentence-embedding cosine similarity: Instructions whose embeddings (computed via a sentence-transformer model) have cosine similarity ≥ 0.85 with any prior instruction in the corpus are flagged as near-duplicates and rejected.

2. Token-level 4-gram overlap: Instructions sharing more than 50% of their 4-grams with any prior task are rejected, regardless of embedding similarity. This catches template-level repetition that embedding models might miss (e.g., "Apply X formatting to column Y" with different X and Y values would have moderate embedding similarity but near-identical 4-gram overlap).

3. Slot-template diversity rule: Any single instruction template (identified by pattern matching on the instruction structure) is limited to at most 3 instantiations per application. For example, the template pattern "Apply <formatting_style> to column <column_letter>" can appear at most 3 times for libreoffice_calc—after that, the Generator is prompted with a "diversify" instruction that forces it to vary the surface form.

Tasks failing any of these checks are returned to the Generator for re-synthesis under a structured diversification prompt.

Instruction Length and Verb Distribution

The paper reports distributional statistics on the resulting instructions (Appendix F.1.5): mean length 54.1 words, median 41, with 90% of instructions fitting under 108 words. Instructions under 30 words are typically single-step actions on familiar artifacts; instructions of 100+ words encode multi-stage workflows with explicit acceptance criteria. The top imperative verbs reflect the action vocabulary the agent must learn: open (4,585 tasks), create (2,728), set (1,448), add (1,028), extract (891), configure (728), apply (605), insert (562), build (440), read (427), change (361), write (290), convert (265), remove (227), export (183), check (170), find (139).

The Adversarial Generator-Discriminator Loop

This is the core mechanism that produces internally consistent (task, setup, reward) tuples. The design principle is adversarial coupling with information isolation: if the same agent writes both the environment setup and the reward function, the reward tends to re-check the construction procedure (e.g., verifying that a specific function was called, a specific file path was written) rather than measuring genuine task completion from the agent's perspective. This produces tuples that are trivially solvable by an RL policy that learns to trigger the right internal state transitions rather than completing the intended task—reward hacking at the data-generation level.

The solution is to split the two roles between separate subagents separated by a strict information barrier, so that the Discriminator can only draw on the task semantics themselves when designing the reward.

The Five Agreement Conditions

A tuple (t, s_init, s_gold, r) is accepted by the loop if and only if all five of the following conditions hold simultaneously:

• C1 (Initial-state executability): initial_setup.py runs to completion on the initial virtual machine V_init without raising any Python exception. This ensures the environment setup is well-formed and produces a launchable desktop or browser state.

• C2 (Golden-state executability): golden_patch.py runs to completion on the golden virtual machine V_gold without exception. Note that golden_patch.py is executed on a separate VM from initial_setup.py—the golden VM starts from the same base snapshot and golden_patch.py constructs the post-task state independently, without access to the initial VM's files.

• C3 (Golden-state reward): Evaluating reward.py on the golden VM state yields a score of exactly 1.0. This is the positive control: the reward function must recognize the intended correct outcome as perfect completion.

• C4 (Initial-state reward): Evaluating reward.py on the initial VM state yields a score of 0.0. This is the negative control: the reward function must not accidentally award credit for properties that already exist in the pre-task state. If reward(V_init) > 0, the reward is measuring pre-existing state rather than task completion, and the tuple cannot provide useful RL signal.

• C5 (Reward integrity): reward.py contains no member of a curated forbidden-pattern list (detailed below). This is a static analysis check that catches common reward-hacking antecedents at write time before execution.

The conjunction of C1–C5 ensures the tuple is loop-level consistent: the environments are buildable, the reward is sensitive in the correct direction at both endpoints, and the reward's code does not contain known cheating patterns. Loop-level consistency is necessary but not sufficient for training utility—that is what the dataset-level filter addresses (Section 3.4, Filter section below).

The Information Barrier: Process-Level Isolation

The information barrier is implemented as process-level isolation between the Generator and Discriminator subagents. Each subagent executes as a separate LLM inference process with a disjoint working directory and file-system view. The following access matrix is enforced (Table 1 in the paper):

The critical constraint is the row for initial_setup.py and golden_patch.py: the Discriminator has no read access to these scripts. This means the Discriminator cannot reverse-engineer the setup procedure by inspecting how the environments were constructed. Its only task-specific signal is the natural-language instruction-context pair (t, c) and a read-only view of the two VMs through a restricted state API. The state API exposes file contents, process listings, and window titles but does not expose execution traces, script source code, or the file-system paths of the setup scripts.

The reason this barrier is "load-bearing" (as the paper's Orchestrator prompt explicitly states) is that without it, the Discriminator's most efficient strategy is to read golden_patch.py, identify what it changed, and write a reward that checks exactly those changes. This produces a reward that is perfectly correlated with the golden patch but completely opaque to the actual task semantics—an RL agent trained on such tuples would learn to trigger the specific state transitions that the golden patch encodes rather than learning general computer-use skills.

The Forbidden-Pattern Static Scan (Anti-Hacking)

The Discriminator-emitted reward.py is scanned at write time against a curated list of forbidden patterns. Any match aborts the round and triggers a structured re-prompt to the Discriminator with the matched pattern as feedback. The pattern list targets specific reward-hacking antecedents observed during pipeline development:

1. Direct Boolean assignment to a verification flag without computation: e.g., chart_verified = True. This pattern assigns credit unconditionally, without any environment inspection.

2. Placeholder verification: a flag is assigned a constant value before being conditionally added to the score, with no intervening evaluation that depends on the environment.

3. Hardcoded success: a function returns a constant in {0.5, 1.0} along the success path with no inspection of the environment state.

4. Bare existence scoring: a positive score is awarded purely on os.path.exists(...) without any check on the file's contents, size, or metadata. An agent could satisfy this by creating an empty file at the correct path.

5. Subprocess usage: the reward shells out to external processes via subprocess.run or os.system. External processes are non-reproducible (they depend on system state that varies across VM instances) and easily spoofed (an agent could modify the process command to always return success).

6. Comment-only verification: a score increment is preceded by a comment asserting a check (e.g., # assume X is correct) without code performing the check.

The scan is implemented as a combination of regex matching and Python AST traversal. The paper includes verbatim examples of rejected patterns in Appendix E.3, showing two concrete instances: one where a Discriminator writes if chart_verified: score += 0.5 with chart_verified assigned to True without inspection, and another where a reward awards points on os.path.exists('/home/user/report.xlsx') without checking the file's contents.

Feedback Protocol and Iterative Convergence

The loop is capped at K = 5 rounds. The paper deliberately avoids increasing this bound: empirically, tuples that fail to converge by round 5 are dominated by ambiguous task instructions and unsolvable specifications that no amount of script revision will fix. Surfacing these to the gap-fill pass (Pass 2 of task generation) is more cost-effective than additional inner-loop iterations.

Each unsuccessful round produces a structured REVIEW.md document from the Discriminator containing:

• The failing condition(s) among C1–C5.
• The observed reward values (r_init, r_gold).
• The matched forbidden pattern (if any).
• A free-text recommendation for the Generator.

The Generator consumes REVIEW.md as its sole feedback signal and revises its scripts accordingly. The feedback is targeted: if C3 failed because r(V_gold) = 0.65 (partial credit rather than 1.0), the Generator knows to fix the golden patch to match what the reward expects. If C4 failed because r(V_init) = 0.15 (initial state accidentally earns points), the Generator knows to remove task-completion artifacts from the initial setup. If a script execution error occurred, the Generator fixes the specific exception trace.

This iterative refinement with structured feedback is what distinguishes the pipeline from a simple "generate once, verify once" approach. The adversarial pressure—the Discriminator actively trying to write a reward the Generator cannot satisfy, and the Generator iteratively closing the gap—drives both subagents to produce artifacts that are more robust than what either could produce in isolation.

Worked Example: A Two-Round Convergence

The paper provides a concrete trace of one production task through two adversarial rounds (Appendix E.4). The task (calc_gao_008) requires the agent to add two named scenarios to a workbook, where each scenario stores a distinct 5-tuple of cell values.

Round 1: The Generator's golden_patch.py introduces both scenarios but accidentally populates the "Optimistic" scenario with the "Pessimistic" values (a copy-paste regression). The Discriminator detects the mismatch when scoring the golden state—only the Pessimistic scenario matches the expected values—and emits a structured REVIEW.md with score 0.65 on C3 (golden-state reward) and the failing component identified: "The Optimistic scenario currently stores B2:B6 = (350000, 140000, 100000, 60000, 30000), which are the same values as the Pessimistic scenario."

Round 2: The Generator addresses the Round-1 feedback by writing distinct inputCells for the two scenarios. The Discriminator re-scores both endpoints and returns a PASS verdict with all five conditions satisfied.

This pattern—a tightly-scoped diagnostic in Round 1 followed by a targeted fix in Round 2—accounts for the majority of multi-round convergences in the released dataset.

Loop Termination and Throughput

If Round 5 also fails (all five conditions not met), the tuple is rejected and logged for offline analysis. The paper reports that the per-stage filter yield (Section 3.4, Filter section below) shows that approximately 3,100 tuples are rejected at the LLM majority-voting filter stage for P0 defects—these represent tuples that passed the loop-level checks but had deeper structural problems. An additional 1,278 tuples are rejected at the teacher-rollout stage. The final corpus is the 32,112 verified tuples from a starting pool that was substantially larger (the paper does not report the exact initial generation count, but the filter yields suggest a total loop-accepted pool of approximately 36,490 tuples, of which ~12% are rejected by the filters).

The Dataset-Level Filter: Closing the Loop-to-Training Gap

Loop-level convergence (C1–C5 satisfied) ensures the tuple is internally consistent, but it does not guarantee training utility. The adversarial loop operates at the level of pairwise endpoint consistency—the reward distinguishes initial from golden—but cannot detect higher-order issues:

• Ambiguous instructions that could be interpreted in multiple valid ways, where the reward encodes only one interpretation.
• Infeasible tasks that no agent policy can actually solve, either because the step budget is too tight or because the required action sequence is unreachable from the initial state.
• Trivial tasks that any agent completes on the first attempt, providing little RL signal.
• Reward hacking that passes the per-task forbidden-pattern scan but exploits subtler loopholes (e.g., semantic checks that are too loose, state proxies that correlate with but do not guarantee task completion).

The dataset-level filter addresses these through two independent stages.

Stage 1: LLM Majority Voting

Each loop-accepted tuple is scored by an ensemble of V heterogeneous critic models (the paper does not specify the exact value of V, but the aggregation rule requires a strict majority, implying V ≥ 3). Each critic receives the complete four-tuple (t, s_init, s_gold, r) and emits a structured JSON verdict with the following fields:

The severity rubric is:

• P0 (reject): Fatal setup, reward, or environment problem; mismatch between (t, s, r) that cannot be repaired by query revision alone.
• P1 (modify-query): The query is missing essential context, contains harmful ambiguity, or leaks process in a way that biases evaluation.
• P2 (keep-or-modify): Non-fatal quality issues; the tuple is usable but not ideal.
• P3 (keep): No meaningful issue beyond surface style.

The aggregation rule is: A tuple is admitted to the next filter stage if a strict majority (⌈V/2⌉ + 1) of voters return verdict ∈ {keep, modify_query}. Among admitted tuples, the canonical instruction is selected by majority vote over voter verdict: if the majority is modify_query, the released instruction is the highest-confidence revised_query among modifying voters; if the majority is keep, the original instruction is retained. Ties default to rejection. The ensemble's per-tuple severity is reported as the maximum severity assigned by any voter, providing a worst-case-aware quality grade in the released metadata.

Stage 2: Teacher-Model Rollout Verification with VLM-as-a-Judge

Critic-admitted tuples additionally pass through a teacher-rollout verification stage. A strong teacher policy (Claude-Sonnet-4-6) is rolled out on each tuple for N_teach trials (the paper does not specify the exact value). Each rollout is scored twice:

1. Programmatic reward r(s, τ) ∈ [0, 1]: the reward.py function executed on the final VM state after the teacher's trajectory τ.
2. VLM-as-a-judge score r̂_vlm(s, τ) ∈ {0, 1}: a separate VLM judge that consumes the final-state screenshot, the task instruction, and a checklist of acceptance criteria distilled from t. This judge provides an independent signal of task completion that does not rely on the programmatic reward's instrumentation.

The two scores measure different aspects of correctness. r is grounded in environment state (file contents, structured-state diffs, library introspection) and exhibits low variance but only sees what the reward author chose to instrument—if the reward checks that a spreadsheet cell contains a formula but does not check the font size, that gap goes undetected. r̂_vlm is grounded in the final visual state and catches surface-level failures the programmatic reward misses (a UI that loaded with the right data but in the wrong color theme, a chart whose values are correct but axis labels are missing). Concordance between the two scores is a strong signal that the reward function is faithful to the task's intent. Persistent disagreement is a sign that either the reward is too narrow or the task itself is under-specified.

Tuples are categorized by (r̄, r̄̂_vlm), the means of the two scores across the N_teach trials:

The paper does not specify the exact threshold δ, but the categorization logic implies a moderate value (likely in the range 0.1–0.3) since it is used to detect qualitative disagreement rather than measurement noise.

Disagreement-flagged tuples that survive Discriminator refinement re-enter teacher rollout. Tuples that fail to converge across two refinement passes are dropped to an offline-analysis bucket for pipeline improvement (feeding into the gap-fill and edge-case passes of task generation).

Per-Stage Yield and Filter Efficiency

The paper reports approximate per-stage rejection counts (Section 3.4 of the appendix):

• LLM majority-voting stage: rejects approximately 3,100 tuples on P0 setup/reward defects, P1 query issues, or persistent voter disagreement. This stage primarily catches static specification errors that survive the loop's agreement conditions but fail end-to-end critique—ambiguous language, semantic inconsistency between instruction and reward, and reward functions that depend on assumptions not enforced by the environment.

• Teacher-rollout stage: rejects an additional 1,278 tuples that the teacher cannot solve in N_teach trials or that score trivially on the first trial. This stage primarily catches infeasibility and triviality that only manifest under realistic agent behavior.

The two stages contribute roughly 70/30 to total filter loss, meaning the LLM voting stage is the heavier filter. The final corpus is the 32,112 verified tuples.

The filter is what makes the pipeline practically usable for RL training rather than merely an academic demonstration. Loop-level consistency alone would produce a dataset with a substantial fraction of degenerate tuples—data points where the reward signal does not align with task success, or where no policy can realistically earn the reward. These degenerate tuples would inject noise into the RL optimization, potentially causing the instability (oscillations, reward-success decoupling) that the paper explicitly notes does not occur in their training runs (Section 4.1, Figure 7). The filter is thus not an optimization; it is a load-bearing component of the data-generation stack.

CUA-GYM-HUB: The Environment Synthesis Pipeline

The diversity of CUA-GYM tasks is upper-bounded by the diversity of available environments. If all training tuples were grounded in the same 2–3 desktop applications, the RL policy would overfit to those specific interfaces and fail to generalize. However, real websites and desktop applications are fundamentally unsuitable as RL training environments because:

• Authentication gates require sign-in (OAuth, SSO, passwords) that cannot be automated across thousands of ephemeral VM instances.
• Rate limits and terms of service prevent programmatic, high-frequency interaction.
• Non-reproducible state—real applications have server-side state that cannot be injected, inspected, or reset through a clean API. Two training rollouts on the same task would see different data, breaking the verifiability contract.

CUA-GYM-HUB resolves this by synthesizing self-contained mock web applications that preserve the interactive fidelity of their real counterparts while exposing full programmatic control over their state. The suite covers 94 applications drawn from two grounding sources:

1. Widely used products spanning communication (Slack, Teams, Discord, Gmail), productivity (Notion, Google Docs, Google Sheets, Miro), e-commerce (Amazon, Shopify, eBay), code hosting (GitHub, GitLab), cloud consoles (AWS, Azure), CRM (Salesforce, HubSpot), analytics (Google Analytics, Tableau), and healthcare/legal (Epic, Clio, DocuSign).

2. Long-tail targets selected to broaden domain coverage into specialized occupational categories identified by O*NET taxonomies—applications that are essential for specific professions but would be under-represented in a head-only selection.

The paper provides a complete inventory (Table 3, Appendix B.2.1) and representative screenshots (Figure 10, Appendix B.2.2) of 32 mocks spanning the major categories.

Multi-Agent Synthesis Pipeline: Plan, Dev, Web

Each mock is produced by a three-agent pipeline (Figure 2) that operates without per-mock human intervention, enabling the synthesis of 94 applications at scale:

Plan Agent. Given an application name, the Plan Agent performs structured web research (documentation crawls, screenshot collection, feature inventory across user-role personas) and emits four artifacts:

• DESIGN.md: The design system specification—color palette, typography tokens, spacing scales, component styles. This ensures visual consistency across the mock's pages.

• assets/README.md: UI layout descriptions and primary user workflows, derived from reference screenshots and documentation. This is the source of truth for what interactive elements must exist and how they should behave.

• assets/data_model.md: Entity definitions for the in-memory state—what objects exist (users, channels, messages, tasks, issues, products), their fields, and their relationships. For a Slack mock, the data model defines users, channels, messages (with sender, timestamp, thread parent, reactions), and workspace settings. For a Jira mock, it defines projects, sprints, boards, issues, and the issue-status workflow graph.

• TODO.md: A prioritized P0/P1/P2 work queue for the Dev Agent, enumerating every page, component, interactive element, and state transition that needs to be implemented.

The Plan Agent's role specification explicitly excludes authentication flows, real network communication, and persistence beyond browser localStorage. The mock's contract is to be a faithful interactive sandbox, not a functional clone of the reference service.

Dev Agent. The Dev Agent consumes the Plan Agent's artifacts and implements the mock as a Vite + React single-page application following a fixed project layout:

The Dev Agent works in file-based coordination with the Plan and Web agents: it reads TODO.md, DESIGN.md, assets/README.md, and assets/data_model.md from the Plan Agent, and reads TEST.md (functional/visual bug reports) and AUDIT.md (code-level issues) from the Web Agent. Its priority order for any work cycle is: AUDIT P0 issues > TEST P0 issues > AUDIT P1 issues > TODO P0 items > TODO P1 items > TODO P2 items, ensuring code-correctness regressions are addressed before new feature work.

Web Agent. The Web Agent operates a headless Playwright browser against the deployed mock and verifies the UI against DESIGN.md and assets/README.md. It:

• Exercises every interactive element catalogued in TODO.md.
• Performs visual diff against reference screenshots when available.
• Verifies that all four state-API endpoints (/post, /go, /state, /upload) respond correctly under session-isolation testing.
• Produces two reports: TEST.md (functional and visual bug reports) and AUDIT.md (code-level issues: dead event handlers, untracked state mutations, missing keys in the diff API, broken routes).

Both reports use the same P0/P1/P2 severity scale that the Dev Agent's priority order consumes.

Iterative convergence. The Plan, Dev, and Web agents iterate until TEST.md and AUDIT.md both report zero P0 and zero P1 issues, or until a per-mock round budget is exhausted. Each round consists of: Web Agent test pass → Dev Agent fix pass → Web Agent re-test. The Plan Agent re-engages only when fundamental specification gaps are surfaced (e.g., a feature appears in reference screenshots but is absent from TODO.md). Mocks failing to converge within the round budget are flagged for manual review and excluded from the released suite until resolved.

The paper reports structural statistics on the released mocks (Appendix B.2.4): mean source LOC of 6,127 (summed over JavaScript/TypeScript/CSS/HTML files under src/, excluding node_modules and build artifacts), mean 15 route components (React Router <Route> declarations), and mean 5.9 data-model entities (top-level entities in the in-memory data model). The bulk of mocks (91%) sit in the 2.5K–10K LOC band, consistent with the design constraint of a self-contained SPA implementing roughly 10–20 navigable views with a small typed in-memory schema. Outliers above 10K LOC (e.g., the AWS Console mock, the Canvas LMS mock, the Expedia mock) cover broader feature surfaces.

The Unified State API

Every mock implements a four-endpoint HTTP API co-resident with the SPA (implemented as a Vite middleware plugin, no separate process required). The API is what makes these mocks programmatically controllable environments rather than static website replicas:

POST /post?sid=<sid> — State injection and lifecycle control. The request body carries:

• action: one of "set", "set_current", "reset", "merge".
• state: a JSON object encoding the target session state.
• merge (optional boolean): controls whether "merge" action performs a deep merge or replacement.

Action semantics:

• "set": Writes state to the session's initial snapshot. Used by initial_setup.py to materialize the pre-task world state (e.g., populate a Slack workspace with specific channels, users, and messages).
• "set_current": Writes state to the session's current snapshot, leaving the initial snapshot untouched. Used by golden_patch.py and occasionally by initial_setup.py when an in-progress state is needed.
• "merge": Same as "set_current", but performs a deep merge into the existing current snapshot rather than replacing it. This is used when the task adds to existing state rather than overwriting it.
• "reset": Clears both initial and current snapshots and reloads the default seed data.

The response carries { "success": true, "sid": "<sid>", "state_id": "<hash>" }.

GET /go?sid=<sid> — Structural state inspection. The response carries:

• initial_state: the initial snapshot (as injected by "set").
• current_state: the current snapshot.
• state_diff: a flat key-path map from changed fields to { "old": "<value>", "new": "<value>" } pairs.

This is the primary interface used by reward.py. The diff representation is intentionally flat (key-path strings rather than nested objects) because reward authors write linear assertions over individual fields.

The diff is computed under three rules:

• Object fields are compared structurally (recursive descent on nested objects).
• Arrays are treated as ordered sequences; a difference at any index marks the entire array as changed. This avoids false partial-equality matches when the agent reorders rather than edits.
• Whitelisted volatile fields (e.g., lastViewedAt, computed UI-state caches) are masked before diff to prevent passive viewing actions from polluting the diff.

GET /state?sid=<sid> — Raw current state. Returns { "stored_state": "<object>", "has_custom_state": "<bool>", "sid": "<sid>" }. Provided as a lower-level inspection endpoint when the diff abstraction is unnecessary.

POST /upload?sid=<sid> — File upload for tasks requiring user-supplied attachments. Body: multipart/form-data. Response includes a per-file URL that the SPA can resolve. Files are scoped to the sid and discarded on session reset.

Session Isolation

Every API call carries a session identifier sid as a query parameter, scoping all state reads and writes to a per-session storage namespace. This is what enables parallel RL rollouts against a shared pool of mock backends:

• The trainer generates a fresh sid (UUIDv4) for each rollout and persists it to a known file path inside the VM (/tmp/task_web_sid).
• All subsequent initial_setup.py, agent-issued web requests, and reward.py calls share the same sid.
• Sessions are independent: concurrent rollouts on the same mock instance do not interfere, because their state namespaces are disjoint.
• Session state has a TTL of one hour past the last access; expired sessions are garbage-collected without intervention.

The paper illustrates the effect of session isolation with a concrete example (Figure 3): the same email mock can be instantiated with three distinct task-specific initial states—a clean inbox, a deadline-heavy project state with pending coordination threads, and a high-volume backlog after an absence—without any code changes to the mock implementation. The injected state determines which tasks are feasible, and the session isolation ensures that the agent's mutations in one rollout do not affect another.

Integration with reward.py

The canonical reward.py pattern for web-mock tasks illustrates how the state API feeds into verification:

import requests

def verify_task():
    sid = open('/tmp/task_web_sid').read().strip()
    response = requests.get(f'http://localhost:8080/go?sid={sid}').json()
    diff = response['state_diff']
    score = 0.0
    if 'channels[0].name' in diff and diff['channels[0].name']['new'] == 'engineering':
        score += 0.25
    if 'messages.engineering[0].content' in diff:
        score += 0.25
    # ... additional assertions ...
    print(f'REWARD: {score}')
    return score

The reward function:

1. Reads the session identifier from the known file path.
2. Fetches the state diff from the mock's /go endpoint.
3. Iterates through a sequence of partial-credit assertions, each checking a specific key-path in the diff.
4. Accumulates a [0, 1] score and prints it in the REWARD: <score> format that the RL trainer expects.

This pattern is what decouples the reward function from the application's implementation: the reward author (the Discriminator) does not need to know how the mock stores its state internally—it only needs to know what key-paths in the diff correspond to task completion. The unified API provides a stable contract that works identically across all 94 mock applications.

Design Choices: Mocks vs. Real Websites

The paper is explicit about what is sacrificed in the mock approach:

• Authentication is stripped. There are no sign-in flows, no OAuth, no SSO. All state is local to the session.
• All data is synthetic and session-local. The mock has no connection to a real backend; all mutations are in-process state changes.
• Third-party integrations are replaced by deterministic stubs. Where the real application would make outbound network calls (payment processing, email sending, real-time push notifications), the mock exposes a deterministic stub that supports task verification without network dependence.

The paper argues that these sacrifices are necessary for the mocks to function as RL training environments, and that they affect a different capability than what the RL training targets: the agent learns to navigate and manipulate the application interface, not to authenticate or handle network failure modes. The WebArena transfer result (CUA-GYM-A17B improves WebArena from 54.0% to 56.0% despite WebArena using different site clones than the 94 training mocks) is cited as evidence that the skills learned on mocks generalize to real browser environments, supporting the claim that the stripped surface does not prevent transfer.

Domain Skill Files (SKILL.md): Encoding Application Knowledge

Each application in the environment pool ships with a SKILL.md document—a structured knowledge file that both the Generator and Discriminator load at synthesis time. This is how application-specific expertise is injected into the coding agents without requiring them to have memorized every application's API surface.

The standard structure has six sections:

1. Domain-specific concepts and Python libraries. For libreoffice_calc, this section catalogues openpyxl (for spreadsheet creation and manipulation) and pandas (for bulk data operations). For libreoffice_writer, it catalogues python-docx. For GIMP, it catalogues image manipulation libraries. This ensures the agents import the correct toolchain.

2. State and file-system layout. Canonical paths for documents, configuration directories, and asset files. For desktop applications, this includes the expected VM file-system structure (/home/user/, /home/user/Desktop/, /home/user/Documents/). For web applications, this includes the mock URL template and the session-identifier file path.

3. initial_setup.py templates. Parameterized code skeletons for common task contexts (e.g., creating a workbook with specific sheets, populating a Slack workspace with specific channels). The Generator adapts these templates to the specific task's context rather than writing from scratch.

4. golden_patch.py templates. Corresponding templates for constructing the post-task state.

5. reward.py scoring patterns. Examples of progressive partial-credit decomposition for common task types (data-editing, formatting, formula creation, chart construction, document layout). This shows the Discriminator how to structure a reward function for the application.

6. "Bitter lessons" — a curated list of pitfalls observed during development. Each entry is born from a specific debugging episode in early pipeline runs. The paper provides an example from the libreoffice_calc SKILL.md (Appendix A.5.3) with 12 bitter lessons, including:

   • "Formula values are NOT computed by openpyxl. cell.value returns the formula string '=SUM(A1:A10)', not the result. Use data_only=True to get the last-cached value."
   • "Always use 8-char ARGB for colors. PatternFill(start_color="4472C4") silently becomes '004472C4' (alpha=00, transparent). Write 'FF4472C4'."
   • "fgColor is the visible background, not bgColor. cell.fill.fgColor.rgb gives you the background color you see."
   • "Merged cells: only the top-left has data. After merge_cells('A1:D1'), B1/C1/D1 become MergedCell with value=None. Style the top-left cell only."
   • "Copy-then-modify for golden files. Never recreate from scratch if an initial file exists."
   • "Pivot tables cannot be created by openpyxl; only read/preserved. Use a template file with the pivot already built."
   • "showDropDown=False means SHOW the dropdown. In DataValidation, this boolean is inverted."

The bitter-lessons section is the most distinctive feature of the SKILL.md format because each entry encodes knowledge that an LLM Generator or Discriminator would otherwise rediscover at the cost of a wasted adversarial round. By pre-loading these pitfalls, the SKILL.md makes each round of the adversarial loop more productive—the agents spend their iteration budget on task-specific issues rather than rediscovering well-known library quirks.

SKILL.md files are generated from a one-shot research pass over the application's documentation, then refined by inspecting the rejection logs of the first generation pass and surfacing recurring failure patterns as new bitter-lesson entries. They are versioned alongside the synthesis pipeline, and all empirical results in the paper use the version frozen at release.

Integration: The Complete Synthesis Pipeline

Putting the pieces together, the end-to-end pipeline (Algorithm 1, Appendix A.1) operates as follows:

Input: A task instruction t, a context description c, a domain skill file S_dom, and a maximum round budget K = 5.

Step 1 – Provision VMs: The Orchestrator provisions two isolated virtual machines, V_init and V_gold, both booted from the same base snapshot (the OSWorld-standard Ubuntu desktop image).

Step 2 – Adversarial loop: For k = 1, ..., K:

• The Generator reads (t, c, S_dom) and writes initial_setup.py and golden_patch.py.
• initial_setup.py is executed on V_init; golden_patch.py is executed on V_gold (on the golden VM, independently from the initial VM).
• The Discriminator reads (t, c, S_dom) only (no access to the Generator's scripts) and writes reward.py.
• reward.py is scanned against the forbidden-pattern list. If any pattern matches, the round is aborted and the Discriminator is re-prompted.
• reward.py is evaluated on both VMs to obtain r_init and r_gold.
• The Orchestrator checks the five agreement conditions (C1–C5). If all five hold, break out of the loop. Otherwise, the Discriminator emits structured feedback F_k in REVIEW.md, and the Generator revises its scripts.

Step 3 – Acceptance or rejection: If convergence is not reached after K = 5 rounds, the tuple is rejected and logged for offline analysis.

Step 4 – Dataset-level filter: If the loop accepts the tuple (all C1–C5 satisfied), it is submitted to the two-stage filter:

• LLM majority voting across V critics.
• Teacher-model rollout verification with N_teach trials and VLM-as-a-judge cross-check.
• Tuples surviving both stages enter the final released dataset.

Output: A verified tuple (t, s_init, s_gold, r) packaged as a self-contained directory under output/final/<task_id>/ with the following file layout:

The OSWorld evaluator contract (config.json) follows the upstream OSWorld schema verbatim with no extensions, ensuring that CUA-GYM tuples are usable as drop-in OSWorld benchmark items. This compatibility was an explicit design goal: the authors note that "full compatibility with the upstream OSWorld evaluator was an explicit design goal so that CUA-GYM tuples are usable as drop-in OSWorld benchmark items" (Appendix A.6.2).

Cost Economics

The paper reports per-tuple generation cost in terms of both LLM tokens and VM compute (Appendix F.2):

LLM cost: A single accepted tuple consumes approximately 10,000 input tokens and 5,000 output tokens of Claude-Sonnet-4-6, distributed across the four LLM-driven pipeline stages (Task-Gen, Generator, Discriminator, Filter). Input is dominated by the Generator and Discriminator system prompts plus their REVIEW.md context windows; output is dominated by the Discriminator's reward script and the Generator's setup/golden patches. At Claude-Sonnet-4-6's public pricing of $3/million input tokens and$15/million output tokens, this works out to 0.030 + 0.075 ≈ $0.11 of LLM spend per verified tuple. Scaled to the 32,112-tuple corpus, total LLM cost is approximately $3,300–$3,500.

VM cost: The dual-VM execution stage (provisioning two instances, executing setup and golden patches, running the reward function, tearing down) takes approximately 45 minutes of wall-clock time per accepted tuple on Aliyun ECS g8i.xlarge instances (4 vCPU, 16 GB RAM, on-demand pricing ~$0.21 per instance-hour). The cost breakdown is: snapshot restore (~2 min), initial_setup execution (~10 min average), golden_patch execution (~8 min), reward execution including post-config (~15 min, dominated by document re-render and assertion parsing), teardown (~10 min). Amortized VM cost per verified tuple is `0.75 × 2 ×$0.21 ≈ $0.32`. The released corpus therefore carries approximately$10K of cumulative VM compute, dominating the marginal cost by roughly 3× over LLM cost.

The total pipeline cost for 32,112 tuples is therefore on the order of $13K–$14K. This is the cost of generating the entire training dataset, which can then be used to train multiple models, run ablation studies, and serve as a community resource. The paper frames this as evidence that the pipeline is cost-effective: for roughly the cost of a single mid-scale training run, it produces a dataset that multiple research groups can use.

Summary of Major Design Choices

4. Key Insights and Innovations

Innovation 1: The Adversarial Co-Generation Architecture with Information Isolation as a Data-Integrity Primitive

The paper's central conceptual move is to reframe CUA training data generation not as a task-synthesis problem, but as a two-player adversarial game between a Generator (who constructs environment states) and a Discriminator (who writes reward functions), with a strict information barrier preventing the latter from observing the former's construction procedure. This framing is fundamentally different from how prior work has approached the same bottleneck.

What the field did before. Prior approaches to generating CUA training data fall into two camps. One camp (supervised datasets like OpenCUA, AGUVIS) avoids reward generation entirely, providing only trajectory-level demonstrations that cannot drive RL optimization. The other camp either uses VLM-as-a-judge scoring—which the paper and prior work (ZeroGUI's own ablation) confirm introduces destabilizing reward noise—or generates programmatic rewards through code-native browser environments (GUI-Genesis, InfiniteWeb), but these confine themselves to single-application browser tasks and do not scale to the desktop or cross-application regimes. Critically, all prior approaches that do attempt programmatic reward generation use a single-agent paradigm: one LLM writes both the environment setup and the reward function, then checks consistency. The paper identifies this as structurally inadequate because a single agent cannot be trusted with both sides of the verification problem.

The conceptual shift. The adversarial architecture with information isolation is not an incremental improvement—it is a qualitatively different solution to a problem that the field had not clearly articulated. Before CUA-GYM, the bottleneck was described as "CUA training data is hard to generate at scale." The paper reframes this as: the difficulty is not generation per se, but preventing self-consistency from masquerading as task validity. When a single agent writes both the golden state and the reward, the reward tends to re-check the construction procedure—inspecting internal file paths, checking for specific function call traces, verifying properties that the agent's actions cannot actually influence. An RL policy trained on such tuples learns to trigger internal state transitions rather than complete user-goal tasks. This is a data-integrity failure mode distinct from the standard reward-hacking concern (where an RL policy exploits a deployed reward model)—here, the reward is corrupted at generation time, before any agent touches it.

The adversarial loop resolves this by forcing the Discriminator to derive its reward semantics from the task instruction alone, without ever seeing how the environments were built. The five agreement conditions (C1–C5) operationalize this: they ensure the tuple is internally consistent at both endpoints (initial-state reward zero, golden-state reward one) while the information barrier ensures the reward's discriminating properties come from the task semantics rather than the setup artifacts. The forbidden-pattern static scan (six patterns targeting direct assignment, bare existence scoring, subprocess usage, and comment-only verification) acts as a guardrail to prevent the most common failure modes from wasting adversarial rounds.

Significance beyond performance. This contribution matters even if the downstream RL results were weaker than reported, because it provides a reusable architectural pattern for any domain where training instances require co-generation of interdependent artifacts with verification. The pattern—adversarially coupled sub-systems, information barrier, agreement conditions as loop termination criteria, static analysis for known failure modes—is abstract enough to transfer to other multi-artifact generation problems (e.g., generating code + tests + documentation, or scene descriptions + renders + evaluation metrics). The paper does not claim this generality, but the architecture is recognizably a special case of a broader principle: when two artifacts must be consistent but one party's knowledge of the other's construction would corrupt the verification, adversarial coupling with information isolation is the correct design.

Evidence tie-back. The paper provides concrete examples of reward-hacking failures caught by the information barrier (Appendix E.3), showing that without it, the Discriminator produces rewards containing direct Boolean assignments (chart_verified = True) and bare existence checks (os.path.exists(...)) that would pass loop-level verification but provide no RL signal. The per-stage filter yields (approximately 3,100 LLM-voting rejections for P0 defects, plus 1,278 teacher-rollout rejections) demonstrate that even with the barrier, significant filtering is needed—but the fact that the RL training curves in Figure 7 show monotonic improvement without oscillation or reward-success decoupling is indirect evidence that the data surviving both the adversarial loop and the filter is sufficiently clean to support stable optimization.

Innovation 2: Environment Diversity as an Independent Scaling Axis Complementary to Data Volume

The paper's second conceptual contribution is establishing—through controlled ablation—that the diversity of training environments is a scaling axis separable from and complementary to the volume of training trajectories. Prior work on RLVR data scaling across math, code, and terminal operation has focused almost exclusively on task volume: generate more problems, train on more trajectories, get better performance. The implicit assumption has been that environment diversity matters only insofar as it enables the generation of more varied tasks—that the environment is merely the substrate for task generation, not an independent contributor to generalization.

What the field did before. In math RLVR, the environment is irrelevant (the task is a symbolic problem statement). In code RLVR (SWE-smith, SWE-Gym), the environment is the codebase, but scaling has focused on more repositories rather than on the type of diversity they provide. In terminal operation (Endless Terminals), the environment is the containerized system state, and scaling has focused on more tasks. In all these domains, the question "does environment diversity matter independently of task volume?" has not been asked, because the environment space is either trivial (math) or conceptuallly conflated with task space (code, terminal). CUA-GYM is unique in making the environment a first-class scaling investment—the CUA-GYM-HUB mock application suite is not a byproduct but a separately engineered artifact with its own synthesis pipeline—and then demonstrating that this investment pays off in a way that trajectory volume alone cannot replicate.

The conceptual shift. The paper decomposes scaling into two independent axes—data volume (number of training tuples) and environment diversity (number of distinct applications)—and provides the first empirical evidence that these axes are complementary. The key diagnostic result is Figure 8 (Section 4.2): in a teacher distillation setup, training on 80 environments with only 38 trajectories each outperforms training on 10 environments with 300 trajectories each, despite using the same total trajectory count (3K). Doubling trajectory volume on the broadened pool (80 environments, 75 trajectories each) produces a substantially larger gain. The interpretation is that narrow-but-deep training (many trajectories on few environments) overfits to the specific UI layouts, interaction patterns, and visual conventions of those environments, while broad-but-shallow training (fewer trajectories across many environments) forces the model to learn transferable skills—identifying buttons regardless of their visual styling, understanding form-field semantics across different layouts, navigating navigation trees with varying structures.

This finding has direct implications for how CUA training budgets should be allocated. A naive interpretation of data-scaling laws would suggest that more trajectories on a handful of environments should saturate less quickly than the same number of trajectories spread thinly across many. The paper shows the opposite: at current scales, breadth provides value that depth cannot substitute for. This is a genuinely new insight about the structure of CUA skill acquisition, not derivable from prior scaling studies in other domains where the environment space is simpler.

Significance beyond performance. The identification of environment diversity as an independent scaling axis has implications for how the field should invest engineering resources. Prior to this result, one could reasonably argue that synthesizing 94 mock web applications was over-engineering—that a smaller set of well-chosen environments with more tasks per environment would be sufficient. Figure 8 provides counter-evidence: the environment synthesis pipeline is not a nice-to-have augmentation but a load-bearing contributor to downstream performance, and further investment in environment breadth (covering more of the long tail of occupational software) is likely to yield returns that trajectory volume alone cannot.

Evidence tie-back. The experiment is conducted in a teacher distillation setup rather than full RL, which is a limitation (the paper acknowledges this in Section 4.2, noting that the granularity required for an environment-level RL ablation would be prohibitively expensive). However, the controlled comparison—narrow (10 envs, 300 traj each) vs. mid (80 envs, 38 traj each) vs. broad (80 envs, 75 traj each), with all students sharing identical SFT hyperparameters—provides clean evidence for the complementary-axes claim. The fact that the RL-trained models (CUA-GYM-A3B and A17B) improve on the held-out WebArena benchmark (from 40.8 to 44.5 and 54.0 to 56.0, respectively) despite WebArena using different site clones than the 94 training mocks provides convergent evidence that environment diversity during training produces skills that transfer across environments.

Innovation 3: Data Scaling as a Diagnostic for Reward Quality Rather Than Merely a Performance Knob

The paper repurposes data scaling—normally viewed as a means to improve downstream metrics—as a diagnostic instrument for evaluating the quality of the generated rewards themselves. This is a subtle but important conceptual move that distinguishes how CUA-GYM uses scaling curves from how prior RLVR work uses them.

What the field did before. In prior RLVR work (SWE-smith, SWE-Gym, Endless Terminals, DeepSeek-R1), scaling curves are presented as evidence that more data yields better models, with the implicit claim being that the data generation pipeline is producing useful training instances. The metric of interest is the slope and saturation point of the downstream performance curve. If the curve rises and hasn't saturated, the pipeline is working. This is a valid but limited use of scaling data—it tells you that the data is useful, but not why or whether it is as useful as it could be.

The conceptual shift. The CUA-GYM paper introduces a second, diagnostic use of scaling curves: they serve as a test for reward exploitability. A known failure mode of RLVR training with synthetic rewards is that the optimization can decouple from genuine task performance—the reward goes up while the actual success rate stays flat or declines, indicating that the policy has found reward-hacking strategies rather than task-completion strategies. This manifests in training curves as oscillation, collapse, or—most revealingly—divergence between training reward and evaluation success.

The paper explicitly checks for this in Figure 7, noting that "all three runs improve monotonically from the SFT baseline without the oscillation, collapse, or reward-success decoupling commonly seen in RLVR training." This is presented not merely as a sign that the training is going well, but as evidence that the information-barrier design is working: the rewards are sufficiently faithful to the underlying tasks that optimizing against them genuinely improves task performance, rather than optimizing against reward artifacts. In other words, the clean scaling curves validate the data-generation architecture, not just the training recipe.

Significance beyond performance. This reframing matters because it provides a principled way to evaluate data-generation pipelines without relying solely on downstream benchmark numbers, which are confounded by model scale, training algorithm, hyperparameters, and evaluation methodology. A pipeline that produces rewards that induce instability or decoupling at any data scale is a pipeline with a latent quality problem, regardless of whether a particular training run happens to produce a good checkpoint. The paper's demonstration that CUA-GYM rewards produce clean scaling across three data scales (1.4K, 3K, 12K) is therefore not just a "more data helps" result—it is evidence that the pipeline's core design decision (adversarial generation with information barrier) produces robust reward functions that remain aligned with task completion under optimization pressure.

Distinguishing incremental from fundamental. This is a fundamental conceptual contribution, not an incremental refinement, because it changes what scaling curves mean in the context of synthetic RLVR data. Before this work, a scaling curve was an end in itself—a demonstration that more data = better models. After this work, a scaling curve is also a diagnostic—a signal about whether the data is genuinely aligned with the intended task, or whether the pipeline is producing rewards that can be optimized independently of task success. This dual interpretation will be valuable for future data-generation pipelines in any domain where reward exploitability is a concern.

Evidence tie-back. Figure 7 provides the core evidence. The left panel shows OSWorld-Verified scores increasing monotonically with RL training steps for all three data scales, with the 12K run attaining the highest peak and sustaining a visibly higher band. The right panel shows training reward also increasing monotonically, with no decoupling between the two curves (they rise together, maintaining their relative ordering). The paper explicitly contrasts this with the "oscillation, collapse, or reward-success decoupling commonly seen in RLVR training," arguing that the clean curves validate the pipeline's design rather than merely reporting a favorable outcome.

Innovation 4: Emergent Multi-Action Batching as Evidence of Structural Efficiency Learning Under Pure Outcome Rewards

The paper reports an unprompted, emergent behavior during RL training: the policy spontaneously learns to pack multiple tool calls into single turns, compressing trajectories by 33–45% at matched task performance. This behavior was not designed, incentivised through reward shaping, or mentioned in the system prompt—it emerged purely from the pressure of the group-normalized advantage objective under a fixed per-task step budget. The paper uses this as evidence for a broader claim: structural efficiency behaviors can emerge from outcome-level RL without explicit instruction, paralleling the emergence of verification and self-reflection behaviors observed in reasoning-focused RL.

What the field did before. In reasoning RL (DeepSeek-R1, DeepSeekMath), emergent behaviors like chain-of-thought verification, self-correction, and reflection have been documented as spontaneous byproducts of training with outcome rewards on math and code tasks. These behaviors are notable because they were never explicitly programmed—the models discovered that internal reasoning steps improved their probability of producing correct final answers under the reward function. In the GUI agent domain, prior work on batching and action efficiency has been predominantly architectural: models are explicitly designed to emit multiple actions per turn (UI-TARS, some CUA foundation models), or trajectory compression is achieved through post-hoc distillation. The idea that an agent trained with single-action-per-turn rollouts might spontaneously discover multi-action batching purely through RL had not been demonstrated.

The conceptual shift. The paper interprets the emergence of multi-action tool calls not as a curiosity but as evidence of a general principle: when the reward is defined over task outcomes rather than action sequences, and when the agent faces a step budget that imposes a real cost on inefficiency, the RL optimization will discover structural optimizations that the system designer did not anticipate. This is significant because it suggests that the standard approach of hand-designing action spaces (single-action-per-turn, with human-engineered macro actions for common sequences) may be unnecessary—the agent can learn efficient action-packing on its own, potentially discovering patterns that human designers would miss.

The paper's analysis of which actions get batched and which do not provides further depth. Table 7 shows that the dominant batched patterns are deterministic input chains (type → key, accounting for 3,942 steps) and continuous-scroll preludes (mouse_move → scroll, 1,052 steps). Table 8 shows the complementary pattern: actions with non-deterministic outcomes—right-click (solo rate 98.0%), double-click (94.8%), triple-click (94.6%)—are almost never batched. The interpretation is that the policy has internalized a coarse model of action determinism: it batches actions whose outcomes are predictable without visual feedback (keyboard input, scroll gestures), and defers actions that require re-observing the screen to decide the next move (clicks that trigger context menus or state-dependent UI changes). This internal model was not taught—it was discovered through the RL optimization's implicit pressure to maximize reward under a step constraint.

Distinguishing incremental from fundamental. This is a significant finding but more domain-specific than Innovations 1–3. It does not introduce a new architectural principle or scaling axis, but it does provide the first evidence that the emergent-efficiency phenomenon observed in reasoning RL generalizes to a fundamentally different domain (GUI interaction). The parallel is not superficial: in both cases, a model trained with pure outcome rewards spontaneously develops behaviors that improve efficiency (structured reasoning in one case, action batching in the other) without any explicit instruction to do so. This suggests that the phenomenon may be a general property of RL with outcome rewards under resource constraints, not a quirk of reasoning tasks.

Evidence tie-back. Figure 9 (Section 4.3) provides the core evidence, showing average tool calls per model step rising from approximately 1.0 at SFT initialization to a stable 1.4–1.9 band during RL training. The claim of 33–45% trajectory shortening at matched task performance is reported in the text of Section 4.3. Tables 7 and 8 (Appendix D.1) provide the fine-grained evidence for which action categories are batched and which are avoided, supporting the interpretation that the policy has learned a model of action determinism rather than blindly batching everything.

5. Experimental Analysis

Evaluation Methodology

Dataset. The primary evaluation benchmark is OSWorld-Verified, a subset of the OSWorld computer-use benchmark with ground-truth programmatic evaluation. The paper uses a test split of 369 tasks (for Qwen3.5-35B-A3B-scale evaluation; 360 for the A17B scale) spanning desktop and web domains. Additionally, WebArena serves as a held-out browser benchmark whose site clones are disjoint from the 94 mock web applications used during training, testing cross-platform transfer. CUA-GYM training data itself consists of 10,858 sampled verified tuples from the full 32,112-tuple corpus.

Base model(s). Experiments use two parameter scales of the Qwen3.5 family (Qwen Team, 2026): Qwen3.5-35B-A3B (a 35B-total-parameter Mixture-of-Experts model with 3B activated parameters) and Qwen3.5-397B-A17B (397B total, 17B activated). The paper states these were chosen as representative open-source MoE backbones at two distinct capability levels, enabling the study of how RLVR gains scale with base model strength.

Metrics. The primary metric is task success rate (%) — the fraction of evaluation tasks for which the agent's final trajectory achieves a terminal reward of 1.0 under the OSWorld-Verified evaluator's programmatic reward functions. These evaluator-provided rewards are distinct from the training rewards generated by CUA-GYM's pipeline. For WebArena, the native benchmark's success metric is used. Training reward is additionally tracked as a diagnostic signal during RL.

Baselines. The paper compares against:

• Untrained base models: Qwen3.5-35B-A3B and Qwen3.5-397B-A17B, both after SFT warm-up on 3,578 curated teacher demonstrations but before RLVR training. This is the direct comparison for measuring the value added by RLVR on CUA-GYM data.
• Prior open-source CUAs at comparable scales: EvoCUA-8B (46.1%), EvoCUA-32B (56.7%), OpenCUA-32B (34.8%), OpenCUA-72B (45.0%), Step-GUI-8B (40.2%), all evaluated on OSWorld-Verified. These are cited from their respective papers and serve as external reference points for the state of open-source CUA performance.
• Proprietary frontier models: Claude Sonnet 4.6 (72.9%), Claude Opus 4.7 (78.0%), GPT-5.5 (78.7%) on OSWorld-Verified, and Claude Sonnet 4.6 (65.6%) on WebArena. These are upper-bound references, not direct competitors—the paper does not claim to surpass them.
• Majority voting is used as a baseline in the data-scaling and search analyses (Figures 3, 4), though less prominent in the main results table.

Generation budget / compute accounting. For RL training, the generation budget is measured in model steps per rollout, capped at max_turns = 100 per episode. Each step consumes one agent observation-action cycle. For the FLOPs-matched comparison (Section 7, discussed elsewhere), the paper uses standard approximations from scaling laws literature: pretraining FLOPs X = 6ND_pretrain and inference FLOPs Y = 2ND_inference, where N is parameter count. For the main RL results, compute is not directly budget-matched across models—comparisons are at the same training hyperparameters and data scale rather than at equal FLOPs.

Cross-validation / statistical protocol. The RL results are reported from single training runs due to compute cost. The paper explicitly notes this limitation in Section 6: "our largest RL runs are reported with single seeds due to their compute cost, so we interpret the results as evidence that verified task and environment scale are valuable in the studied regime, not as a final characterization of the pipeline's ceiling or deployment robustness." For the data-scaling ablation (Figure 7), three training runs at different data volumes (1.4K, 3K, 12K tuples) are compared, all initialized from the same SFT checkpoint with fixed hyperparameters. The environment-scaling experiment (Figure 8) uses a teacher distillation setup rather than full RL, with three settings differing only in training set composition, sharing identical SFT hyperparameters. No confidence intervals or error bars are reported across the paper's main results.

Main Quantitative Results

Core RLVR Training Results on OSWorld-Verified

The central result is Table 5b and Figure 5a: RLVR training on CUA-GYM data produces substantial and consistent improvements over SFT-initialized base models at both parameter scales. CUA-GYM-A3B lifts the Qwen3.5-35B-A3B base from 54.5% to 62.1% (+7.6 percentage points). CUA-GYM-A17B lifts the Qwen3.5-397B-A17B base from 62.2% to 72.6% (+10.4 percentage points).

Several properties of these results are notable. First, the gain increases at the larger scale (+10.4 pp vs. +7.6 pp), rather than diminishing—this is the regime where prior work on GUI agent RL has most often observed plateauing, making the continued improvement significant. Second, CUA-GYM-A3B at 62.1% matches the untrained Qwen3.5-397B-A17B base at 62.2%, meaning the RLVR-trained 3B-activated model achieves performance comparable to an untrained ~14× larger model (in total parameters; ~5.7× larger in activated parameters). This is the efficiency claim that anchors the paper's practical value proposition.

Third, the per-domain breakdown (Figure 5a) reveals that gains are concentrated in specific application categories rather than uniformly distributed. The largest improvements for CUA-GYM-A3B are in multi-app workflows (+21.5 pp, from 30.0% to 51.5%), libreoffice_calc (+14.9 pp, from 39.0% to 53.9%), and vs_code (+13.6 pp, from 42.0% to 55.6%). These are precisely the categories that require coordinating multiple application states or manipulating complex document models—skills that supervised fine-tuning alone struggles to teach but RL with programmatic rewards can reinforce through trial and error. Conversely, os-level tasks show zero gain (+0.0 pp, 70.0% to 70.0%) for A3B, and thunderbird shows a regression (-13.3 pp, 80.0% to 66.7%) for A17B, which the paper does not discuss but which suggests that certain application domains may already be near the base model's ceiling or that the RL optimization can overfit in narrow domains with small evaluation sample sizes (n=18 for thunderbird at the A3B scale, n=15 at A17B).

Cross-Platform Transfer to WebArena

The same RLVR-trained checkpoints also improve on WebArena, a held-out browser benchmark whose site clones (GitLab, Reddit, Shopping, Map) are disjoint from the 94 mock web applications in CUA-GYM-HUB. CUA-GYM-A3B lifts the base from 40.8% to 44.5% (+3.7 pp); CUA-GYM-A17B lifts the base from 54.0% to 56.0% (+2.0 pp). The absolute gains are smaller than on OSWorld-Verified (which is expected, since WebArena sites were never seen during training), but the direction is consistent and the result is non-trivial: it demonstrates that skills acquired on synthesized web mocks generalize to visually and structurally different real browser environments. This is the key evidence for the paper's claim that the mock synthesis pipeline of Section 2.2 produces generalizable skills rather than overfitting to specific visual templates.

However, two caveats apply. First, the WebArena gains are smaller at the larger scale (+2.0 pp vs. +3.7 pp), which is the opposite pattern from OSWorld-Verified and may indicate that the larger base model already captures more of the transferable web-navigation skill during pretraining, leaving less room for RLVR on mocks to add value. Second, the paper does not report WebArena results for the SFT-only checkpoint (the SFT warm-up uses Claude-Sonnet-4-6 teacher demonstrations on CUA-GYM tasks, which may themselves transfer to WebArena), so it is not possible to attribute the gain specifically to RLVR versus the combination of SFT warm-up and RL.

Comparison with Prior Open-Source and Proprietary Models

In the context of existing open-source CUAs (Table 5b), CUA-GYM-A3B at 62.1% substantially outperforms all prior open-source models at comparable or larger activated parameter counts: EvoCUA-32B (56.7%), OpenCUA-72B (45.0%), Step-GUI-8B (40.2%). CUA-GYM-A17B at 72.6% approaches the proprietary frontier: it is within 0.3 pp of Claude Sonnet 4.6 (72.9%) and within 5.4 pp of Claude Opus 4.7 (78.0%). The paper does not claim to surpass these proprietary models, but the proximity is meaningful given the model scale difference (Qwen3.5-397B-A17B is a 17B-activated open model versus proprietary models of unknown but likely larger scale). The paper does not report confidence intervals or statistical significance for these comparisons, and the test sets are modest (369 and 360 tasks for the two scales), so the exact ranking should be treated as indicative rather than definitive.

Notably, Kimi-K2.6 achieves 73.1% on OSWorld-Verified, slightly above CUA-GYM-A17B's 72.6%, but the paper does not discuss this comparison or analyze what training methodology Kimi-K2.6 used—a notable omission given that it is the closest open-source competitor on this benchmark.

Data Scaling: More CUA-GYM Tuples → Better RL Performance

Figure 7 (Section 4.1) demonstrates that scaling the volume of verified CUA-GYM training data yields measurable improvements in downstream RL performance, with no evidence of saturation at the studied scale. Three training runs on Qwen3.5-35B-A3B, using 1.4K, 3K, and 12K verified CUA-GYM tuples (all initialized from the same SFT checkpoint with fixed GSPO hyperparameters), show:

• Monotonic improvement from SFT baseline. All three runs improve over the SFT initialization (OSWorld-Verified score ~0.53 at step 0), with the 12K run departing from the baseline earliest.
• Preserved ordering throughout training. The 12K run sustains the highest OSWorld-Verified band across the full 100-step training window, the 3K run next, the 1.4K run lowest. The relative ordering is stable rather than crossing, indicating that data scale affects the asymptotic ceiling rather than merely the convergence rate.
• No saturation at 12K. The 12K curve exhibits no inflection toward a plateau. The paper interprets this as evidence that "the data-scaling regime accessible through this pipeline has not been exhausted" (Section 4.1), implying that generating and training on additional CUA-GYM tuples would likely yield further gains.
• Clean training dynamics. The right panel of Figure 7 shows training reward rising monotonically for all three runs, with no decoupling from the OSWorld-Verified score (the curves track each other in relative ordering). The paper explicitly contrasts this with "oscillation, collapse, or reward-success decoupling commonly seen in RLVR training," arguing that it validates the information-barrier design's ability to produce non-exploitable rewards.

The paper also reports a complementary scaling experiment using teacher distillation (Figure 8, Section 4.2), where Claude-Sonnet-4-6 is rolled out on CUA-GYM tasks, successful trajectories are retained, and Qwen3.5-35B-A3B is fine-tuned via standard SFT. Doubling trajectory volume on a broadened environment pool (80 environments, 3K → 6K trajectories) produces a "substantially larger" gain than adding environments at fixed data (10 envs/3K traj → 80 envs/3K traj). The exact OSWorld-Verified scores should be reported from Figure 8: the broad setting (80 envs, 6K traj) achieves the highest bar, the mid setting (80 envs, 3K traj) next, and the narrow setting (10 envs, 3K traj) the lowest. The paper states that "narrow versus mid isolates environment diversity at fixed total data; mid versus broad isolates trajectory volume at fixed environment coverage." Both axes contribute, with trajectory volume contributing more in this particular setup.

Environment Scaling: A Complementary Axis

The environment-scaling result (Figure 8) is the paper's most unique finding about CUA training dynamics. The headline: training on 80 environments with only 38 trajectories each outperforms training on 10 environments with 300 trajectories each, despite using the same total 3K trajectory count. This demonstrates that environment diversity and data volume are complementary scaling axes—you cannot compensate for narrow environment coverage by simply generating more tasks on the environments you have. The paper explicitly notes that this finding "justifies the engineering investment in broadening environment coverage" through the CUA-GYM-HUB synthesis pipeline.

This is a genuinely non-obvious result. A naive reading of scaling laws would predict that concentrating training data on fewer environments would produce stronger per-environment performance (more practice on each interface), and that the agent would then generalize from those well-learned environments to new ones. The data shows the opposite: breadth forces generalization that depth alone cannot produce. The paper does not explore why this is the case mechanistically (e.g., whether it is due to visual overfitting, action-pattern memorization, or a fundamental property of GUI skill acquisition), but the empirical result is clean.

The caveat is that this experiment uses teacher distillation (SFT on Claude-generated trajectories) rather than RL. The paper acknowledges that "the granularity required for an environment-level ablation makes RL-based evaluation prohibitively expensive" (Section 4.2). While the distillation setup still provides evidence about the value of environment diversity, the magnitude of the effect may differ under RL—where the agent actively explores rather than imitating a teacher—and the interaction between environment diversity and RL exploration remains uncharacterized.

Emergent Multi-Action Tool Calls

This is reported in Section 4.3 and depicted in Figure 9. During RL training of CUA-GYM-A3B, the average number of tool calls per model step rises from approximately 1.0 at SFT initialization to a stable 1.4–1.9 band, representing a sustained behavioral shift rather than a transient fluctuation. The paper reports that this compresses trajectories by 33–45% at matched task performance, meaning the policy learns to accomplish the same tasks in substantially fewer model turns.

The claim is supported by fine-grained trajectory analysis in Appendix D.1 (Tables 7 and 8). Across 1,105 evaluation rollouts of the Qwen3.5-397B-A17B RL checkpoint at training step 30, totalling 22,361 model steps, the per-step batching distribution is: 69.6% single-call, 26.7% two-call, 2.4% three-call, 0.8% four-call, and 0.5% five-or-more-call steps. The most frequent batched sequences are deterministic input chains (type → key, 3,942 occurrences) and continuous-scroll preludes (mouse_move → scroll, 1,052 occurrences), together accounting for approximately 75% of all batched steps. The complementary pattern is equally informative: pointer-click variants with non-deterministic outcomes (right-click 98.0% solo, double-click 94.8% solo, triple-click 94.6% solo) are almost never batched, while mechanical sub-gesture actions (scroll, key_down, key_up, left_click_drag) are never emitted alone—they appear exclusively as components of larger batched sequences.

The paper interprets this as evidence that the policy has internalized a model of action determinism: it batches actions whose outcomes are predictable without visual feedback, and defers actions that require re-observing the screen. This interpretation is plausible given the data, though it is observational rather than causal—the paper does not run an ablation where the step budget is varied to test whether the batching behavior scales with budget pressure, which would strengthen the claim.

Ablation Studies and Robustness Checks

Data scale ablation (Figure 7): Training Qwen3.5-35B-A3B on 1.4K, 3K, and 12K CUA-GYM tuples shows monotonic improvement with data volume. The 12K run attains the highest OSWorld-Verified peak, departs from the SFT baseline earliest, and sustains a visibly higher performance band across training. No saturation is observed, indicating the data-scaling regime is not exhausted. Importantly, all three runs show clean training dynamics without reward-success decoupling—the paper explicitly interprets this as validation that the information-barrier design produces non-exploitable rewards.

Environment diversity ablation (Figure 8): In a teacher distillation setup (Claude-Sonnet-4-6 trajectories → SFT on Qwen3.5-35B-A3B), training on 80 environments with 38 trajectories each outperforms training on 10 environments with 300 trajectories each at the same total 3K trajectory count. Doubling trajectory volume on the broadened pool (6K trajectories, 75 per env on average) produces a further gain. The paper concludes that environment diversity contributes value that trajectory volume alone cannot recover.

Multi-action emergence analysis (Figure 9, Tables 7-8): Not a traditional ablation, but a behavioral analysis showing that RL training spontaneously induces multi-action batching (1.0 → 1.4–1.9 tool calls per step), compressing trajectories by 33–45% at matched task performance. The policy selectively batches deterministic action chains (keyboard input, scrolling) while avoiding batching across actions with non-deterministic outcomes (context-menu clicks). The base model at SFT initialization emits approximately one call per step; the RL-induced batching is a learned behavior.

RL training stability diagnostic (Figure 7, right panel): The paper presents the clean monotonic training reward curves—without oscillation, collapse, or decoupling from evaluation performance—as an implicit ablation validating the information-barrier design. The argument is that exploitable rewards (which would be produced without the information barrier and forbidden-pattern scan) induce characteristic instabilities that are conspicuously absent from the CUA-GYM training runs. This is a form of negative result: the absence of instability is the signal.

SFT warm-up vs. direct RL: The paper does not ablate the SFT warm-up stage. All RL runs are initialized from a model fine-tuned on 3,578 teacher demonstrations. It is therefore not possible to attribute the RL gains to the interaction of CUA-GYM rewards with the SFT initialization versus what RL alone would achieve from the base model. Given the paper's framing of CUA-GYM as an RLVR data contribution, this is a notable missing ablation—would RL on raw base models show similar or different scaling behavior?

GSPO hyperparameter ablations: The paper notes in Appendix C.3.2 that KL penalty β was tested at values of 10⁻³ and 10⁻² in early prototyping and "observed neither training-stability gains nor task-success improvements at the data scales reported here." All reported runs use β = 0 (no reference policy). This is a minor negative result but worth noting.

ReST^EM revision model ablation: Appendix K (Figure 16) reports a negative result where an attempt to further optimize the revision model using ReST^EM (Singh et al., 2024) backfires: "additional sequential revisions substantially hurt performance." The paper hypothesizes that on-policy data collection exacerbates spurious correlations in revision data. This result is mentioned in the appendix but not discussed in the main text.

No combination of PRM search with revisions: The paper notes in Section 8 that "we did not experiment with PRM tree-search techniques in combination with revisions." Since the paper studies these as complementary axes (verifier optimization via search, proposal distribution improvement via revisions), the failure to combine them is a limitation, though the paper treats it as future work rather than a missing ablation.

Oracle vs. predicted difficulty bins for compute-optimal allocation: The paper notes in Section 3.2 that both oracle (ground-truth pass@1) and predicted (PRM score-based) difficulty bins produce similar compute-optimal scaling gains, with curves largely overlapping (Figures 4 and 8). This is a robustness check on the difficulty estimation method rather than a pipeline ablation, but it addresses a key concern about circularity.

Critical Assessment

Claim 1: "RLVR on CUA-GYM data improves base models by +7.6 pp (A3B) and +10.4 pp (A17B) on OSWorld-Verified"

Does the experiment demonstrate this? Yes, directly. Table 5b and Figure 5a report exact numbers, and the improvement is consistent across both model scales. The experiment compares the SFT-initialized base (after teacher-demonstration fine-tuning) to the RLVR-trained checkpoint, both evaluated on the same OSWorld-Verified test split.

What are the caveats? (1) The SFT initialization includes 3,578 teacher demonstrations from Claude-Sonnet-4-6, meaning the base model is not "raw"—it has already benefited from a strong teacher's behavior. The 7.6 pp and 10.4 pp gains are therefore the marginal value of RLVR over strong SFT initialization, not the total value of CUA-GYM data. The paper does not report base model performance without SFT warm-up, so the full contribution of CUA-GYM (SFT + RL) versus a completely untrained base is unknown. (2) The results are single-run, with no confidence intervals, on relatively small test sets (369 and 360 tasks). At 360 tasks, a 1 pp difference corresponds to roughly 3–4 tasks. The per-domain sample sizes are even smaller (some domains have n < 20), making domain-level gains like thunderbird's -13.3 pp potentially noise. (3) The paper reports OSWorld-Verified as the primary benchmark, and the same CUA-GYM pipeline was used to generate OSWorld-format training tuples. There may be format-specific advantages (e.g., the agent learns OSWorld's screenshot resolution, VM conventions, and action-observation timing) that do not transfer to other CUA evaluation frameworks.

Claim 2: "CUA-GYM-A3B matches the untrained A17B base at roughly 10× fewer total parameters"

Does the experiment demonstrate this? Yes, the numbers support this: CUA-GYM-A3B at 62.1% vs. Qwen3.5-397B-A17B base at 62.2% (Table 5b). The claim is accurate as stated. However, three qualifications matter.

First, the comparison is against the SFT-initialized base, not the raw pretrained model. The A17B base has been fine-tuned on 3,578 teacher demonstrations—it is not a "raw" base model. A fairer statement would be "matches the SFT-warmed A17B model" or something similar.

Second, "10× fewer total parameters" refers to total parameters (35B vs. 397B), but both models are Mixture-of-Experts, where activated parameters matter more for inference cost. The comparison is approximately 5.7× in activated parameters (3B vs. 17B), which is still impressive but less dramatic.

Third, the comparison does not account for inference-time compute. The A3B model with RLVR training required 100-step rollouts with multi-action batching, while the A17B base was evaluated with standard single-action-per-turn inference. The paper does not report inference FLOPs for either model at evaluation time. If the A3B model takes more inference steps or more tokens per step to achieve its 62.1%, the wall-clock or FLOPs advantage over the A17B base may be smaller than the parameter count suggests. The paper's own finding about emergent multi-action batching (trajectory compression by 33–45%) partially mitigates this concern but does not eliminate it, since the compression is relative to the model's own earlier behavior, not to the A17B base's evaluation behavior.

Claim 3: "Performance scales smoothly with both data volume and environment diversity, with the latter identified as a complementary scaling axis"

Does the experiment demonstrate this? The data scaling experiment (Figure 7, RL training with 1.4K → 3K → 12K tuples) demonstrates monotonic improvement with data volume, with no saturation at 12K tuples. This is clean evidence that verified CUA RLVR data provides genuine scaling value. The environment scaling experiment (Figure 8, teacher distillation with 10 envs/3K traj → 80 envs/3K traj → 80 envs/6K traj) demonstrates that environment breadth contributes value beyond what trajectory volume alone provides at fixed total data.

What are the caveats? (1) The environment scaling experiment uses SFT on teacher demonstrations, not RL. The paper's central contribution is RLVR training, but the environment-diversity claim rests on an SFT experiment. The magnitude of the diversity effect under RL—where exploration strategies may interact with environment breadth—is uncharacterized. It could be larger (RL explores more, benefiting more from diversity) or smaller (RL can compensate for narrow environments through more creative exploration). (2) The "scales smoothly" claim for environment diversity is based on three data points (10 envs, 80 envs, 80 envs with doubled data). This is a sparse sampling of the environment-diversity axis, and the paper does not explore intermediate values (e.g., 20, 40, 60 environments) or the interaction between environment count and trajectory count per environment. The claim of "smooth" scaling is inferred from two jumps, not from a densely sampled curve. (3) The paper does not explore which environments contribute most to the diversity benefit. If 80% of the gain comes from adding 5 specific application categories, the environment-diversity axis may be better characterized as "category diversity" or "paradigm diversity" rather than raw environment count. The paper's grounding in O*NET categories and AEI distributions suggests this is their intended interpretation, but the ablation does not disambiguate.

Claim 4: "RL training spontaneously induces multi-action tool calls, compressing trajectories by 33–45%"

Does the experiment demonstrate this? Yes, Figure 9 and the detailed breakdown in Appendix D.1 provide strong evidence that batching emerges during RL and is not present at SFT initialization. The analysis of which actions are batched (deterministic chains) versus which are not (non-deterministic clicks) is thorough and provides mechanistic insight beyond the headline number.

What are the caveats? (1) The causal mechanism is inferred, not tested. The paper attributes batching to "step-budget pressure under group-normalized advantage estimation," but does not run an ablation where the step budget is varied to test whether batching magnitude scales with budget tightness. Without this, alternative explanations (e.g., the RL optimization discovers batching as a generally useful strategy regardless of budget) cannot be ruled out. (2) The 33–45% compression claim is reported in the text without a precise breakdown of which tasks see compression and which do not. The per-step batching distribution (69.6% single-call, 26.7% two-call) suggests the average compression is closer to the lower end of the 33–45% range, but the paper does not compute a rigorous trajectory-level compression statistic. (3) The paper does not report whether the batching behavior transfers to evaluation environments not seen during training (e.g., whether the RL-trained model also batches on WebArena tasks). Since batching is a learned behavior induced by the training environment, its generalization properties are unknown.

Claim 5: "The same checkpoints improve on WebArena, indicating cross-platform transfer"

Does the experiment demonstrate this? Yes, both CUA-GYM-A3B and A17B improve over their bases on WebArena (Table 5b), and WebArena's site clones are disjoint from the 94 CUA-GYM-HUB mock applications. The absolute gains are modest (+3.7 pp and +2.0 pp) but directionally consistent.

What are the caveats? (1) The WebArena improvement is small in absolute terms, and the paper does not report whether the SFT-only checkpoint (teacher demonstrations, no RL) also improves WebArena. If the SFT stage already provides most of the WebArena gain, then the RLVR contribution specifically to cross-platform transfer would be minimal. (2) The WebArena evaluation protocol (number of tasks, task selection criteria) is not described in detail, making it difficult to assess statistical reliability. (3) The claim of "cross-platform transfer" from synthesized mocks to real browser environments would be strengthened by evaluation on additional browser benchmarks (e.g., MiniWoB++, Mind2Web) to test whether the transfer is robust across different website construction styles. The paper evaluates on only one held-out browser benchmark.

Missing Experiments That Would Strengthen the Paper

RL on raw base models without SFT warm-up. All reported RL gains are over an SFT-initialized checkpoint fine-tuned on 3,578 Claude-Sonnet-4-6 teacher demonstrations. The marginal value of RLVR over strong SFT is useful to know, but the total value of CUA-GYM data (SFT + RL) over a completely untrained base is never reported. This matters because the SFT warm-up itself may contribute substantially to the final performance—if the base model is at 40% and SFT lifts it to 54.5%, and RL lifts it to 62.1%, then CUA-GYM's total contribution is +22.1 pp, of which RLVR provides the final increment. Without the base model number, the reader cannot assess this.

Intermediate environment counts in the diversity ablation. The jump from 10 to 80 environments is large, and it leaves open the question of how much diversity is "enough." A curve with 20, 40, 60, 80 environments would reveal whether the diversity benefit is saturating (suggesting that 94 environments may be overkill) or still rising (suggesting that further environment synthesis would be valuable).

RL-based environment diversity ablation. The distillation experiment (Figure 8) provides evidence about environment diversity under SFT, but the paper's core contribution is RLVR. An RL-based environment diversity ablation—even at smaller scale—would provide direct evidence that the diversity axis matters under the training paradigm the paper advocates, not just under supervised imitation.

Multiple random seeds for the main RL results. The single-seed nature of the main results means that the exact percentage gains (62.1%, 72.6%) should be interpreted cautiously. With 360–369 test tasks, the standard error on a single run is approximately 2.5–2.6 percentage points (√(p(1-p)/n) at p ≈ 0.6–0.7). Two runs with overlapping error bars would be insufficient to claim a reliable gap. The paper acknowledges this limitation ("we interpret the results as evidence that verified task and environment scale are valuable... not as a final characterization"), which is honest but does not resolve the statistical uncertainty.

Evaluation on non-OSWorld CUA benchmarks. OSWorld-Verified is the natural evaluation choice given that CUA-GYM tuples follow the OSWorld format, but it creates a potential format-specialization confound. Evaluation on additional CUA benchmarks (e.g., Windows Agent Arena, ScreenSpot, or a live-website benchmark) would test whether the skills learned from CUA-GYM data transfer beyond the OSWorld evaluation ecosystem. The WebArena result is a step in this direction but covers only browser-based tasks, not desktop or cross-application workflows.

Summary of Evidentiary Strength

The paper's strongest claims are those concerning data scaling under RLVR: the Figure 7 experiment cleanly demonstrates that more CUA-GYM tuples → better RL performance, with no saturation and no reward exploitation at the studied scales. This is the paper's core contribution, and the evidence is solid.

The environment diversity finding is intriguing and well-motivated by the O*NET/AEI grounding, but rests on a single SFT distillation experiment with sparse sampling of the diversity axis. It establishes that diversity matters, but not in a way that quantifies the diversity-performance relationship or bounds the value of further environment synthesis investment.

The emergent multi-action batching finding is well-documented behaviorally (Tables 7-8) but causally untested. It demonstrates that the phenomenon occurs, not why.

The cross-platform transfer finding (WebArena) is directionally positive but quantitatively modest and lacks the SFT-only ablation needed to attribute the gain specifically to RLVR.

The absolute performance claims (62.1%, 72.6%) are strong relative to the open-source CUA landscape but subject to single-seed uncertainty and format-specialization concerns. The paper's careful framing ("evidence that... were a substantive bottleneck... not as claims about the pipeline's ultimate ceiling") appropriately scopes the interpretation.

6. Limitations and Trade-offs

Limitation 1: Difficulty Estimation Cost Makes the Compute-Optimal Framework Impractical Without a Cheaper Proxy

The assumption or constraint. The entire compute-optimal allocation framework rests on the ability to estimate prompt difficulty before deciding how to spend the inference budget. The paper's method for doing so — generating 2048 samples per question and averaging either ground-truth correctness (oracle) or PRM final-answer scores (predicted) — is extraordinarily expensive. At 2048 samples per question, the difficulty estimation step alone consumes more compute than the largest test-time budgets studied (256–512 generations). The authors acknowledge this explicitly in Section 3.2:

"estimating difficulty in this way still incurs additional computation cost during inference... our experiments do not account for this cost largely for simplicity"

The consequence. The reported 4× efficiency gains over best-of-N are computed after difficulty is known, without amortizing the cost of learning it. In a realistic deployment, the total cost would be difficulty estimation + strategy execution, and the former could dominate the latter. For any given question, if you must generate 2048 samples just to learn that you only needed 16 generations of beam search, the total cost is 2064 generations — far worse than simply running best-of-256 from the start. The 4× figure should therefore be understood as an upper bound on achievable efficiency under an oracle or near-zero-cost difficulty estimator, not as a realized deployment gain.

What evidence exists in the paper. The paper itself provides no measurement of how much the difficulty estimation overhead erodes the reported gains, because the difficulty estimation cost is excluded from all budget calculations. The predicted difficulty results (Figures 4 and 8) show that the PRM-based difficulty estimator works almost as well as the oracle, but this addresses accuracy, not cost. The PRM-based estimator still requires 2048 samples per question. Section 3.2 flags this as "a key avenue for future work" and Section 8 reiterates the need for "pretraining or finetuning models to directly predict difficulty of a question," but no such model is developed or evaluated.

Mitigation status. The paper does not attempt to mitigate this limitation — it explicitly brackets the difficulty estimation cost and leaves the problem to future work. The authors' proposed direction (training a lightweight classifier to predict difficulty from question text alone) is plausible but unevaluated. An alternative — adaptive difficulty estimation where a small number of initial samples (4–8) provide a rough difficulty signal that determines subsequent allocation — is mentioned in our analysis but not in the paper itself.

Limitation 2: All Results Are on a Single Benchmark (MATH) with a Single Model Family (PaLM 2-S*)

The assumption or constraint. Every result in the paper — search scaling, revision scaling, compute-optimal allocation, FLOPs-matched pretraining comparisons — is evaluated exclusively on the MATH benchmark (500 test questions) using PaLM 2-S* as the base model. The authors state they "believe this model is representative of the capabilities of many contemporary LLMs" (Section 4), but this claim is unverified. Section 8 acknowledges this scope limitation:

"We study only MATH, a single model, and a single model scale. Extending our analysis to other models, benchmarks, and scales is a priority."

The consequence. Several aspects of the findings could be model-specific or benchmark-specific in ways that significantly affect their generality:

• The PRM's quality and over-optimization behavior depend on PaLM 2-S*'s output distribution — specifically, how the base model's solution attempts distribute across difficulty levels, what kinds of errors it makes, and how those errors correlate with step-level features the PRM can detect. A model with very different calibration properties (e.g., one that is more confident but less accurate, or one that makes different types of reasoning errors) could exhibit different difficulty-dependent scaling curves and different over-optimization thresholds.

• The revision model's effectiveness depends on the base model's in-context learning capabilities and its ability to benefit from seeing incorrect attempts before producing a correct one. The paper's training procedure (edit-distance-based pairing of incorrect and correct solutions) is specifically designed for a model that can learn targeted corrections from structurally similar examples. A base model with different in-context learning characteristics might benefit more or less from this procedure.

• MATH consists exclusively of competition-level math problems requiring symbolic reasoning. The difficulty-dependent patterns — beam search hurting easy problems but helping medium ones, sequential revisions dominating on easy problems but balanced ratios being optimal on hard ones — may not generalize to other reasoning domains (code generation, logical deduction, scientific QA, planning tasks) or to tasks requiring factual knowledge rather than inference. For code generation, the nature of "errors" is different (syntax errors vs. logic errors vs. misunderstood requirements), and test cases provide a much richer verification signal than final-answer matching. The optimal allocation of test-time compute might look entirely different.

What evidence exists in the paper. The paper provides no cross-benchmark or cross-model evidence. All confidence in generality rests on the authors' stated belief about PaLM 2-S*'s representativeness, which is not empirically supported. The 500-question test set is split into five difficulty quintiles of ~100 questions each, and the two-fold cross-validation for strategy selection operates on ~50 questions per fold per bin. This is a small sample for selecting the compute-optimal policy, and the selected strategies may not be robust to different test sets or different question distributions.

Mitigation status. Not mitigated. The paper explicitly calls for extension to "other models, benchmarks, and scales" (Section 8) but provides none of this evidence. A practitioner considering deploying compute-optimal test-time scaling on a non-math domain or with a non-PaLM model would need to replicate the entire analysis pipeline (PRM training, revision model training, difficulty binning, strategy sweep, cross-validation) for their specific setting.

Limitation 3: The "14× Larger" Pretraining Baseline Is Not Compute-Optimally Trained and Receives No Test-Time Compute

The assumption or constraint. The FLOPs-matched comparison in Section 7 scales model parameters while holding training data fixed, following the LLaMA paradigm (Touvron et al., 2023) rather than Chinchilla-optimal training (Hoffmann et al., 2022), where both data and parameters are scaled equally. The paper acknowledges this deviation:

"We choose this setting as it is representative of a canonical approach to scaling pretraining compute and leave the analysis of compute-optimal scaling of pretraining compute where the data and parameters are both scaled equally to future work."

Additionally, the larger model uses only greedy decoding — no majority voting, no best-of-N, no beam search, no test-time compute augmentation of any kind.

The consequence. Both design choices weaken the pretraining baseline in ways that make the test-time compute advantages appear larger than they would be against a fairer comparison. A Chinchilla-optimal model trained with 14× more total FLOPs (scaling both parameters and data according to the compute-optimal ratio) would likely outperform a parameter-only-scaled model, since the latter over-allocates to parameters at the expense of training tokens, leaving the model undertrained. This means the reported advantages of test-time compute (e.g., +27.8% on easy questions at low R) may shrink or reverse against a properly compute-optimal larger model.

Similarly, giving the larger model even a modest test-time compute budget — say, best-of-8 or best-of-32 with majority voting — would create a much stronger baseline. The paper's own results show that best-of-N with a verifier provides substantial gains at low budgets (Figure 3). Denying the larger model these gains while giving the smaller model compute-optimal allocation is an asymmetric comparison.

What evidence exists in the paper. The paper is transparent about the parameter-only scaling choice but does not quantify how much it affects the comparison. There is no ablation with a Chinchilla-optimal larger model or with test-time compute applied to the larger baseline. The absence of this evidence means the reader cannot determine whether the reported test-time compute advantage is primarily due to the value of adaptive inference strategies or primarily due to the weak pretraining baseline.

Mitigation status. The paper acknowledges the limitation but does not mitigate it. The choice is described as "representative of a canonical approach to scaling pretraining compute" (which is true for the LLaMA-style models that dominate open-source deployment), but this framing sidesteps the question of whether a fairer comparison would materially change the conclusions. Section 8 does not flag this as a priority for future work.

Limitation 4: Hard Problems (Difficulty Bin 5) Show Near-Zero Improvement Across All Methods and Budgets

The assumption or constraint. The paper's central claim is that test-time compute can amplify existing model capability, but it explicitly acknowledges a hard boundary: "test-time compute cannot create capability from nothing." Section 7's takeaway box states this directly, and the data bears it out starkly. On difficulty bin 5 (the hardest quintile of MATH problems, where the base model's pass@1 is near zero), no method — search, revisions, or compute-optimal combinations — produces meaningful gains regardless of budget. Figure 3 (right) shows bin 5 accuracy hovering at 1–3% for all methods and all budgets. Figure 7 (right) shows bin 5 at roughly 2–3% regardless of the sequential-to-parallel ratio. Figure 9 shows the bin 5 scaling line essentially flat near 0–5% across all test-time compute budgets.

The consequence. If the base model's pass@1 is near zero on a problem class, no amount of search or revision will help — there are no correct solutions in the proposal distribution to find or refine. The entire compute-optimal framework is therefore inapplicable to problems that exceed the base model's fundamental reasoning capabilities, which is precisely the regime where one would most want additional inference compute to compensate. This means that:

• For genuinely novel or out-of-distribution problems that the base model gets wrong for principled reasons (rather than surface-level errors), test-time compute offers no path forward. Only pretraining on more data or larger models can address these.
• The practical deployment value of compute-optimal scaling depends critically on the fraction of a production query distribution that falls into the "easy-to-medium" range (bins 1–4). If a significant fraction of user queries are genuinely hard — requiring capabilities the base model simply lacks — then investing in test-time compute infrastructure will yield diminishing returns.
• Self-improvement pipelines that rely on test-time compute to bootstrap the model's own performance (Section 8's "distilling the outputs of applying additional test-time compute back into the base LLM") are bounded by this same ceiling: if the model cannot produce correct solutions on hard problems even with infinite test-time compute, those problems cannot serve as self-generated training data.

What evidence exists in the paper. The evidence is comprehensive and unambiguous. Every experiment that breaks results down by difficulty bin (Figures 3 right, 7 right, 9) shows bin 5 as a flat line near zero. The FLOPs-matched comparison (Figure 9) shows that on bin 5, pretraining the larger model dominates test-time compute at all values of R, and the gap is large (e.g., -52.9% relative disadvantage for PRM search at high R). The paper is candid about this limitation in the Section 7 discussion.

Mitigation status. The paper does not attempt to solve this — it frames the result as establishing a boundary condition rather than a problem to be fixed. Section 8 suggests that better base models (trained with more pretraining compute) would shift the difficulty distribution, converting some bin-5 problems into bin-4 or bin-3 problems where test-time compute becomes effective. But within the paper's studied regime, the limitation is absolute: test-time and pretraining compute are not 1-to-1 exchangeable, and for the hardest problems, pretraining is unequivocally the better investment.

Limitation 5: Revisions and Search Are Studied Independently, Not Combined, Leaving the Full Potential of the Framework Unexplored

The assumption or constraint. The paper studies two complementary mechanisms — PRM-guided search (verifier optimization) and iterative revisions (proposal distribution modification) — but never combines them. Section 8 explicitly acknowledges this gap:

"we did not experiment with PRM tree-search techniques in combination with revisions"

This is a significant omission because the paper's own framework (Section 2) positions these mechanisms as complementary axes and the empirical results show they have different difficulty-dependent strengths: revisions help most on easy problems, search helps most on medium problems.

The consequence. The current results represent a lower bound on what a fully integrated system could achieve. Several natural combinations are unexplored: (1) using the revision model as the proposal distribution within beam search — at each step of the search tree, the model conditions on previously rejected branches to produce higher-quality candidate steps; (2) using the PRM's per-step scores to guide which revisions to pursue rather than blindly generating a sequential chain — the PRM could detect when a revision is on track versus when to restart from scratch; (3) hierarchical allocation where the compute-optimal policy decides not just between search and revisions but chooses a combined strategy allocating some budget to each.

The paper reports that the PRM trained on base model outputs does not transfer well to revision model outputs due to distribution shift (Appendix J, Figure 15a), requiring a separate ORM trained specifically on revision trajectories. This distribution shift problem would need to be addressed before search and revisions could be combined with a single verifier, but the paper does not explore solutions (e.g., training a unified PRM on a mixture of base and revision model outputs).

What evidence exists in the paper. The paper provides no experimental evidence for combined approaches. The results in Sections 5 and 6 are entirely separate: search results use the base model as the proposal distribution, revision results use the revision model with no tree search. The compute-optimal policies in Figures 4 and 8 select among pure strategies (best-of-N vs. beam vs. lookahead; different sequential-to-parallel ratios) but never consider hybrid strategies. The paper's closing discussion (Section 8) treats the combination as future work.

Mitigation status. Not mitigated. The paper identifies the combination as an important next step but does not take it. A practitioner seeking the strongest possible CUA from a given base model would need to develop the combined search-and-revision methodology independently, including solving the verifier distribution-shift problem.

Limitation 6: The Revision Model Has a 38% Correct-to-Incorrect Reversion Rate, and Attempts to Improve It via On-Policy RL (ReST^EM) Backfire

The assumption or constraint. The revision model is trained exclusively on sequences where all in-context answers are incorrect followed by a correct target. As the paper notes (Section 6.1), this creates a practical problem at inference time:

"since the model was trained only on sequences where all in-context answers are incorrect, at test time the model may encounter correct answers in its context and incorrectly 'revise' them into wrong answers"

The paper reports that approximately 38% of correct answers produced during a revision chain get converted back to incorrect ones in the subsequent step. The mitigation — using majority voting or verifier-based selection across the entire chain rather than always taking the last revision — is a patch, not a solution. It does not prevent the reversion behavior; it only reduces its impact on the final selected answer.

The consequence. The reversion problem fundamentally limits the length of revision chains that can be productively used. Each additional revision step carries a 38% chance of corrupting a correct answer into an incorrect one, creating a random-walk dynamic where longer chains do not monotonically improve. The selection mechanism (picking the best answer from any point in the chain) partially compensates, but it wastes the computation spent on revisions that produce or re-produce correct answers only to lose them. Moreover, the selection mechanism itself introduces variance: if the verifier or majority vote makes an error, a correct answer that was subsequently corrupted will be unrecoverable because the correct version is no longer the chain's terminal state.

The ReST^EM experiment (Appendix K, Figure 16) reveals a deeper fragility. Attempting to further optimize the revision model using on-policy RL (ReST^EM, Singh et al., 2024) caused performance to degrade substantially with sequential revisions. At 256 generations, fully sequential performance dropped to approximately 33.5% compared to roughly 38.5% at the optimal sequential-to-parallel ratio. The paper hypothesizes that "on-policy data collection in ReST^EM exacerbates spurious correlations in revision data, causing the model to fail to learn the revision task properly." This negative result suggests that the revision training methodology is sensitive to the data generation procedure in ways that are not fully understood, and the positive results depend on specific (and somewhat heuristic) choices: offline data construction, edit-distance-based incorrect-correct pairing, and avoiding on-policy data collection.

What evidence exists in the paper. The 38% reversion rate is reported in Section 6.1. The ReST^EM degradation is shown in Appendix K, Figure 16. The paper does not report how the reversion rate changes over the course of RL training — whether RL on CUA-GYM data increases or decreases the tendency to corrupt correct answers — which would be relevant for understanding whether this is a fundamental limitation or a correctable artifact of the training procedure.

Mitigation status. The paper mitigates the reversion problem through selection (majority voting or verifier-based selection across the chain) rather than prevention. This is effective enough to provide the reported sequential-revision gains (Figure 6 right) but does not address the root cause. A more principled solution — such as training the model to recognize when no revision is needed (including examples of correct-to-correct trajectories in the training data), or adding an explicit "keep" action that the model can emit when it judges the current answer to be correct — is not explored. The paper does not discuss whether the reversion rate can be reduced through different training data construction or through the RL training itself, leaving this as an open problem.

7. Implications and Future Directions

How This Work Changes the Landscape

CUA-GYM shifts the CUA post-training bottleneck from data scarcity to data synthesis methodology. Before this work, the field's implicit assumption was that RLVR for computer-use agents was bottlenecked by the inherent difficulty of constructing verifiable training tuples—that writing programmatic rewards for GUI tasks at scale was fundamentally harder than for math or code, where the verification primitives are simpler (equality checks, unit tests). CUA-GYM demonstrates that this difficulty is not inherent but structural: it can be automated by decomposing the problem into adversarially coupled coding sub-tasks with an information barrier. The implication is that the same RLVR recipe that drove breakthroughs in math and code is now mechanically transferable to GUI agents—not through algorithmic innovation but through data-engineering infrastructure that automates the human-expert bottleneck.

The magnitude of this shift is closer to a methodological reframing than a paradigm shift. The paper does not challenge the fundamental GRPO/GSPO optimization framework, nor does it propose a new model architecture or a new RL algorithm. What changes is the feasibility landscape: a pipeline that was previously the province of scarce, expensive human-authored datasets (hundreds of tuples covering a handful of applications) becomes a pipeline that can be run by coding agents at commodity LLM costs (~$0.43 per verified tuple, ~$14K total for the 32K-tuple corpus). This converts CUA RLVR from a boutique capability—available only to organizations with large annotation teams—to a reproducible community resource, provided the synthesis infrastructure is open-sourced (which the paper commits to doing).

The work resolves a latent contradiction in how the field evaluated its own data-generation approaches. Prior to CUA-GYM, the landscape was split between two camps that each satisfied some but not all requirements for CUA RLVR: code-native browser pipelines (GUI-Genesis, InfiniteWeb) achieved programmatic verifiability but at tiny scale and browser-only coverage; VLM-as-a-judge approaches (ZeroGUI, Gym-Anything) achieved scale and coverage but introduced reward noise that the field's own ablations confirmed destabilized RL training. This created a narrative that verifiability and scale were in tension for GUI agents—that you had to choose one or the other. The CUA-GYM pipeline dissolves this tension by showing that automated programmatic reward generation can simultaneously achieve the scale of VLM-based approaches (32K tuples spanning 110 environments) and the deterministic reliability of hand-coded verifiers (as evidenced by the clean RL training curves in Figure 7, without reward-success decoupling). The tension was an artifact of how the generation problem was framed, not a fundamental property of the domain.

The paper redirects research attention toward data-engineering investments rather than algorithmic improvements for CUA post-training. The scaling curves in Figures 7 and 8 establish that current CUA performance is gated by data supply, not by optimization technique. The 12K-tuple run in Figure 7 shows no saturation, and the environment-scaling ablation in Figure 8 shows that environment diversity provides value trajectory volume alone cannot recover. Both findings point in the same direction: investing in better data—more tuples, more diverse environments—will yield higher returns than investing in better RL algorithms at current scales. This is the same inflection point that math and code RLVR reached 12–18 months ago, where the community shifted from "how do we train better?" to "how do we generate more and better training data?" CUA-GYM provides the infrastructure that makes this shift possible for GUI agents.

Certain research directions become less attractive as a consequence. The paper's demonstration that pure outcome rewards (with no process supervision, no step-level verifiers, no reward shaping) produce stable, monotonically improving RL training curves (Figure 7) weakens the case for investing in more sophisticated reward architectures for CUA RLVR. Prior concerns about reward sparsity and credit assignment in long-horizon GUI tasks—which motivated work on step-level verifiers and process reward models—may be less pressing than the field assumed. The GSPO training runs improve smoothly from SFT initialization with no special handling of sparse rewards, suggesting that the combination of a progressive [0, 1] reward function and group-normalized advantage estimation is sufficient for this domain at current scales. The emergent multi-action batching (Section 4.3) further weakens the case for hand-designed action spaces or macro-action engineering—the policy spontaneously discovers efficient action-packing strategies without explicit instruction.

The paper establishes a reusable architectural pattern—adversarial generation with information isolation—that may transfer beyond CUA data synthesis. The pattern is general: when a training instance requires co-generation of interdependent artifacts (setup, golden state, reward) where knowledge of one artifact's construction would corrupt the verification of another, splitting the generation between adversarially coupled sub-systems with an information barrier is a principled solution. The paper does not explore this generality, but the architecture is recognizably applicable to any domain where verification must be derived from task semantics rather than from construction traces. Potential transfers include: synthesizing code-review training data (where the reviewer must not see the author's intentions), generating multi-turn dialogue evaluation rubrics (where the evaluator must not see the dialogue plan), or constructing embodied agent tasks (where the reward designer must not see the environment-construction procedure).

Follow-Up Research This Work Enables

1. Scaling CUA-GYM data volume by one to two orders of magnitude to locate the saturation point of the RLVR data-performance curve. The paper's 12K-tuple RL training run in Figure 7 shows no evidence of saturation—the curve continues rising at the final training step. This leaves open the question of where the curve does saturate: at 30K tuples? 100K? 300K? Running the same GSPO training recipe on progressively larger subsets of the full 32K-tuple corpus (and potentially extending the corpus by running the generation pipeline for more wall-clock time) would characterize the scaling law for CUA RLVR data and determine whether the current 32K-tuple dataset is near-saturating or still in the steep improvement regime. A strong follow-up would report OSWorld-Verified performance at 4–6 data volumes spanning at least one order of magnitude (e.g., 3K, 12K, 32K, 64K, 128K tuples if the pipeline can be scaled that far), fitting a power-law or logarithmic scaling curve to extrapolate the saturation point. This would directly inform community decisions about whether further data synthesis investment is warranted.

2. Characterizing the environment-diversity scaling curve under RL rather than SFT distillation. The paper's environment-diversity evidence comes from a teacher-distillation experiment (Figure 8), not from RL. The interaction between environment diversity and RL exploration is uncharacterized—it is possible that RL benefits more from diversity (because exploration discovers more varied strategies across environments) or less (because RL can compensate for narrow environments through creative exploration within each). A strong follow-up would run the GSPO training recipe at a fixed total trajectory count (e.g., 6K trajectories) across several environment-pool sizes (e.g., 10, 20, 40, 80 environments), measuring OSWorld-Verified performance for each. This would directly test whether the diversity benefit observed under SFT transfers to the paper's core RLVR paradigm, and would locate the point of diminishing returns for environment breadth—answering whether the 94-environment CUA-GYM-HUB suite is over-provisioned or under-provisioned relative to what RL can exploit.

3. Training and evaluating on a held-out application category to test the limits of cross-environment transfer. The WebArena result (Table 5b) provides preliminary evidence that skills acquired on CUA-GYM-HUB mocks transfer to visually and structurally different browser environments. But WebArena covers only four site categories (GitLab, Reddit, Shopping, Map), three of which have close analogs in the CUA-GYM-HUB suite (code hosting, social media, e-commerce). A stronger test would be to hold out an entire application category during training (e.g., exclude all e-commerce mocks from the training set, train on the remaining 80+ environments, then evaluate on a real e-commerce benchmark or on a synthesized e-commerce mock that was never seen during training). This would distinguish between within-category generalization (transferring from one code-hosting mock to another code-hosting site) and cross-category generalization (transferring UI interaction skills to entirely unseen application paradigms). A negative result—where performance on the held-out category fails to improve over the base model—would establish the boundaries of the mock-based training approach and suggest that some application categories require explicit training coverage.

4. Ablating the information barrier to quantify its contribution to reward quality. The paper argues that the adversarial Generator-Discriminator architecture with information isolation is the load-bearing design choice that prevents reward hacking at generation time, and the clean RL training curves (Figure 7) are presented as indirect validation. But the paper never runs the ablation that would directly test this claim: run the same pipeline without the information barrier (i.e., let the Discriminator read the Generator's setup scripts), generate a dataset of equivalent size, train on it, and compare training dynamics and final performance. The prediction from the paper's argument is that the no-barrier dataset would produce RL training curves with oscillation, reward-success decoupling, or lower asymptotic performance—signatures of reward exploitability. A null result (no difference in training dynamics) would challenge the paper's central architectural claim and suggest that the adversarial loop's iterative refinement, rather than the information barrier specifically, is the active ingredient in reward quality. This experiment would be expensive (requires generating a second 32K-tuple dataset and running full RL training on it) but would provide the cleanest test of whether the information barrier is necessary or merely precautionary.

5. Testing whether CUA-GYM-trained policies improve on live-website benchmarks beyond WebArena. The paper's cross-platform transfer evidence is limited to a single held-out benchmark (WebArena) whose site clones, while disjoint from the training mocks, share the same underlying benchmark infrastructure. Evaluating the CUA-GYM-trained checkpoints on a live-website benchmark—where the agent interacts with real, production websites rather than benchmark clones—would test a stricter form of generalization. Candidates include Mind2Web (cross-website task generalization), WebArena's live-website variant (if available), or a newly constructed benchmark of real-world tasks on live sites (e.g., booking a flight on a real travel site, filling a form on a real government portal). The question is whether skills learned on simplified mocks (no authentication, no rate limits, deterministic state) transfer to the messier, non-deterministic reality of production websites. A negative result—where the CUA-GYM-trained model underperforms a model trained on real-website demonstrations—would establish an important boundary on mock-based training and motivate hybrid approaches that combine mock and real-website data.

6. Extending the pipeline to mobile GUI automation. The paper's pipeline is designed for desktop and web environments. Extending CUA-GYM to mobile GUI automation (Android or iOS) would test the generality of the adversarial co-generation architecture and potentially unlock RLVR training for a domain where the data bottleneck is even more acute—mobile app automation tasks are currently almost entirely dependent on human demonstration data. This would require: (1) a mobile environment substrate analogous to the CUA-GYM-HUB mock web applications (potentially using Android emulators with state-injection APIs), (2) adapting the Generator and Discriminator skill files and action primitives to mobile interaction patterns (touch, swipe, pinch, long-press), and (3) adapting the task-generation taxonomy to mobile app categories drawn from app-store taxonomies rather than O*NET occupational categories. A successful extension would demonstrate that the paper's core contribution—the adversarial co-generation architecture—is not specific to desktop/web interaction but generalizes to any GUI domain where task semantics, environment state, and reward verification are co-constraining.

Practical Applications and Downstream Use Cases

1. Cost-efficient CUA model development for organizations with limited annotation budgets. The paper's cost analysis (Appendix F.2) establishes that generating 32K verified RLVR tuples costs approximately $14K in LLM and VM compute—roughly the cost of a single mid-scale training run or a few weeks of a human annotator's time. For a research lab or startup aiming to build a competitive CUA model, this means the data-generation bottleneck is no longer the primary cost driver. The pipeline produces a dataset that can train multiple model variants, support ablation studies, and serve as a community resource. With the full synthesis pipeline, CUA-GYM-HUB environments, and 32K-tuple dataset being open-sourced, an organization can either use the released data directly (for SFT warm-up and RLVR training on their own base model) or run the pipeline to generate additional data tailored to their specific application domains. The paper's 62.1% and 72.6% OSWorld-Verified results for A3B and A17B-scale models establish a strong baseline that a well-resourced team could aim to extend by scaling data volume, environment diversity, or model size.

2. Targeted data generation for vertical CUA applications. The CUA-GYM pipeline's design—with per-application SKILL.md domain files, a feature-taxonomy-driven task sampling matrix, and an environment synthesis pipeline that can produce new mocks for arbitrary target applications—makes it naturally suited for vertical customization. An organization deploying CUAs for a specific domain (e.g., financial services, healthcare administration, legal document processing) could: (1) synthesize mock environments for their domain-specific applications (a claims-processing portal, an EHR system mock, a contract-management interface), (2) run the adversarial co-generation pipeline to produce verified RLVR tuples grounded in those mocks, and (3) fine-tune a base CUA model on this domain-specific data. The paper's environment-scaling result (Figure 8) suggests that adding even a modest number of domain-specific environments (10–20) to a broader training pool would yield improvements that trajectory volume on the general pool alone cannot achieve. This makes CUA-GYM not just a dataset but a data-generation toolkit that can be pointed at new application domains as they become relevant.

3. Bootstrapping self-improvement pipelines for CUA models. The paper's clean RL training dynamics (Figure 7) and lack of reward exploitation at current data scales suggest that CUA-GYM-generated rewards are sufficiently robust to serve as training signals in automated self-improvement loops. An organization could: (1) train an initial CUA model using the released CUA-GYM data, (2) deploy this model to interact with a wider set of environments (including real websites, where programmatic rewards are unavailable but VLM-based or human feedback can provide sparse signals), (3) collect successful trajectories from the deployed model, and (4) use the CUA-GYM pipeline to generate verified RLVR data from those trajectories (by extracting the task instruction and final state, then running the adversarial loop to produce a verified reward function). This would create a virtuous cycle where the model's own successful behaviors become the seed data for generating additional verified training tuples, scaling the dataset without additional human annotation. The paper does not demonstrate this loop, but its pipeline architecture—which separates task specification from reward generation—is well-suited to it, since new task specifications can come from deployed model interactions rather than from the synthetic task generator.

4. Benchmarking and stress-testing CUA robustness. The CUA-GYM corpus's explicit difficulty distribution (45% hard tasks) and cross-application coverage (38% of tasks involve multiple applications) make it a valuable stress-test suite for evaluating CUA model robustness, independent of its use as training data. A model that performs well on standard single-application benchmarks may fail on the long-horizon, multi-step, cross-application workflows that CUA-GYM's hard tasks represent. Organizations developing or deploying CUAs could use the CUA-GYM task corpus as an out-of-distribution evaluation set, measuring not just aggregate success rate but per-difficulty and per-category breakdowns (analogous to Figure 5a's domain-level analysis). The open-sourcing of the complete pipeline means that new evaluation tasks can be generated on demand for applications not covered by the released corpus, providing a mechanism for continuous robustness testing as the model's deployment domain expands.

When to Prefer This Method

The paper does not explicitly position the CUA-GYM pipeline against named alternative data-generation methodologies (beyond the high-level comparison in Table 6c and the related-work discussion in Section 5), so a structured decision matrix with "prefer X when Y" tradeoffs would be speculative rather than paper-grounded. The paper's contribution is the pipeline itself—a specific architectural solution to the structural bottleneck of co-generating consistent (task, setup, reward) tuples—and it is best understood as the first demonstration that automated coding agents can replace human experts for this task at the 32K-tuple scale. Whether future pipelines improve on this architecture (e.g., by replacing the adversarial loop with a different verification mechanism, or by using stronger coding agents that require fewer rounds) does not change the paper's core claim: the data bottleneck is structural rather than inherent, and the adversarial co-generation pattern resolves it.

Where the paper does articulate a clear tradeoff, it is between its approach (programmatic reward generation) and VLM-as-a-judge approaches. The tradeoff is captured by the paper's observation that "VLM-as-a-judge frameworks... introduce reward noise that destabilizes policy optimization" (Section 1), while the CUA-GYM pipeline achieves "deterministic verifiable rewards" but at the cost of requiring synthesized mock environments with unified state APIs rather than operating on real, live websites. This is a verifiability versus environment fidelity tradeoff: prefer CUA-GYM-style programmatic rewards when stable RL training dynamics are the priority and when investment in mock environment synthesis is feasible; prefer VLM-based rewards when coverage of real, unmodified environments is the priority and when reward noise can be tolerated (e.g., in SFT or rejection-sampling pipelines rather than full RL). The paper does not quantify this tradeoff experimentally (it does not train on VLM-reward data as a comparison), but the architectural choice is clear from the design motivations.