ScaleEnv: Scaling Environment Synthesis from Scratch for Generalist Interactive Tool-Use Agent Training

ArXiv: 2602.06820

🎯 Pitch

ScaleEnv automates the creation of fully executable training environments, and scaling the number of synthesized domains from 2 to 16 produces a monotonic improvement in zero-shot agent performance on completely unseen benchmarks—constituting an 'Environment Scaling Law'. Training a Qwen-3-8B model within these environments delivered a +12.5 point gain on the challenging τ²-Bench retail domain.

---

1. Executive Summary

This paper introduces ScaleEnv, a fully automated framework that synthesizes high-fidelity, executable interactive environments and verifiable tasks entirely from scratch for training generalist tool-use agents. Training Qwen-3 models via Group Relative Policy Optimization (GRPO) on these synthesized environments—spanning 16 diverse domains with procedurally tested tools and graph-expanded task states—yields consistent zero-shot generalization improvements across all seven unseen domains of τ²-Bench and VitaBench (e.g., Qwen3-SE-8B gains +12.5 points on τ²-Bench Retail and +9.0 points on VitaBench In-store over its base model). The paper further establishes an Environment Scaling Curve, demonstrating that generalization performance improves monotonically as training domains scale from 2 to 16 while holding task count fixed, with no observed plateau at 16 domains—establishing that environmental diversity functions as a critical, independently scalable axis for robust agent learning even when the evaluation domains are semantically and structurally out-of-distribution from training.

2. Context and Motivation

The Core Problem: Interactive Environments Are the Missing Ingredient for Agent Training

The paper addresses a fundamental bottleneck in training capable LLM agents: the acute scarcity of high-quality, interactive environments for reinforcement learning. As the authors state in the opening paragraph, the field has made rapid progress in scaling pretraining data and model parameters (Kaplan et al., 2020; Grattafiori et al., 2024; OpenAI, 2025), but transforming text generators into autonomous agents requires a qualitatively different training paradigm. Agents must learn to interact with dynamic systems—taking actions, observing outcomes, and iteratively refining their behavior based on environmental feedback. This capability cannot emerge from static text corpora alone; it requires immersion in environments where tools can be called, databases are modified, and the consequences of actions propagate through state transitions.

The significance of this problem is both practical and conceptual. Practically, organizations investing in agentic AI face a chicken-and-egg problem: developing robust agents requires diverse interactive environments, but constructing such environments is labor-intensive, domain-specific, and fundamentally limited by what developers can anticipate. Conceptually, the paper argues that environment scaling represents a third axis alongside data scaling and parameter scaling in the path toward generalist agents—and one that has been systematically underexplored. The authors frame this as a necessary condition for achieving agents that exhibit robust zero-shot generalization rather than brittle template matching.

Why This Gap Has Persisted: The Trilemma of Diversity, Realism, and Scalability

The paper identifies three dimensions that any effective agent training environment must satisfy simultaneously, and shows why existing approaches consistently fail on at least one axis (Section 2.2):

Realism (fidelity of execution and feedback). An environment must provide truthful, deterministic feedback. When an agent calls refund_order(order_id="ORD-502"), the environment must either execute the refund and update the database state correctly, or raise an appropriate error (e.g., the order is already in "shipped" status and cannot be refunded). If the feedback is hallucinated or inconsistent, the agent receives noisy training signals that prevent it from learning precise, logic-grounded decision-making. This requirement is deceptively demanding—it means the environment must maintain internally consistent database states across all tables, enforce preconditions for every tool call, and produce accurate return values for both successful and failed operations.

Diversity (breadth of domains, tools, and task structures). To develop generalist capabilities rather than narrow expertise, agents must be exposed to a wide variety of tools, database schemas, dependency structures, and interaction patterns. Training on a single domain (e.g., airline booking) teaches domain-specific heuristics; training across domains that vary in action space size, state complexity, and inter-tool dependency density teaches transferable reasoning strategies. The paper explicitly argues (Section 5.3) that environmental diversity is an independently scalable axis—increasing the number of training domains while holding task count fixed improves generalization, suggesting that variety matters as much as, or more than, volume.

Scalability (capacity for automated, unbounded generation). For environments to serve as training data at RL scale, their creation must be automated. Manual construction by domain experts is too slow, expensive, and limited in scope. This requirement is particularly acute because RL exploration demands environments that support open-ended interaction: agents will inevitably attempt tool calls that deviate from the optimal trajectory, and the environment must handle these gracefully rather than crashing. This means the environment must populate database tables not just for the "happy path" solution, but for all plausible states an agent might wander into during exploration—a combinatorially larger undertaking than constructing static evaluation benchmarks.

Prior Approaches and Their Specific Failures

The paper organizes existing approaches into three categories, diagnosing the precise failure mode of each (Section 2.2):

Real-world environments (Fang et al., 2025; Xu et al., 2025; Yao et al., 2026) collect actual APIs or remote services. These offer genuine realism—the tools are real, the state transitions are authentic. However, they suffer from a fundamental scalability constraint: the number of accessible real-world APIs is finite, and many are behind authentication walls, rate limits, or restrictive terms of service. More critically, as the authors note:

"the lack of diverse state-altering tasks, combined with prohibitive latency and costs, creates a bottleneck for scalable agent training"

Real APIs are designed for human users performing specific workflows, not for agents exploring arbitrary action sequences. They often lack the write operations (e.g., cancel_order, modify_reservation) that create meaningful state transitions for RL, since production systems typically protect these operations. Safety policies further restrict what actions are permissible, narrowing the explorable action space. The result is an environment that is realistic but impoverished: the agent can query but rarely act, which is insufficient for learning interactive behavior.

LLM-simulated environments (Liu et al., 2024; Chen et al., 2025; Li et al., 2025; Team et al., 2025; Ye et al., 2025) address scalability by outsourcing environment simulation to language models. Rather than executing real code, an LLM is prompted to generate plausible tool responses and database updates based on the agent's actions. This approach is arbitrarily scalable—new domains can be described in natural language rather than implemented in code—and eliminates latency and cost concerns. However, the paper identifies a fatal flaw: hallucination and state inconsistency. LLM simulators "frequently fail to maintain authentic environment states" (Section 2.2). An LLM might generate a plausible-sounding refund confirmation without actually checking whether the order exists in the database, or might produce inconsistent feedback across turns (e.g., first stating that a flight has 5 seats available, then 3 turns later stating it has 7). Kadavath et al. (2022) and Zhang et al. (2025) are cited to establish that LLMs' self-knowledge of their own outputs is unreliable, making them unsuitable as stateful simulators where deterministic correctness is required. The paper's position is that text generation cannot substitute for code execution when training agents that must learn to manipulate ground-truth state.

Existing synthetic environments (Cai et al., 2025, AutoForge; Song et al., 2026, EnvScaler) represent the closest prior work to ScaleEnv—frameworks that programmatically generate executable environments. Both approaches recognize that execution-based verification is essential, but the paper argues they fall short on specific dimensions:

• AutoForge generates environments from external documentation (e.g., API reference pages). This constrains scalability: the diversity of generated environments is bounded by the availability and coverage of documentation. If no documentation exists for a domain, AutoForge cannot generate an environment for it. The paper characterizes this as "the limited scalability of document-based generation."

• EnvScaler synthesizes environments programmatically but "struggles to construct complex, user-interactive tasks." The paper does not elaborate on the specific mechanism of this failure, but the implication is that EnvScaler can generate the infrastructure (tools, databases) but does not reliably produce coherent task specifications where user intent, environment state, and tool availability are aligned.

• Both approaches share a more fundamental weakness: "inadequate consistency between the generated tasks and their corresponding environmental states, undermining the reliability of the resulting sandboxes." This is the central technical challenge ScaleEnv is designed to solve. A task like "book a refundable flight to Chicago for under $400" is only meaningful if the environment actually contains a Chicago flight priced under$400 with a refundable fare class. If the task is generated independently of the environment state—or if the environment state is populated without verifying that the task is solvable—the resulting training data is corrupted. An agent may receive a task it cannot complete, or may find a solution path that was not intended, creating inconsistent reward signals that degrade RL training.

How ScaleEnv Positions Itself

ScaleEnv's positioning is defined by its response to the trilemma above. The paper does not claim to invent environment synthesis from scratch—it acknowledges prior work in both LLM-simulated and synthetic environments. Rather, it claims to be the first framework that simultaneously satisfies all three requirements through a specific architectural innovation: decoupling domain construction from task instantiation, and building execution-based verification into every stage of both pipelines (Section 4).

The key insight is that environment reliability and task coherence can be guaranteed algorithmically rather than probabilistically. The paper rejects the prevailing approach of trusting LLMs to generate correct code and consistent states, and instead treats LLM outputs as hypotheses to be verified—tools are validated through procedural testing, database instances are validated by executing reference trajectories against them, and environment expansion is gated on successful execution. This shifts the role of LLMs from "oracle" to "proposer," with the execution environment serving as the ultimate arbiter of correctness.

The paper also positions environment scaling as a new paradigm for data-centric agent training (Section 5.3, Figure 3). Prior work on tool learning (Section 2.1) has evolved from supervised fine-tuning on static demonstrations (Schick et al., 2023; Qin et al., 2023; Liu et al., 2024) toward reinforcement learning for self-exploration (Luo et al., 2025; Jin et al., 2025; Lu et al., 2025). However, the paper argues this transition has been bottlenecked by the lack of scalable RL environments. ScaleEnv aims to be the environment-generation framework that enables this next stage—providing the diverse, verifiable sandboxes that make large-scale RL exploration feasible for generalist tool-use agents.

A particularly important aspect of the positioning is the strict Out-Of-Distribution (OOD) evaluation regime described in Section 5.2 and visualized in Figure 4. The training domains (16 synthesized environments spanning from Smart Home to Healthcare Telemedicine) are semantically and structurally disjoint from the evaluation domains (τ²-Bench's Retail, Airline, Telecom; VitaBench's Delivery, In-store, OTA). The t-SNE visualization shows clear spatial separation between training and evaluation tool embeddings. This is not merely a robustness check—it is central to the paper's claim that environmental diversity drives generalization rather than memorization. If the gains were attributable to the model learning domain-specific patterns that happen to transfer to evaluation, the scaling curve would plateau as domains saturated the relevant feature space. The fact that performance continues to improve at 16 domains without plateau suggests the model is acquiring genuinely transferable reasoning strategies.

The Broader Research Context

The paper situates itself within a broader shift in how the field thinks about agent training. The traditional pipeline—collect human demonstrations, fine-tune via behavioral cloning, evaluate on held-out instances of the same domain—produces agents that are brittle and domain-bound. The RL-based paradigm that has succeeded in game-playing (AlphaGo, OpenAI Five) and coding (DeepSWE) relies on environments that provide clear reward signals and support unbounded exploration. ScaleEnv can be understood as attempting to generalize this paradigm to the much more open-ended domain of interactive tool use, where the "game" is not a fixed set of rules but rather an arbitrary domain defined by tool schemas and database constraints.

The paper also implicitly engages with the sim-to-real transfer problem, though from an unusual direction. In robotics, sim-to-real concerns whether skills learned in simulation transfer to physical environments with different dynamics and noise characteristics. In the LLM agent context, the "simulation" is the synthesized training environment and the "real" is the unseen evaluation benchmark. The paper provides evidence that transfer is robust—the t-SNE visualization and the consistent gains across all 7 evaluation domains suggest that the synthesized environments are diverse enough to cover the reasoning patterns needed in evaluation, even though the specific tools and schemas differ.

What the Paper Explicitly Does NOT Address

The paper is transparent about scope limitations that are relevant to understanding its positioning:

• It does not claim to solve the problem of training agents for domains that lie outside the base model's knowledge. The domain synthesis process starts from a keyword (e.g., "Job Seeking") and relies on the LLM's world knowledge to generate plausible tool schemas. Domains requiring specialized or esoteric knowledge that the LLM does not possess cannot be synthesized.

• It does not investigate whether the synthesized environments are faithful to real-world tool behavior. A synthesized "Airline" domain may have tools, schemas, and constraints that differ from actual airline APIs. The paper's claim is about internal consistency (the tools work correctly with respect to their own defined schemas), not about external fidelity (whether they match real APIs).

• It does not provide a theoretical model of why environmental diversity improves generalization. The Environment Scaling Curve is an empirical observation, not a derived law. The paper hypothesizes that diversity forces the model to learn "abstract, domain-agnostic reasoning strategies" (Section 5.3) but does not attempt to identify what these strategies are or provide a mechanistic account of how scaling diversity induces them.

• It does not address the cost of environment synthesis in the context of the total RL training budget. Table 9 reports token consumption (~546k tokens per domain, ~93.2k tokens per task), but the reader must infer that synthesis cost is amortized across many training episodes and is therefore negligible relative to the RL exploration cost.

3. Technical Approach

3.1 Reader Orientation

This paper describes ScaleEnv, an automated pipeline that generates fully functional, code-verified interactive environments—complete with executable tools, populated databases, and solvable tasks—for training LLM agents via reinforcement learning. The core problem it solves is the scarcity of high-fidelity training environments for interactive tool-use agents: existing approaches either lack realism (LLM simulators hallucinate state), lack diversity (real-world APIs are finite and restrictive), or lack coherence (synthetic environments produce tasks that don’t match their database states). The solution takes the shape of a two-phase, execution-gated generation pipeline that first builds a domain’s logical skeleton (tools, databases, and their dependencies) and then expands that skeleton into rich, verifiable task instances by sampling dependency-constrained tool chains and iteratively populating database states to support both the intended solution and arbitrary exploration.

3.2 Big-Picture Architecture (Diagram in Words)

The system has five major components arranged in two sequential phases:

Phase 1: Executable Graph Construction

1. Schema Definition Module — takes a domain keyword (e.g., "Job Seeking") and produces formal tool interface specifications (tool schemas) and derived database schemas with integrity constraints.
2. Code Implementation and Procedural Testing Module — generates executable Python code for databases and tools, then validates correctness by running generated unit tests against matched database instances; failures trigger an iterative debug-then-retest loop.
3. Tool Dependency Graph Builder — analyzes pairwise relationships between verified tools (data flow, pre/post-conditions, shared tables) to construct a directed graph capturing valid execution orderings.

Phase 2: Task Instantiation via Graph Expansion 4. Task Initialization Module — samples executable seed tool chains from the dependency graph, constructs an initial database state that supports the chain’s execution, injects distractor records, and synthesizes a grounded user instruction. 5. Controlled Environment Expansion Module — iteratively expands the initial state by adding dependency-satisfiable tools from the graph and their required database entries, gating further expansion on complexity metrics and an LLM-based feasibility oracle, with a fallback mechanism to ensure minimum exploration space.

Information flows as follows: domain keyword → Schema Definition → Code Implementation + Procedural Testing → Tool Dependency Graph → Seed Chain Sampling → Task Initialization (database state + user instruction) → Controlled Environment Expansion → final verifiable task instance. Each stage produces artifacts that are validated by execution before feeding into the next stage.

3.3 Roadmap for the Deep Dive

• First, the formal problem setup (Section 3, "Preliminaries")—what a domain foundation, environment, and task are, and how agent interaction is modeled as a POMDP. This establishes the vocabulary and constraints that the synthesis pipeline must satisfy.
• Second, Executable Graph Construction (Section 4.1)—how tool and database schemas are defined, how code is generated and verified via Procedural Testing, and how the Tool Dependency Graph is built. This is the "skeleton" that ensures code-level reliability.
• Third, Task Initialization with Seed Tool Chains (Section 4.2.1)—how the first seed task is created by sampling a chain from the graph, building a supportive database state, and synthesizing user instructions grounded in that state. This establishes the base unit of task construction.
• Fourth, Controlled Environment Expansion (Section 4.2.2)—how the minimal task environment is expanded into a rich, interactive sandbox through dependency-aware topological expansion and LLM-gated chain extension, with specific complexity metrics and feasibility gates. This is where interaction completeness is achieved.
• Fifth, the Reward Specification (Section 4.1.1)—the rule-based evaluation mechanism that compares final database states using domain-specific matching policies, and why it’s chosen over LLM-as-a-judge.

3.4 Detailed, Sentence-Based Technical Breakdown

This is a systems/methods paper that presents a fully automated framework for generating code-verified, interactive training environments and verifiable tasks. Its core idea is that environment reliability, task coherence, and interaction completeness can be guaranteed by making code execution—not LLM text generation—the arbiter of correctness at every stage of the synthesis pipeline.

---

Formal Problem Setup: Domains, Environments, Tasks, and the POMDP (Section 3)

The paper formalizes the synthesis target before describing the pipeline. This is essential because each subsequent component’s design choices are driven by the requirements of the formal model.

A domain foundation is defined as:

$\mathcal{B} = \langle \Sigma, \mathbb{T} \rangle$

where $\Sigma$ is a set of database schemas (defining the valid space of environment states $\mathcal{S}^{env}_{valid} = \{s^{env} \mid s^{env} \models \Sigma\}$), and $\mathbb{T}$ is the set of executable tools (functions/APIs) available in that domain.

What it defines: The domain foundation specifies the entire possibility space for a given domain—what states can exist and what operations can be performed. It does not specify any particular state; it only defines the rules. This is analogous to defining the rules of chess (piece movements, board structure) without setting up a specific board position.

Why this decoupling matters: By separating the domain definition (what is possible) from environment instantiation (what is actual), the framework can generate arbitrarily many different environments from a single domain foundation—each with different database contents, different users, and different tasks—while guaranteeing that all generated environments respect the same structural constraints. This is the key architectural decision that enables scalable, diverse task generation.

An environment is defined as:

$\mathcal{E} = \langle s^{env}_0, \mathbb{T} \rangle$

where $s^{env}_0$ is a specific, populated database instance (conforming to $\Sigma$) that serves as the initial hidden state, and $\mathbb{T}$ is the tool set inherited from $\mathcal{B}$. The environment $\mathcal{E}$ represents the external world at episode start.

What it defines: The environment is a concrete instantiation of a domain foundation—a specific database filled with specific rows, paired with the full tool set. This is the "board position" the agent finds itself in at the start of an interaction.

A task binds an environment to a specific user with specific goals:

$\psi = \langle \mathcal{E}, u, P_{user} \rangle$

where $u$ is the user intent (hidden goal), and $P_{user}$ is the user profile containing all necessary contextual information (permissions, location, history, preferences).

Why the user profile is separate from intent: The user profile $P_{user}$ provides the factual context that constrains what is permissible (e.g., "this user can only book flights in economy class"), while the intent $u$ specifies what needs to be accomplished. Separating them allows multiple tasks to share the same underlying user profile but differ in goals, and vice versa.

Agent interaction as a POMDP. The paper models the interaction between an LLM agent and the environment as a Partially Observable Markov Decision Process. This formalization is critical because it clarifies what information the agent does and does not have access to—specifically, the environment state $s^{env}_t$ is hidden from the agent and must be inferred through tool call results and user feedback.

The POMDP is defined as:

$\mathcal{M} = \langle \mathcal{S}, \mathcal{A}, \mathcal{O}, \mathcal{T}, \mathcal{R} \rangle$

where:

• $\mathcal{S}$ is the state space. Each state $s_t = (s^{env}_t, h_t, u)$ bundles three components: the current environment state $s^{env}_t$ (hidden database contents), the interaction history $h_t$ (all previous agent actions and environment/user responses), and the user intent $u$ (the hidden goal). The history is the only component visible to the agent; the environment state and user intent must be inferred.
• $\mathcal{A} = \mathcal{A}_{resp} \cup \mathcal{A}_{tool}$ is the action space, partitioned into natural language responses (chat messages to the user) and tool execution commands (function calls to $\mathbb{T}$).
• $\mathcal{O} = \mathcal{O}_{resp} \cup \mathcal{O}_{tool}$ is the observation space, partitioned into user feedback (natural language replies from a user simulator) and tool execution results (return values from tool calls).
• $\mathcal{T}: \mathcal{S} \times \mathcal{A} \rightarrow \mathcal{S} \times \mathcal{O}$ is the state transition function. The behavior depends on action type: a tool action $a_t \in \mathcal{A}_{tool}$ deterministically updates both $s^{env}_t$ (the database is modified by the tool execution) and $h_t$ (the action and its return value are appended to history). A response action $a_t \in \mathcal{A}_{resp}$ updates only $h_t$ (the user simulator's reply is appended) while $s^{env}_t$ remains unchanged.
• $\mathcal{R}$ is the reward function, producing a scalar outcome $r = \mathcal{R}(s^{env}_T, u)$ at termination that evaluates whether the final database state satisfies the user intent.

Why this POMDP structure drives the synthesis requirements: The partial observability—the agent cannot see $s^{env}_t$ or $u$—means that training environments must support information-gathering actions (tools that query database state) and must maintain state consistency across arbitrarily many information-gathering steps. The deterministic transition function for tool actions means that the environment must behave identically each time a given tool is called with the same arguments on the same state—LLM-based simulators violate this because they may hallucinate different results on repeated calls. The observation space partition means that both tool execution and user feedback must be generated reliably: tool execution through actual code, user feedback through a separate simulator (the paper uses Qwen2.5-72B-Instruct for this purpose; Section 5.1).

---

Executable Graph Construction: Schema Definition and Code Implementation (Sections 4.1.1 and 4.1.2)

This phase builds the reusable domain skeleton. Starting from a single domain keyword (e.g., "Job Seeking"), the system produces three artifacts: formal tool and database schemas, verified executable code for both tools and databases, and a tool dependency graph. The defining characteristic of this phase is that every artifact is validated by actual code execution before being accepted.

Tool Schema Synthesis (Section 4.1.1)

The process begins with a top-down synthesis approach:

1. An LLM receives the domain name and is prompted to conceptualize the domain logic and generate the Tool Schema—a formal specification of all tools $\mathbb{T}$ in the domain. The schema includes precise functional descriptions, parameter lists, and logical pre/post-conditions. The paper gives the example that submit_application logically necessitates a preceding upload_resume—these dependencies are encoded in the schema and serve as the basis for later dependency graph construction and task coherence verification.

2. The schema is not merely descriptive; it is structured to support subsequent automated verification. Each tool's interface is defined with enough rigor that a code generator can produce an implementation and a test generator can produce valid and invalid input cases.

Why top-down rather than bottom-up: The alternative would be to start from database schemas and derive tools from them. The top-down approach (tools first, databases derived) ensures that the tools reflect meaningful user-facing operations rather than arbitrary database operations. The tool set is designed around what a user might want to accomplish; the database is then designed to support those operations.

Database Schema Derivation and Mapping (Section 4.1.1)

Once the tool schemas exist, a separate Database Agent analyzes them to reverse-engineer the required database structure:

1. The agent examines each tool's parameters and functional description to infer what tables and fields must exist. For example, the presence of a submit_application tool implies the existence of an Application table and a referenced Job table in the database.

2. Through de-duplication and filtering across all tools, the agent produces a consolidated Database Schema for $\mathcal{S}^{env}_{valid}$—the complete set of table structures and integrity constraints (primary keys, foreign keys, data types, default values) that define the valid state space.

3. Simultaneously, the agent establishes a tool-database mapping: an explicit specification of which tables are read from or written to by each tool. This mapping is used in two downstream contexts: (a) it guides the code generator in implementing tool logic that correctly interacts with the database layer, and (b) it is one of the dimensions used to construct the Tool Dependency Graph (tools that share tables have a dependency relationship).

Why derive the database from tools rather than designing both independently: This ensures structural alignment—there are no database tables that no tool ever touches (unnecessary complexity) and no tool that references non-existent tables (an execution error). The derivation process guarantees that the database schema is both necessary (every table supports at least one tool) and sufficient (every tool's data requirements are met).

Database Implementation and Verification (Section 4.1.2)

With the database schema defined, the system generates executable code:

1. An LLM translates the database schema into executable code (Python classes with field definitions, type constraints, and foreign key relationships). The paper shows a concrete example in Appendix D, Listing 1, where the JobSeekingDB class contains typed dictionaries for job_application, application_note, application_stage, interview_schedule, and interview_feedback, each with specific field definitions and constraints.

2. Concurrently, the system generates test scripts that validate the database implementation against integrity constraints (e.g., foreign key referential integrity, data type enforcement, required field presence).

3. The test scripts are executed. Any execution failure triggers a Debug Agent that analyzes error tracebacks and iteratively refines the database code. The loop continues until all tests pass.

Why generate tests concurrently with code rather than after: The paper does not elaborate, but the logic is consistent with the overall philosophy: by generating tests simultaneously, the system ensures that the test logic is aligned with the schema definitions and not retroactively fitted to whatever code was produced. This prevents the degenerate case where tests are weakened to match buggy code.

Tool Implementation via Procedural Testing (Section 4.1.2)

This is the most architecturally significant mechanism in the framework. The paper frames tool code generation as fundamentally unreliable when done directly:

"generating valid tool code is a non-trivial process involving intricate logic and interactions across multiple databases, and direct generation is prone to hallucination"

The Procedural Testing mechanism addresses this by making execution the gatekeeper:

1. A Code Agent implements the tool logic based on the tool schema and the verified database code. This produces candidate tool implementations (e.g., the delete_job_application and archive_old_applications functions shown in Appendix D, Listing 2, with their argument parsing, database access, state mutation, and return value construction).

2. A Test Agent simultaneously synthesizes unit test cases and matched database instances. These database instances are specifically populated to test both successful execution paths and error paths—they contain the records needed for a successful operation (e.g., an application ID that exists in the database for testing deletion) and deliberately omit records needed for error path testing (e.g., an application ID that does not exist).

3. The tool code is executed against the matched database instances. The system evaluates the outcome against three distinct result categories:

   • Success: The execution completes without error, and the resulting database state transitions exactly match the expected states defined in the test case. For example, after delete_job_application("APP-001"), the database must contain all previous records except the one with ID "APP-001," and the return value must contain the correct deletion confirmation with timestamp.

   • Anticipated Rejection: The tool correctly identifies invalid inputs and raises the pre-defined exceptions specified in the schema. For example, calling delete_job_application("NONEXISTENT-ID") must raise a KeyError with a specific message format, not a generic Python error or a silent failure.

   • Unexpected Failure: Any runtime error or state inconsistency that does not fall into the anticipated rejection category indicates a defect. In this case, a Debug Agent receives the error logs and iteratively rectifies either the tool implementation (if the logic is wrong) or the database instance (if the test data violates constraints) until the procedural test is satisfied.

Why three outcome categories rather than binary pass/fail: Binary pass/fail would not distinguish between "the tool correctly rejects bad input" and "the tool crashes on bad input." The three-category system ensures that tools handle edge cases gracefully and predictably—the agent in training will inevitably call tools with invalid arguments during exploration, and the environment must respond with meaningful error feedback rather than crashing. This is a direct consequence of the Interaction Completeness requirement: the environment must handle the full action space, not just the optimal trajectory.

Why this mechanism is the central reliability guarantee: The Procedural Testing loop ensures that every tool in the deployed environment has been demonstrated to work correctly on concrete database instances before any RL training begins. Unlike LLM-based simulation where correctness is probabilistic ("the model usually generates plausible responses"), code that passes procedural testing is deterministically reliable for the covered paths. The limitation, which the paper implicitly acknowledges, is that procedural testing covers only the test cases generated by the Test Agent—there may be untested edge cases. However, the subsequent Environment Expansion phase (Section 4.2.2) provides additional coverage by exercising tools on the constructed environment states.

---

Tool Dependency Graph Construction (Section 4.1.3)

With verified tools in place, the system constructs a directed graph that captures executable ordering constraints:

1. A Tool Dependency Agent systematically evaluates pairwise relationships between all verified tools along three dimensions:

   • Data flow: Does the output of tool A serve as an input parameter to tool B? For example, search_jobs returns job IDs that are required as input to submit_application.
   • Pre/post-conditions: Do the logical prerequisites of tool B require that tool A has been executed first? For example, submit_application logically requires that upload_resume has been called.
   • State dependencies: Do tools A and B read from or write to shared database tables? Two tools that both modify the job_application table have a state dependency—their execution order matters for the final database state.

2. Based on this analysis, the agent establishes directed edges representing causal links between tools, consolidating the atomic tools into a unified Tool Dependency Graph $G$.

What the graph represents: Each node is a tool; a directed edge from tool A to tool B means that A's output or side effects are required for B to execute meaningfully. The graph is not required to be acyclic—it represents all possible valid tool sequences, and some sequences may loop (e.g., repeatedly searching for and applying to jobs).

Why construct this graph explicitly rather than letting tasks be generated freely: The graph serves as the feasibility constraint for all subsequent task generation. When the Task Initialization module samples tool chains (Section 4.2.1), it uses $G$ to ensure that the sampled sequence respects dependency constraints. Without this graph, LLM-generated tool chains might include impossible sequences (e.g., submit_application followed by search_jobs using the returned application ID, when submit_application requires a job ID that hasn't been obtained yet). The graph provides a structural guarantee that generated trajectories are logically coherent.

---

Task Initialization with Seed Tool Chains (Section 4.2.1)

This is the first stage of Phase 2, where the reusable domain skeleton (tools, databases, and their dependency graph) is instantiated into a concrete, verifiable task. The process has three sub-stages:

Executable Seed Tool Chain Sampling

The system generates a seed tool chain $C_1 = (a_1, a_2, \dots, a_k)$—a sequence of tool calls that represents a valid reference solution to the task to be constructed. Crucially, the tool chain is formulated as executable code:

1. An LLM is prompted with the Tool Dependency Graph $G$ and the relevant database schema. It generates a code snippet that calls the tools in sequence, with concrete arguments.

2. Representing the chain as executable code achieves two things simultaneously: (a) it forces the LLM to specify actual parameter values rather than abstract descriptions, and (b) it inherently satisfies data flow constraints—the output of tool $a_i$ is programmatically propagated as the input to tool $a_{i+1}$ because the code defines variable bindings.

Why represent the chain as code rather than as a declarative sequence: A declarative specification (e.g., "Step 1: search for jobs matching 'software engineer', Step 2: submit application to job ID J-123") leaves parameter binding ambiguous—there is no guarantee that job ID J-123 actually exists in the database or was returned by Step 1. The executable code forces joint modeling of the tool sequence and its arguments, where the LLM must explicitly bind outputs to inputs, making the chain's internal consistency verifiable by running it.

Initial State Construction with Distractor Injection

Given the verified seed chain $C_1$, the system constructs an initial environment state $s^{env}_0$ that supports the chain's execution:

1. An LLM-based generation pipeline synthesizes a database instance $s^{env}_0$ populated with all records required for $C_1$ to execute successfully—the job posting must exist, the application table must be writable, the user profile must have appropriate permissions, etc.

2. Validation: The synthesized $s^{env}_0$ is validated by actually executing $C_1$ against it. If the chain executes successfully and produces the expected final state, the initialization is accepted. If execution fails, the state is iteratively patched.

3. Distractor injection: The system populates database tables with additional records that act as distractors. For example, alongside the target job posting, the job_listing table may contain 20 other postings with similar but not identical titles; alongside the target application, the application table may contain records for other applicants. The density of these distractors is dynamically scaled according to predefined task complexity.

Why distractors are essential for RL training: Without distractors, the environment contains only the records relevant to the correct solution. The agent would learn to simply read all available records and use them—a degenerate strategy that exploits the sparsity of the environment rather than learning to filter relevant from irrelevant information. Distractors force the agent to discriminate between functionally relevant and functionally orthogonal data, which is the core reasoning challenge that generalizes to real-world tool use.

Why the density scales with task complexity: The paper mentions this but does not elaborate on the scaling function. The rationale is presumably that harder tasks should require more sophisticated information filtering, so they receive proportionally more distractors. A simple task ("find my most recent application") might have few distractors; a complex task ("find all applications submitted in the last month that are still under review and schedule follow-up interviews") would have many.

Instruction Synthesis

With the verified seed chain $C_1$ and environment state $s^{env}_0$ in place, the system generates the user-facing components:

1. An LLM synthesizes the user profile $P_{user}$ (containing information like name, permissions, location, preferences) and the user instruction $u$ (the natural language expression of the intent).

2. Grounding constraint: The LLM is explicitly constrained to generate $u$ that is grounded in $C_1$ as the reference solution. This means the instruction must be answerable by executing $C_1$, and no aspect of the instruction may reference entities, states, or operations that are not supported by $C_1$ and $s^{env}_0$. The paper states this prevents "the introduction of external priors or hallucinations unsupported by the underlying environment."

3. The reward specification $\mathcal{R}$ is derived directly from the final state after executing $C_1$: $s^{env}_{gt}$ (the ground-truth environment state) serves as the evaluation target. This ensures exact alignment between the seed chain (the intended solution), the instruction (what the user asks for), and the reward (how success is measured).

Why this three-way alignment is the core guarantee of task coherence: In prior work (AutoForge, EnvScaler), tasks and environments were generated with weaker coupling, leading to cases where the task was unsolvable in the provided environment or where multiple solutions produced different final states. By deriving the instruction from the chain and the reward from the chain's final state, ScaleEnv ensures that there is at least one known correct solution, that the instruction accurately describes what that solution achieves, and that the reward function correctly identifies success.

---

Controlled Environment Expansion (Section 4.2.2)

The task initialization stage produces an environment that supports the seed chain but is sparse—the database contains only the records needed for the reference solution plus distractors, and the action space is artificially narrow. RL training on such sparse environments encourages the agent to memorize the single correct trajectory rather than learning to explore. Controlled Environment Expansion addresses this by iteratively enriching the environment until it supports open-ended interaction while preserving task solvability.

The process operates in two nested loops: first, expanding around the seed chain $C_1$ to create a local subgraph; second, introducing additional seed chains $C_2, \dots, C_n$ to diversify the supported trajectories, gated by a learned expansion policy.

Dependency-Aware Topological Expansion

Starting from the seed chain $C_1$, the system constructs a local subgraph $\mathcal{H}_1 = K(C_1) \subset G$ through a constrained expansion algorithm:

1. Initialization: $\mathcal{H}_1 = C_1$ (the subgraph initially contains only the tools in the seed chain).

2. Dependency-Aware BFS: The system iteratively traverses the dependency graph $G$ and considers adding new tool nodes $v \in G$ to $\mathcal{H}_1$. A node $v$ is eligible for addition if and only if:

   • All of $v$'s input dependencies (tools whose outputs $v$ requires as parameters) are already present in $\mathcal{H}_1$.
   • All of $v$'s state dependencies (tables that $v$ reads from) are populated by tools already in $\mathcal{H}_1$.

Why this constraint is essential: The paper explicitly warns against "naive stochastic injection of tools," which "risks introducing dependency dead-ends—nodes whose prerequisite inputs cannot be satisfied by the current available tool output." If a tool $v$ requires a parameter that no tool in $\mathcal{H}_1$ can produce, then $v$ cannot be meaningfully executed—it would always fail or produce garbage output. Including such dead-end nodes in the environment would create training episodes where the agent attempts a tool call and receives an error that is not informative about its action quality but rather about an environment artifact. This would inject noise into the RL signal.

3. Execution and refinement: For each newly added tool node $v$, the system executes it with arguments derived from $\mathcal{H}_1$ (synthesized to be valid given the existing environment state) and refines the environment if any errors arise. The existing database state is expanded with any new records needed for $v$ to execute successfully, and any bugs in $v$'s implementation that were not caught by the initial Procedural Testing (because the test cases didn't cover this particular argument configuration) are patched.

4. The result is $\mathcal{H}_1$, a subgraph of $G$ containing $C_1$ and all dependency-satisfiable tools that can be reached from it, with a database state $s^{env}_0$ that has been expanded and verified to support arbitrary execution of any tool in $\mathcal{H}_1$.

What this achieves: Entity Consistency is preserved (because the expansion only adds tools whose dependencies are satisfied) and Interaction Completeness is achieved for the subgraph (because the database has been populated to handle valid calls to any tool in $\mathcal{H}_1$). The agent can now explore freely within $\mathcal{H}_1$ without encountering dead ends.

LLM-Gated Chain Expansion

Since $\mathcal{H}_1$ is derived from a single seed chain $C_1$, the diversity of supported trajectories is limited. The system attempts to extend this by introducing additional seed chains $C_2, C_3, \dots, C_n$ and expanding each similarly. However, this expansion cannot proceed indefinitely because:

• Each new chain should introduce genuinely new tools, not retread tools already in the existing subgraph $\mathcal{H}_n = \bigcup_{i=1}^{n} K(C_i)$.
• The set of candidate tools $\mathcal{D}_n = G \setminus \mathcal{H}_n$ shrinks as expansion proceeds.
• At some point, the remaining tools in $\mathcal{D}_n$ may not support any coherent chain, or the chains they support may be too trivial to add meaningful diversity.

To decide when to stop expanding, the system employs a parametric gating policy $\pi$ implemented via an LLM:

1. Input metrics: The LLM receives three quantitative inputs about the current state of the environment:

   • Number of available tools: $|\mathcal{D}_n|$, the count of tools in the dependency graph that have not yet been incorporated into the expanded subgraph.

   • Structural Complexity of the current subgraph:

     $c(\mathcal{H}_n) = \frac{|V_{\mathcal{H}_n}| + \lambda|E_{\mathcal{H}_n}|}{S_{\text{sat}}}$

     where $V_{\mathcal{H}_n}$ is the number of tool nodes in the subgraph, $E_{\mathcal{H}_n}$ is the number of directed edges (dependencies) between those tools, $\lambda = 0.5$ is a weight that down-weights edges relative to nodes (dependencies are counted as half as important as the tools themselves), and $S_{\text{sat}} = 50$ is a saturation constant.

     What it computes: A normalized score representing how structurally rich the current subgraph is. Tools contribute individually (through $|V|$); their interconnections contribute with reduced weight (through $\lambda|E|$). The denominator $S_{\text{sat}} = 50$ sets the scale—when the numerator reaches 50, the complexity score is 1.0, indicating saturation.

     Why nodes are weighted more than edges ($\lambda = 0.5$): This reflects the intuition that adding a genuinely new category of tool (a node) contributes more diversity than adding a new dependency between existing tools (an edge). The edge count can grow quadratically with the node count, so without the down-weighting, a dense subgraph of a few tools would score higher than a sparse subgraph of many diverse tools.

   • Feasibility Score: $g(\mathcal{D}_n) \in [0, 1]$, the success rate of an "Oracle" agent in identifying executable tool chains within the remaining candidate tools $\mathcal{D}_n$. The oracle is instantiated as Qwen3-235B-A22B augmented with a best-of-k search strategy (with $k = 16$ rollouts). Intuitively, if the oracle can reliably find valid chains in $\mathcal{D}_n$, there is still value in expanding; if it consistently fails, the remaining tools likely cannot support coherent tasks.

2. Gating decision: The LLM receives a prompt $\pi(|\mathcal{D}_n|, c(\mathcal{H}_n), g(\mathcal{D}_n))$ and is instructed to "balance the trade-off between diversity and solvability." The model outputs a compatibility score $p \in [0, 1]$. If $p \geq \tau$ (where $\tau$ is a threshold; the paper does not specify its value), the system samples a new seed chain $C_{n+1} \subseteq \mathcal{D}_n$ and repeats the expansion process:

   • Expand $C_{n+1}$ into its dependency subgraph $K(C_{n+1})$ using the same Dependency-Aware BFS procedure.
   • Execute tools in $K(C_{n+1})$ and refine the environment.
   • Merge into the cumulative subgraph: $\mathcal{H}_{n+1} = \mathcal{H}_n \cup K(C_{n+1})$.

3. Minimum exploration constraint: The system enforces a hard minimum $|\mathcal{H}_n| \geq 20$ on the final subgraph size. If the gating policy terminates before this threshold is reached, the system randomly samples valid auxiliary chains from $\mathcal{D}_n$ and merges their dependency subgraphs to satisfy the constraint. This ensures sufficient exploration space even when the LLM-based gating is conservative.

Why use an LLM-based gating policy rather than a simple threshold on complexity: The decision of whether to continue expanding depends on a nuanced trade-off. A pure threshold on $c(\mathcal{H}_n)$ would ignore whether the remaining tools can form coherent chains—it might keep expanding into a set $\mathcal{D}_n$ that contains many tools but no valid multi-step sequences. The feasibility score $g(\mathcal{D}_n)$ provides this signal, but combining it with $c(\mathcal{H}_n)$ and $|\mathcal{D}_n|$ into a single decision requires balancing competing factors that an LLM, trained on reasoning tasks, can approximate better than a hand-tuned heuristic.

What this entire expansion procedure achieves: The final subgraph $\mathcal{H}_n$ and its associated environment state $s^{env}_0$ satisfy both critical requirements. Entity Consistency is guaranteed because every tool added to the subgraph had its dependencies verified at addition time. Interaction Completeness is guaranteed because the execution-and-refinement step populates the database to handle all valid tool calls within $\mathcal{H}_n$. The agent can explore freely, attempt any sequence of tools in $\mathcal{H}_n$, and always receive meaningful, deterministic feedback. This is the key differentiator from LLM-simulated environments, where exploration outside the intended trajectory produces hallucinated responses.

---

Reward Specification (Section 4.1.1)

The paper explicitly rejects the LLM-as-a-judge paradigm for reward computation in favor of a deterministic rule-based evaluator. This is a consequential design choice motivated by two problems with LLM-based rewards:

"it often suffers from high computational overhead and vulnerability to reward hacking"

The rule-based evaluator operates by directly comparing the agent's final database state $s^{env}_T$ against the ground-truth state $s^{env}_{gt}$ (obtained by executing the seed chain $C_1$). However, different types of data require different comparison criteria—exact character-level matching for a price is appropriate, but exact matching for a free-text comment is unnecessarily strict. The paper introduces a three-tier classification of database columns:

1. Exempt Fields: Dynamically generated IDs (e.g., UUIDs, auto-incrementing primary keys) and optional columns that do not affect task success. These are excluded from the comparison entirely.

2. Hard Constraints: Critical data such as timestamps, quantities, prices, and status codes that require strict character-level or numerical equality. If the ground truth has status = "confirmed" and the agent produced status = "confirmed", the comparison passes; any deviation fails.

3. Semantic Alignment: Descriptive text fields (e.g., feedback comments, notes) that require fuzzy semantic matching. These fields are compared using an embedding-based similarity metric rather than exact string equality, allowing for paraphrased but semantically equivalent content.

Why this classification is necessary for robust RL: A naive exact-match reward would penalize an agent that wrote "The application was submitted successfully" when the ground truth was "Application submitted successfully"—a difference in phrasing that is irrelevant to task completion. This would create spurious negative reward signals that discourage the agent from generating natural language content. Conversely, a purely fuzzy reward would fail to penalize an agent that booked a flight for $350 when the requirement was "under$300" but the semantic meaning of "$50 over budget" might be judged as "close enough" by an LLM evaluator. The tiered approach applies strictness where precision matters and flexibility where semantics matter.

Why rule-based over LLM-as-a-judge: The LLM-as-a-judge paradigm (where an LLM reads the agent's trajectory and outputs a reward) is widely used because it can handle open-ended outputs without requiring structured ground truth. However, the paper cites Gabor et al. (2025) and Pan et al. (2024) to establish vulnerability to reward hacking—agents learn to generate outputs that trigger high reward scores from the judge LLM rather than outputs that actually satisfy the task. The rule-based approach eliminates this attack vector because the reward is computed deterministically from the database state: the agent either modified the correct records or it didn't. The computational overhead argument is also significant for RL at scale—each episode in GRPO with group size $G$ requires $G$ reward computations, and replacing LLM inference with database queries reduces this cost by orders of magnitude.

Design implication: The suitability of rule-based rewards is a direct consequence of the framework's commitment to executable environments. Because every task is grounded in a database state with a verifiable ground truth, the reward can be computed deterministically. This would not be possible in open-ended tasks without structured state representations, which is why the POMDP formalization (Section 3) commits to database states as the representation of $s^{env}$.

---

Summary of Design Choices and Their Justifications

• Phase decoupling (domain construction separate from task instantiation): Enables arbitrarily many diverse tasks per domain while amortizing the cost of domain construction across all tasks. Ensures that domain-level reliability (verified tools, consistent schemas) is established once and reused.
• Procedural Testing with three outcome categories: Guarantees deterministic reliability for tool execution on covered paths and ensures graceful error handling for invalid inputs—both essential for RL exploration.
• Tool Dependency Graph as the feasibility backbone: Provides a structural constraint that prevents hallucinated tool sequences during task generation. Every sampled chain is guaranteed to be internally consistent.
• Executable seed chain (tools + arguments as code): Forces joint modeling of action sequences and their parameter bindings, eliminating the disconnect between "what tool to call" and "what arguments to pass."
• Distractor injection with dynamic density: Prevents the agent from exploiting environment sparsity and forces information-filtering capabilities that generalize.
• Dependency-Aware BFS for subgraph expansion: Guarantees that every tool added to the environment is actually executable with the current database state, preventing dead ends during exploration.
• LLM-gated chain expansion with three-dimensional input (tools available, structural complexity, feasibility score): Balances diversity against solvability using learned heuristics while providing hard guardrails (minimum subgraph size).
• Tiered reward classification (Exempt, Hard, Semantic): Applies precise matching where it matters (prices, quantities, status codes) and flexible matching where natural variation is acceptable (descriptions, comments), avoiding both false negatives and reward hacking.
• Rule-based rewards over LLM-as-a-judge: Eliminates reward hacking vulnerability and reduces computational cost, enabled by the framework's commitment to structured, verifiable state representations.

4. Key Insights and Innovations

Innovation 1: Environment Synthesis as a Verification Problem, Not a Generation Problem

The most fundamental conceptual move in this paper is reframing environment synthesis from "generate tools and databases using LLMs" to "generate candidate tools and databases, then verify them by executing them." This shift seems subtle—after all, every synthesis pipeline generates, and many test—but the paper makes verification the central organizing principle rather than a final quality check. The Procedural Testing mechanism doesn't validate tools after they're built; it's the construction process itself. A tool isn't "generated and then tested"; it's "generated, executed, debugged, re-executed" in a loop where execution failures are the primary feedback signal for refinement.

Prior work on synthetic environments treated verification as a filtering step: generate many candidates, score them with an LLM or a heuristic, keep the good ones. AutoForge (Cai et al., 2025) validates generated environments but the generation process itself is driven by documentation coverage, not by execution feedback. EnvScaler (Song et al., 2026) produces executable environments but the paper argues it struggles with task-environment coherence—precisely the failure mode that execution-gated generation would catch. LLM-simulated environments (Liu et al., 2024; Li et al., 2025) bypass verification entirely, trusting the LLM to produce consistent responses—a trust the paper argues is fundamentally misplaced.

The significance of this reframing extends beyond the specific mechanism. It establishes a design principle with broad applicability: when generating artifacts that must interact with deterministic systems (databases, compilers, physics engines), the generation process should be structured around the execution feedback loop, not around producing "plausible-looking" outputs. This principle is not new to software engineering—test-driven development and continuous integration embody it—but applying it to LLM-driven synthesis, where the generator's outputs are probabilistic and often hallucinated, is a distinctive contribution. The paper essentially argues that LLMs should be demoted from "oracle" to "proposer" in synthesis pipelines, with execution environments serving as the ground-truth arbiter.

The evidence for why this matters comes from Table 3, the ablation on Executability Verification (EV). When tools are synthesized without the procedural testing loop and environment states are not patched through iterative execution, performance degrades consistently across all τ²-Bench domains. The training data in the "w/o EV" condition contains "tool calls that appear semantically plausible but fail during runtime due to unsatisfied preconditions or mismatched database states." The resulting "noisy rollouts introduce conflicting reward signals, preventing the policy from learning precise, logic-grounded decision-making." This is not merely a performance difference—it's evidence that unverified environments teach the agent the wrong thing, reinforcing behaviors that look correct linguistically but fail logically.

Innovation 2: The Environment Scaling Curve as an Empirical Discovery

The paper's second distinctive contribution is the empirical characterization of environmental diversity as an independently scalable axis for agent training, formalized through what the authors call the Environment Scaling Curve (Figure 3). This insight is not a method but a finding about how agent capabilities develop, and it carries implications for resource allocation in agent training that parallel the impact of scaling laws for pretraining.

Prior work in tool learning has operated under an implicit assumption that more training data improves performance, but has not systematically investigated whether the variety of environments matters independently of the volume of tasks. The paper's Domain Scaling Analysis (Section 5.3) controls for this by fixing the total number of tasks (1024) and varying only the number of unique domains (N = 2, 4, 8, 16). The result—monotonically improving zero-shot generalization across both τ²-Bench and VitaBench as domain count increases, with no plateau at N=16—demonstrates that diversity has an effect orthogonal to quantity. Adding more tasks from the same domain would not produce the same generalization improvement as adding an equivalent number of tasks distributed across new domains.

This finding challenges a common intuition in data-centric ML: that more data from broader distributions is inherently better, but that the benefit eventually saturates. The absence of a plateau at 16 domains suggests the saturation point for environmental diversity is much further out than might be expected, making environment scaling a particularly high-leverage investment. It also provides an empirical counterpoint to the view that agents primarily benefit from domain-specific fine-tuning—the consistent transfer to semantically and structurally OOD domains (visually confirmed in Figure 4's t-SNE plot) suggests the model is acquiring genuinely domain-agnostic reasoning strategies, not just broader coverage of surface patterns.

The significance of this finding is amplified by its practical implications. If environmental diversity is a more efficient lever than task quantity per domain for improving generalization, then the economics of agent training shift: invest in generating many diverse domains with modest task counts rather than exhaustively populating a few domains with many task variants. This is precisely what ScaleEnv's architecture enables—domain construction is a fixed cost amortized across tasks—and the scaling curve validates that this architecture's design priorities are aligned with what actually drives generalization.

It is worth noting that this is an empirical observation, not a theoretical law. The paper does not provide a mechanistic account of why diversity improves generalization (e.g., whether it forces the model to learn abstract planning strategies, meta-cognition about tool selection, or robustness to distribution shift in tool descriptions). The Environment Scaling Curve is an important starting point for that investigation, but it leaves open the question of what specific capabilities are being acquired and why variety induces them. This distinguishes it from pretraining scaling laws (Kaplan et al., 2020; Hoffmann et al., 2022), which are backed by theoretical models of loss decomposition. The Environment Scaling Curve is more akin to an early-stage empirical regularity—compelling and actionable, but awaiting theoretical grounding.

Innovation 3: Graph Expansion as a Formalization of Environment Completeness

The paper introduces a precise, operationalizable definition of what it means for a training environment to be "complete" for RL, and then provides a constructive algorithm for achieving that completeness. This is a conceptual contribution that addresses a problem most prior work either ignores or handles ad hoc.

The two criteria—Entity Consistency and Interaction Completeness—may initially seem like obvious requirements. Of course training environments should be internally consistent and support arbitrary exploration. But the paper's contribution is not stating the requirements; it's providing the first formal characterization that is tightly coupled to the synthesis pipeline. Entity Consistency is not "the environment makes sense" but specifically "an entity appearing in one table must map correctly to corresponding entities in related tables"—a property that can be verified by foreign key validation and foreign key traversal testing. Interaction Completeness is not "the agent can do anything" but specifically "for any valid tool calling action, the environment must return a valid, semantically meaningful observation"—a property that can be guaranteed constructively through the Dependency-Aware BFS expansion algorithm.

Prior work treated environment completeness as a post-hoc quality metric—generate an environment, then evaluate whether it's consistent and complete. AutoForge and EnvScaler both suffer from "inadequate consistency between generated tasks and their corresponding environmental states," which the paper attributes to the absence of a constructive completeness guarantee in their pipelines. The Dependency-Aware BFS algorithm (Section 4.2.2) provides this guarantee: by only adding tools whose input and state dependencies are already satisfied by tools in the current subgraph, and by executing each added tool to populate the database with its required records, the algorithm ensures that every tool in the final environment is executable on the final database state. Completeness is not something you check for; it's something the construction process ensures.

This is a fundamental shift from verification-checking to verification-constructing, and it represents a design pattern with applicability beyond agent environments. Any system that must generate a complex artifact with guaranteed internal consistency—program synthesizers, database schema generators, API composition tools—could adopt a similar approach: define the dependency structure, then expand the artifact iteratively with a constraint that each addition's prerequisites are satisfied by what already exists.

The evidence that this matters comes indirectly from the Domain Stability Analysis (Table 5). Training on two different, non-overlapping subsets of domains produces consistent improvements over the baseline across all VitaBench domains. This stability suggests that the synthesis pipeline reliably produces high-quality environments regardless of which specific domains are generated—a direct consequence of the constructive completeness guarantee. Without it, some environments would be high-quality and some would have dead ends, and the results would be sensitive to which subset was chosen.

Innovation 4: Decoupling Domain Construction from Task Instantiation as an Architectural Principle

At first glance, the two-phase architecture—constructing a domain foundation first, then instantiating tasks from it—appears to be an engineering convenience: build once, reuse many times. But the paper elevates this to an architectural principle with deeper implications for scalability and reliability. The key insight is that the requirements for domain-level correctness (tools are bug-free, schemas are consistent, dependencies are valid) are qualitatively different from the requirements for task-level correctness (the environment state supports the instruction, distractors don't invalidate the solution, all valid tool calls return meaningful observations). By decoupling them, the paper can apply different verification strategies to each: procedural testing with deterministic execution for domain correctness, and graph-based expansion with iterative state patching for task correctness.

This decoupling is not just about modularity; it's about separating concerns that have different failure modes and different verification costs. Domain-level errors (a tool with a bug, a database with a missing foreign key constraint) would corrupt every task generated from that domain, making them catastrophic failures that must be caught early. Task-level errors (a distractor that accidentally provides a shortcut, an instruction that is underspecified) affect individual training episodes and are less catastrophic but harder to detect because they require reasoning about semantic alignment between instructions and database states. The two-phase architecture applies rigorous, execution-based verification to the high-stakes domain layer and more flexible, expansion-based verification to the lower-stakes task layer.

Prior approaches conflated these concerns. LLM-simulated environments generate tools and tasks in the same probabilistic process, making both equally vulnerable to hallucination. AutoForge's documentation-driven approach ties domain construction to available documentation, constraining domain diversity independently of task diversity. The paper's architecture decouples them entirely: domain construction is driven by LLM world knowledge (what tools would a "Job Seeking" domain need?), while task instantiation is driven by graph sampling and state expansion. This means the diversity of domains and the diversity of tasks can be scaled independently—you can have many domains with few tasks each, or few domains with many diverse tasks each, depending on what the scaling analysis suggests is optimal.

The practical consequence is that the architecture makes the Environment Scaling Curve actionable. If the optimal strategy for generalization is to maximize domain diversity (which the curve suggests), the two-phase architecture supports this directly: invest in generating many domain foundations, then generate a modest number of tasks per domain. If the optimal strategy were to maximize task diversity within fewer domains, the architecture would support that equally well. The architecture doesn't prescribe the allocation; it makes the allocation a tunable parameter rather than a structural constraint.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The paper uses two evaluation benchmarks: τ²-Bench (Barres et al., 2025) and VitaBench (He et al., 2025). τ²-Bench contains three domains—Retail, Airline, and Telecom—where agents must engage in multi-turn dialogue while adhering to lengthy textual policy documents. VitaBench contains four domains—Cross, Delivery, In-store, and OTA (Online Travel Agency)—designed to test ambiguous user needs requiring proactive information retrieval and complex multi-step planning. Both benchmarks are used exclusively for evaluation; the paper's 16 training domains are entirely disjoint from these evaluation domains (Section 5.2).

• Base model(s). The paper trains two model sizes: Qwen3-8B and Qwen3-32B (Yang et al., 2025). Both are instruction-tuned base models. The Qwen3-SE (ScaleEnv) variants are produced by fine-tuning these base models via GRPO on ScaleEnv-synthesized environments. The 8B model is trained with a rollout batch size of 1024 and the 32B model with a rollout batch size of 2048. Both models are trained for 48 steps with a learning rate of 1 × 10⁻⁶. The choice of Qwen3 is motivated by it being a strong open-weights model family with available 8B and 32B variants that represent different capability tiers (Section 5.1).

• Metrics. The primary metric is task success rate (accuracy), measured as the fraction of tasks where the agent's final output satisfies the ground-truth evaluation criteria specific to each benchmark. For τ²-Bench, success is determined by whether the agent's actions comply with domain policies and satisfy user requirements. For VitaBench, success is determined by whether the agent correctly resolves the user's ambiguous intent through appropriate tool use. Additionally, the paper reports Pass@4 for VitaBench (Table 2), which measures the probability of generating at least one correct solution within four independent attempts—this captures the model's upper-bound performance when allowed multiple tries. Results in Table 4 are reported as Avg@4, indicating average accuracy across four attempts rather than the pass rate.

• Baselines. The paper evaluates against a comprehensive set of open-weights models (Table 1): GPT-OSS-120B-A5B, Qwen3-235B-A22B-2507, Kimi-K2-0905, Seed-OSS-36B, and xLAM-2-32B-fc-r. For the ablation studies, the key baselines are: (1) "w/o EV" — training on environments synthesized without the execution-based verification step (tools are never subjected to actual parameter-driven execution; environment states are not iteratively patched); and (2) LLM-as-a-Judge — using an LLM-based reward evaluator instead of the rule-based evaluator (Table 4). The base Qwen3-8B and Qwen3-32B models serve as the zero-shot baselines for the Qwen3-SE variants, allowing isolation of the training effect.

• Generation budget / compute accounting. The paper does not frame results in terms of a per-task generation budget in the way that test-time compute scaling papers do. Instead, the "budget" is the total RL training compute: each model is trained on a fixed set of 1024 tasks across N domains (where N ∈ {2, 4, 8, 16} in the scaling analysis), with the same number of GRPO training steps (48) and the same batch sizes. The compute comparison is between different training data distributions (more domains vs. fewer domains) at fixed total training compute, not between different inference-time strategies. Token consumption for environment synthesis is reported in Table 9 (~546k tokens per domain foundation, ~93.2k tokens per task) but this cost is not folded into the training budget comparison.

• Cross-validation / statistical protocol. There is no cross-validation or formal statistical significance testing reported. The Domain Scaling Analysis (Section 5.3, Figure 3) sweeps N from 2 to 16 but each data point represents a single trained model per configuration—there are no error bars or confidence intervals on the scaling curves. The Domain Stability Analysis (Table 5) tests two distinct subsets of domains (Set A and Set B, each containing 4 domains) as a robustness check, but this serves as a qualitative consistency verification rather than a statistical protocol. The paper does not report multiple random seeds or variance across training runs.

Main Quantitative Results

Zero-Shot Generalization to Unseen Domains

The headline result is that training on ScaleEnv-synthesized environments produces consistent, cross-domain zero-shot improvements over base models across all seven evaluation domains, spanning both τ²-Bench and VitaBench (Table 1).

For the 8B model scale:

• Qwen3-SE-8B achieves 50.9% on τ²-Bench Retail compared to Qwen3-8B's 38.4% (+12.5 points absolute, a 32.6% relative improvement).
• On τ²-Bench Airline: 37.5% vs. 30.5% (+7.0 points).
• On τ²-Bench Telecom: 27.2% vs. 21.5% (+5.7 points).
• On VitaBench Cross (the most challenging domain with ambiguous user intents): 3.0% vs. 1.5% (+1.5 points—note the low absolute performance indicating substantial room for improvement).
• On VitaBench Delivery: 26.3% vs. 18.3% (+8.0 points).
• On VitaBench In-store: 23.8% vs. 14.8% (+9.0 points).
• On VitaBench OTA: 7.0% vs. 4.5% (+2.5 points).

For the 32B model scale:

• Qwen3-SE-32B achieves 63.6% on τ²-Bench Retail vs. 59.5% (+4.1 points, a more modest 6.9% relative improvement—the base 32B model already performs substantially better than the 8B model, leaving less headroom).
• On τ²-Bench Airline: 48.0% vs. 48.0% (+0.0 points—the only domain without improvement).
• On τ²-Bench Telecom: 30.9% vs. 27.2% (+3.7 points).
• On VitaBench Cross: 10.8% vs. 5.3% (+5.5 points, more than double the base model).
• On VitaBench Delivery: 31.3% vs. 27.0% (+4.3 points).
• On VitaBench In-store: 34.5% vs. 22.5% (+12.0 points, the largest absolute gain for the 32B model).
• On VitaBench OTA: 12.5% vs. 4.5% (+8.0 points).

Several observations from these results:

• Gains are uniformly positive across 13 of 14 domain-model-size comparisons (the sole exception being τ²-Bench Airline at 32B, which shows no change, not a degradation).
• Gains are larger at the 8B scale than at the 32B scale, consistent with the interpretation that smaller models benefit more from environment-driven RL training because they have less emergent tool-use capability from pretraining.
• The absolute magnitude of improvement varies substantially across domains, from as low as +0.0 points (Airline, 32B) to as high as +12.5 points (Retail, 8B). This variation is not explained in the paper but may relate to how well the synthesized training domains' reasoning patterns transfer to each specific evaluation domain.
• On the hardest VitaBench domain (Cross), both models show very low absolute performance even after training (3.0% for 8B, 10.8% for 32B). This domain tests "ambiguous user needs requiring proactive information retrieval"—tasks like inferring "recommend light food" when the user says "I am sick"—and the low scores suggest that the synthesized environments may not adequately cover this class of latent-intent reasoning.

Performance Upper Bound Analysis (Pass@4)

Table 2 reports Pass@4 on VitaBench, revealing that ScaleEnv training elevates the model's capability ceiling even when individual attempts fail:

• On the Cross domain, Qwen3-32B's Pass@4 improves from 5.3% (base) to 10.8% (Qwen3-SE-32B)—from approximately 0.053 to 0.108, which is more than double. The paper states: "in the complex cross-domain subset, our method nearly doubles the success potential."
• A similar pattern holds across other VitaBench domains, though specific Pass@4 values for all domains are not enumerated in the main text.
• The improvement in Pass@4 relative to Pass@1 (not explicitly reported) indicates that the model learns not just to produce correct solutions more consistently, but also to have correct solutions somewhere in its generation distribution, even for tasks where it typically fails. This suggests that ScaleEnv training broadens the model's exploration capabilities even when exploitation (picking the right answer) remains difficult.

Domain Scaling Analysis

Figure 3 presents what the paper calls the "Environment Scaling Curve"—the relationship between the number of training domains and zero-shot generalization performance, holding the total number of tasks fixed at 1024. The key findings are:

• Monotonic improvement: Performance on both τ²-Bench (Figure 3b) and VitaBench (Figure 3a) improves steadily as the number of training domains increases from N=2 to N=16. At N=0 (the base model with no ScaleEnv training), τ²-Bench Pass@4 starts at a baseline. At N=2, there is a visible jump. The curve continues to rise through N=4, N=8, and N=16.

• No observed plateau at N=16: The paper explicitly states: "performance has not yet fully plateaued at N=16." This is significant because it suggests that the saturation point for environmental diversity—the point at which adding more domains no longer improves generalization—is beyond the range tested. The implication is that investing in more diverse environments would continue to yield returns.

• The scaling trend holds across both benchmarks: The upward trajectory is consistent for both τ²-Bench (which tests policy-following and constraint adherence) and VitaBench (which tests ambiguous intent resolution and proactive reasoning), suggesting that the acquired capabilities are broad rather than benchmark-specific.

• Task count is controlled: Because the total number of tasks is fixed at 1024, increasing the number of domains necessarily decreases the number of tasks per domain (1024/16 = 64 tasks per domain at N=16 vs. 1024/2 = 512 tasks per domain at N=2). The fact that performance improves despite fewer tasks per domain directly supports the paper's central claim that environmental diversity matters independently of task quantity.

Specific values are not quoted in the text for every N, but the figure (Figure 3) shows Pass@4 scores on τ²-Bench rising from roughly where the base model sits (at N=0) upward through N=16. The paper does not provide a table of exact values for each N, which makes precise comparison of the magnitude of gains at each step difficult.

Ablation: Executability Verification

Table 3 reports the ablation of execution-based verification (EV), comparing Qwen3-8B trained with and without the Procedural Testing and environment state patching loop. The results are reported as Avg@4 on τ²-Bench domains:

• Removing EV causes consistent degradation across all domains of τ²-Bench. The paper states: "the removal of execution verification leads to a consistent degradation in performance across all domains."
• Specific numbers are not quoted in the main text for individual domains, but the trend direction and consistency across all domains are clearly stated.
• The paper attributes the degradation to the training data containing "tool calls that appear semantically plausible but fail during runtime due to unsatisfied preconditions or mismatched database states." In the specific example given, an agent attempts to "refund a non-existent order"—the LLM-generated tool sequence looks plausible but the database doesn't contain the referenced order, so execution fails. The agent receives an error reward through no fault of its strategy, creating a noisy training signal.

Ablation: Reward Mechanism

Table 4 compares the rule-based evaluator against the LLM-as-a-Judge paradigm, averaged across the three τ²-Bench domains (Retail, Airline, Telecom):

• The rule-based approach yields superior performance across all metrics reported in Table 4.
• The paper attributes this advantage to two factors: (1) LLM-based judges are "susceptible to reward hacking, where agents optimize for linguistic alignment at the expense of logical correctness," and (2) the rule-based approach "minimizes computational overhead by replacing expensive LLM inference with efficient rule-based verification."
• The computational overhead argument is particularly important for RL at scale: each GRPO training step requires reward computation for multiple rollout trajectories, and LLM-based reward computation for each trajectory would multiply this cost substantially.

Domain Stability Analysis

Table 5 addresses the concern that performance gains might stem from specific "lucky" domain choices rather than from the synthesis framework's general robustness:

• Two non-overlapping subsets of synthesized domains are tested: Set A (wedding planning, knowledge management, job seeking, healthcare telemedicine) and Set B (express logistics, job seeking, email management, pet care). Note that "job seeking" appears in both sets, which is an oddity—this may be a typo in the paper or there may be multiple variants of the job seeking domain.
• Both subsets consistently outperform the baseline across all VitaBench metrics reported in Table 5.
• The paper interprets this as evidence that "EnvZero produces high-fidelity environments reliably across diverse scenarios," though the actual name "EnvZero" appears to be an internal designation for the framework that differs from the paper's title—this inconsistency is not explained.
• This consistency across arbitrary domain subsets strengthens the claim that the synthesis pipeline, rather than particular domain choices, drives the observed improvements.

Ablation Studies and Robustness Checks

• Executability Verification (EV) removal (Table 3): Training without procedural testing and state patching consistently degrades performance across all τ²-Bench domains because unverified tools produce runtime failures—not just missing rewards, but actively misleading training signals where agents learn to make tool calls that appear correct but fail at execution time. This is the most critical ablation because it directly tests the paper's central claim that execution-based verification is essential.

• Reward mechanism (Table 4): Rule-based rewards outperform LLM-as-a-Judge, with two distinct failure modes for the LLM approach: reward hacking (agents learn to satisfy the judge rather than the task) and computational cost (LLM inference for every reward computation at scale). This ablation is important because LLM-as-a-Judge is the prevailing paradigm in much of the RLHF and agent training literature—the paper provides evidence that it is suboptimal when structured state representations enable deterministic evaluation.

• Domain stability (Table 5): Training on two different random subsets of 4 domains each produces consistently better-than-baseline results across all VitaBench domains tested. This rules out the possibility that the results are driven by a single high-quality domain or an unusually favorable domain combination. However, the absence of error bars or multiple random seeds means we cannot quantify the variance—we can only observe that two specific subsets both outperform the baseline.

• Model scale comparison (Table 1): The improvements from ScaleEnv training are not limited to a single model size—both 8B and 32B variants show gains, with larger absolute improvements at the 8B scale. This suggests the method is robust to model capacity, though the diminished gains at 32B (including the zero-gain Airline result) may indicate ceiling effects where larger models already possess substantial tool-use capability from pretraining.

• OOD verification (Figure 4, Appendix A): The t-SNE visualization shows clear spatial separation between the training domains' tool embeddings and the evaluation domains' tool embeddings. This is not an ablation in the traditional sense but serves as a crucial robustness check: if the training and evaluation domains were semantically overlapping or adjacent in embedding space, the generalization results could be explained by interpolation rather than genuine transfer. The clear separation provides evidence that the observed improvements come from the agent learning abstract reasoning strategies that transfer across semantic gaps, not from memorizing tool patterns that happen to match evaluation domains.

• Environment scaling (Figure 3): Running the same experiment with N=2, 4, 8, 16 domains while holding the total task count at 1024 serves as an implicit ablation on the question "do more domains help, or just more data?" By controlling for total task count and observing monotonic improvement with domain count, the experiment shows that diversity per se drives generalization, not just the volume of training data.

Critical Assessment

Does the paper genuinely demonstrate zero-shot generalization to unseen domains?

The paper's primary claim—that training on ScaleEnv environments improves performance on unseen, OOD benchmarks—is supported by the consistent, across-the-board improvements in Table 1. These are genuinely zero-shot: the training domains (16 synthesized environments) are disjoint from the evaluation domains (7 benchmark environments from τ²-Bench and VitaBench), and the t-SNE visualization in Figure 4 confirms semantic and structural separation. The gains are not marginal (e.g., +12.5 points on τ²-Bench Retail at 8B) and appear across both benchmarks and both model scales.

However, there are important caveats. First, the gains are variable in magnitude across domains and relatively modest on the hardest tasks (VitaBench Cross improves from 1.5% to 3.0% at 8B—still an extremely low success rate). The paper frames this as a positive ("nearly doubles the success potential") but the absolute performance remains far from usable, and it's not clear that further environment scaling would close this gap given that the Cross domain tests latent-intent reasoning that may not be well-represented in the synthesized training environments.

Second, the comparison is between ScaleEnv-trained models and their base model counterparts at zero-shot. The paper does not compare against models fine-tuned on alternative training data (e.g., SFT on human demonstrations, SFT on LLM-generated trajectories, RL in LLM-simulated environments), which would provide a more informative baseline. The base model comparison establishes that ScaleEnv training is better than nothing, but does not establish that ScaleEnv is better than alternative approaches to agent training. The open-weights baselines in Table 1 (GPT-OSS-120B, Qwen3-235B, etc.) are provided for context but are not matched on training data or compute.

Third, the generalization claim is limited to tool-use benchmarks with structured API interactions. ScaleEnv synthesizes environments that follow the pattern of "call tools, observe results, call more tools"—a pattern shared by τ²-Bench and VitaBench. The paper does not test whether the learned capabilities transfer to different interaction paradigms (e.g., web navigation, code execution, physical reasoning) or to tasks requiring different reasoning primitives.

Does the Environment Scaling Curve genuinely show that diversity drives generalization?

The Domain Scaling Analysis (Figure 3) is the paper's most novel empirical contribution, and it credibly demonstrates that increasing domain count while holding task count fixed improves generalization. The monotonic upward trend and the absence of a plateau at N=16 both support the claim that diversity is an independently scalable axis.

However, several aspects of this experiment warrant scrutiny:

• N=2 to N=16 is a small range for establishing a scaling law. The paper frames this as "environment scaling" in analogy to data and parameter scaling laws, but 2 to 16 domains is a narrow window compared to scaling laws that span orders of magnitude. The fact that the curve hasn't plateaued at N=16 could mean there are substantial further gains to be had, or it could mean the curve would plateau at N=20 and the current window is simply too narrow to observe the saturation point. Without extrapolation beyond N=16, the "scaling curve" is more of a "scaling trend."

• There are no error bars or confidence intervals on Figure 3. Each data point represents a single trained model. Without replication across random seeds, we cannot distinguish genuine scaling effects from noise—particularly at small N where the specific domains selected (which change between N=2, 4, 8, 16) could drive performance variation independently of the number of domains. The Domain Stability Analysis (Table 5) provides some reassurance that different domain subsets produce consistent improvements, but it tests only N=4 and does not quantify variance.

• The domains are not randomly sampled from a larger distribution—they are the 16 domains that the synthesis pipeline produced. The scaling curve shows what happens when you add these particular domains, which may not generalize to arbitrary domain additions. If the synthesis pipeline tends to produce domains with certain characteristics (e.g., similar dependency graph structures, similar abstraction levels), the curve may reflect saturation in the pipeline's output diversity rather than in the model's capacity to benefit from diversity.

• Task count is held fixed at 1024, which means that as N increases, tasks per domain decrease (from 512 at N=2 to 64 at N=16). The paper interprets the performance improvement as evidence that domain diversity matters more than task quantity per domain. But an alternative interpretation is that 64 tasks per domain is sufficient for learning, and 512 tasks per domain provides diminishing returns—the improvement at larger N could be due to avoiding overfitting to a small number of domains rather than to the inherent value of diversity. An experiment that varied tasks per domain independently of domain count (e.g., N=4 with 256 tasks vs. N=8 with 256 tasks) would disambiguate this.

Is the ablation on Executability Verification convincing?

The EV ablation (Table 3) shows that removing execution-based verification degrades performance, which supports the claim that verification matters. However, the ablation conflates two distinct effects:

1. Tools are not tested for correctness → tools may have bugs → agents receive noisy tool execution feedback.
2. Environment states are not patched to ensure solvability → tasks may be impossible → agents receive negative rewards regardless of their actions.

These are different failure modes (buggy tools vs. unsolvable tasks), and the "w/o EV" ablation removes both simultaneously. A more informative ablation would isolate them: train on environments where tools are verified but states are not patched (testing the importance of solvability guarantees alone), and train on environments where states are patched but tools are not verified (testing the importance of tool correctness alone). The current ablation establishes that the combination matters, but does not tell us which component is more important or whether both are necessary.

Additionally, the paper does not report the failure rate of tools in the "w/o EV" condition—we don't know what fraction of tool calls actually failed. If the failure rate is high (e.g., 50% of tool calls produce errors), the ablation is unsurprising: of course training on broken environments hurts. If the failure rate is low (e.g., 5%), the degradation is more interesting because it suggests that even occasional failures are sufficient to disrupt RL training through noisy reward propagation.

Are there missing experiments that would strengthen the paper?

Several experiments would substantially strengthen the paper's claims but are absent:

1. Comparison to SFT on static demonstrations. The paper frames ScaleEnv as an RL environment generator, but RL is only one training paradigm—does ScaleEnv provide benefits over simply fine-tuning on demonstrations of correct tool-use trajectories? A baseline where the base model is fine-tuned via SFT on the seed chains (which are known to be correct) would establish whether RL exploration in the expanded environment adds value beyond supervised learning on the reference solutions.

2. Comparison to LLM-simulated environments at matched scale. The paper argues LLM-simulated environments are unreliable due to hallucination, but this is an empirical claim that should be tested: train a model via RL in LLM-simulated environments (using the same task specifications) and compare performance to ScaleEnv-trained models. Without this comparison, the superiority of execution-based environments remains asserted rather than demonstrated.

3. Fixed-domain, varied-task scaling. The paper varies domain count while fixing total task count, but does not test the reverse: fix N=4 domains and vary task count from, say, 256 to 4096. This would establish whether increasing task density within domains produces gains comparable to increasing domain count, providing a more complete picture of the scaling landscape.

4. Curated domain subsets. The domains used at each N in Figure 3 are subsets of the full 16-domain set. At N=2, which 2 domains were chosen? At N=16, all 16 domains are used. If the domains added between N=8 and N=16 are particularly valuable for generalization, the observed improvement might be due to domain selection rather than domain count. Testing multiple random subsets at each N (or reporting variance across subsets) would address this.

5. Scaling beyond 16 domains. The paper's most intriguing finding is that performance hasn't plateaued at N=16. Generating and evaluating at N=32 or N=64 would substantially strengthen the scaling law claim—particularly if the curve does eventually plateau, which would characterize the saturation point and provide practical guidance on how many domains to synthesize.

What specific claims hold conditionally, and under what conditions?

• "ScaleEnv improves generalization": Holds across all 7 evaluation domains at the 8B scale, and across 6 of 7 domains at the 32B scale. The improvement is robust but varies in magnitude—it is largest on domains that test information retrieval and multi-step tool execution (VitaBench In-store: +12.0 points at 32B) and smallest on domains where the base model is already strong (τ²-Bench Airline at 32B: +0.0 points).

• "Environmental diversity is critical for robust generalization": Holds for the tested range (2 to 16 domains) but the shape of the scaling curve beyond 16 domains is unknown. The claim that diversity "is more critical than task quantity" is supported by the controlled experiment but is domain-dependent: at small N, increasing diversity helps; at some larger N, the curve may plateau and task quantity within domains may become more important.

• "ScaleEnv circumvents the limitations of data scarcity": The paper demonstrates that ScaleEnv can generate 16 domains of training environments, which is sufficient for the observed improvements. Whether it "circumvents" scarcity depends on whether 16+ domains is sufficient for saturating the gains, which is unknown. If the scaling curve continues upward, then 16 domains does not circumvent scarcity—it only partially addresses it, and substantially more domains would be needed.

Genuine weaknesses in the experimental design

1. Single benchmark family and model family. All evaluations use τ²-Bench and VitaBench with Qwen3 models. Without testing on other benchmarks (e.g., BFCL, ToolBench, API-Bank) or other model families (e.g., Llama, Mistral), the generalization claim is narrower than the paper implies. Qwen3's particular strengths and weaknesses may interact with ScaleEnv training in ways that don't transfer.

2. No human evaluation of environment quality. The paper relies on execution-based verification to guarantee tool correctness, but does not assess whether the synthesized environments are realistic or useful from a human perspective. A "Job Seeking" domain with tools that pass procedural testing but don't reflect real hiring workflows would train agents on a simplified task that doesn't transfer to real job-seeking applications. This may not matter for benchmark performance (which only tests tool-use capability), but limits the claim that ScaleEnv produces "high-fidelity" environments.

3. The reward specification classification (Exempt, Hard, Semantic) is described but not ablated. The paper introduces a three-tier matching scheme for reward computation but does not test whether all three tiers are necessary. Training with only Hard constraints (exact matching) vs. Hard + Semantic alignment would reveal whether the fuzzy matching for text fields actually prevents reward hacking or is an unnecessary complication.

4. Small number of training steps (48). Both models are trained for only 48 GRPO steps. This is a small number for RL training and may mean the models are undertrained—further training might increase the magnitude of gains or reveal different scaling patterns. The paper does not report training curves or justify the 48-step choice.

5. The user simulator is a fixed LLM (Qwen2.5-72B-Instruct) with no analysis of its reliability. The user simulator generates natural language feedback during RL training. If the simulator produces inconsistent or unrealistic feedback, it could introduce noise that interacts with environment quality in unexamined ways. The paper does not report any evaluation of the simulator's output quality or its impact on training.

6. Limitations and Trade-offs

1. Environment Synthesis Relies on LLM Knowledge of the Domain, Not Ground-Truth Expertise

The entire domain construction pipeline begins with an LLM conceptualizing domain logic from a keyword (e.g., "Job Seeking") and generating tool schemas, database schemas, and dependency relationships based on its internal world knowledge (Section 4.1.1). The paper is transparent that this is a "top-down synthesis approach" starting from a "specific domain name," but does not address the fundamental dependency this creates: the synthesized environments are only as domain-accurate as the LLM's pretraining knowledge.

The consequence is twofold. First, the tools and workflows generated for a domain may be simplified or distorted relative to real-world practice. A synthesized "Airline" domain might include tools for booking, canceling, and refunding flights, but might omit complexities like fare class hierarchies, interline agreements, overbooking policies, or regulatory compliance checks—not because the synthesis pipeline fails, but because the LLM (the domain knowledge source) has an incomplete or oversimplified model of airline operations. An agent trained in this simplified environment learns to navigate a domain that is internally consistent but externally unrealistic. The paper's evaluation on τ²-Bench and VitaBench tests tool-use capability, not domain fidelity, so this limitation does not manifest in the benchmark results—but it would become apparent if the trained agent were deployed against real airline APIs with real-world constraints.

Second, this dependency creates a knowledge ceiling: domains that the LLM has weak or outdated knowledge about will produce correspondingly weak environments. The paper's 16 synthesized domains (Appendix B, Figure 5) cover domains like "Smart Home," "Healthcare Telemedicine," "Agriculture Environment," and "Job Seeking"—all domains where a general LLM likely has reasonable (if superficial) world knowledge. The framework would not work for esoteric, proprietary, or rapidly evolving domains where the LLM's knowledge is insufficient to generate coherent tool schemas and database structures. The paper does not investigate this boundary or characterize what happens when the LLM's domain knowledge is inadequate.

Evidence: The paper provides no experiment that tests the realism of synthesized environments against real-world equivalents. The t-SNE visualization (Figure 4) shows that synthesized and evaluation domains are semantically separated, but this demonstrates diversity, not fidelity. The Procedural Testing mechanism (Section 4.1.2) verifies that generated code executes correctly with respect to its own specifications; it does not verify that the specifications are faithful to real-world domain requirements. The Domain Stability Analysis (Table 5) shows robustness across different synthesized domains but does not compare synthesized environments to human-designed ones.

Mitigation status: The paper does not address this limitation. It does not propose verification of domain accuracy against external knowledge sources, does not incorporate human domain expert review into the pipeline, and does not acknowledge the gap between internal consistency and external fidelity. The "Impact Statement" acknowledges that "without proper oversight, this capability could be misused to construct environments that model harmful or unethical behaviors," but this concerns misuse rather than the fundamental limitation that LLM-based domain knowledge is a noisy and potentially incomplete source for synthesis.

2. The Environment Scaling Curve Covers Too Narrow a Range to Qualify as a Scaling Law

The paper's central empirical finding is the Environment Scaling Curve—the observation that zero-shot generalization improves monotonically as training domains increase from N=2 to N=16 while holding total task count fixed at 1024 (Section 5.3, Figure 3). The paper frames this as a scaling law, stating that "environmental diversity is critical for robust agent learning" and that "performance has not yet fully plateaued at N=16, suggesting that further scaling of environmental diversity remains a promising direction."

The limitation is that 2 to 16 domains is too narrow a range to establish a scaling law in the sense that term is used in the field (e.g., Kaplan et al., 2020; Hoffmann et al., 2022, where scaling trends span multiple orders of magnitude in parameters and tokens). The curve has only four data points (N=2, 4, 8, 16), each representing a single trained model with no error bars or confidence intervals. The absence of a plateau at N=16 does not indicate that a plateau does not exist at, say, N=32 or N=64—the curve could be approaching an asymptote that is simply not visible within the tested range.

The consequence is that practitioners cannot reliably extrapolate. If an organization is deciding whether to invest in synthesizing 100 domains rather than 16, the current data provides no basis for predicting the marginal benefit. The curve could continue rising linearly (in which case 100 domains would yield large gains), could taper off at N=20 (in which case domains beyond 16 have minimal value), or could even decline if very diverse domains introduce interference effects that the current range is too small to detect. The paper's conclusion that "scaling environmental diversity is more critical than task quantity" (Section 6) overinterprets data from a small-N regime.

Furthermore, the fixed total task count (1024) means that as N increases, tasks per domain decrease (from 512 at N=2 to 64 at N=16). The improvement at higher N could partially reflect that 512 tasks per domain provides diminishing returns and 64 tasks per domain is sufficient for learning—not that domain diversity per se is the driver. An experiment that independently varied domain count and tasks per domain would disambiguate this, but the paper does not conduct one. The Domain Scaling Analysis (Figure 3) demonstrates a correlation between domain count and performance but does not establish the causal mechanism.

Evidence: Figure 3 displays exactly four training points per benchmark (N=2, 4, 8, 16) plus the N=0 base model baseline. No confidence intervals, error bars, or multiple random seeds per N are reported. The paper does not provide tabulated values for the curve, making it impossible to compute the marginal gain per additional domain or to fit a functional form. The Domain Stability Analysis (Table 5) tests two subsets of 4 domains each (Set A and Set B) as robustness checks, but this tests stability at a single N rather than the shape of the curve.

Mitigation status: The paper explicitly states that "performance has not yet fully plateaued at N=16" and frames this as motivation for future work: "further scaling of environmental diversity remains a promising direction." This is a transparent acknowledgment of the limited range, but the paper's rhetorical framing as a "scaling curve" and its conclusions about the primacy of diversity over quantity go beyond what the data supports.

3. The Difficulty Estimation / Environment Quality Assessment Cost Is Not Accounted For

ScaleEnv's headline result is that training on synthesized environments improves downstream task performance (Table 1). However, the synthesis process itself is computationally expensive, and this cost is not folded into any efficiency comparison. The paper reports token consumption in Table 9: synthesizing a single domain foundation (including schemas and code) requires approximately 546,000 tokens, and generating a single verifiable task consumes roughly 93,200 tokens. These are LLM inference costs incurred before any RL training begins.

The consequence is that the true "cost" of the performance gains reported in Table 1 includes both the RL training compute and the synthesis compute. For the Domain Scaling Analysis with N=16 domains and 1024 tasks, the synthesis cost is approximately 16 × 546K + 1024 × 93.2K ≈ 104 million tokens of LLM inference, using multiple high-performance models (Deepseek-V3.2, GLM-4.7, GPT-5.1, Qwen3-32B; Section 5.1). The paper treats this as a fixed, amortized cost—but for practitioners deciding whether to adopt ScaleEnv, the question is whether the synthesis cost plus RL training cost produces better performance per total dollar than alternatives (e.g., supervised fine-tuning on human demonstrations, RL in LLM-simulated environments, or simply using a larger base model without environment-specific training). No such comparison is provided.

This limitation is particularly acute for the domain scaling conclusion. The paper argues that more domains are better, which implies synthesizing many domains. But the synthesis cost scales linearly with the number of domains and tasks. If the marginal improvement from adding domains diminishes (a possibility the limited N range of the scaling curve cannot rule out), the synthesis cost per unit of performance gain may become prohibitive well before the model's capacity to benefit from diversity is exhausted. The paper provides no analysis of cost-effectiveness as a function of domain count.

Evidence: Table 9 reports token consumption. The paper does not report wall-clock time, GPU-hours, or dollar cost for synthesis, nor does it report the computational cost of the Procedural Testing loops (how many debug iterations are typically required before tools pass? How many tool calls are executed during testing?). The paper does not fold synthesis costs into any performance-efficiency metric or compare against the cost of alternative data generation approaches.

Mitigation status: The paper acknowledges the existence of synthesis costs by reporting them (Table 9, Appendix B.2) but does not discuss their relationship to the claimed benefits or provide guidance on cost-benefit tradeoffs. This is not framed as a limitation by the authors; it is an omission in the analysis.

4. No Comparison to RL in LLM-Simulated Environments or SFT Baselines

The paper's central claim about the necessity of executable, code-verified environments rests on an argument from first principles (Section 1, Section 2.2): LLM-simulated environments hallucinate state and produce inconsistent feedback, and real-world environments lack diversity and state-altering capabilities. The ablation on Executability Verification (Table 3) demonstrates that removing verification from ScaleEnv degrades performance, confirming that verification matters within the ScaleEnv pipeline.

However, the paper does not provide a direct empirical comparison between ScaleEnv-trained agents and agents trained via alternative approaches at matched cost. Specifically:

• No RL in LLM-simulated environments: The paper argues that LLM-simulated environments are unreliable, but does not train a model via RL in such environments and compare performance. The argument remains theoretical. It is possible that the noise from LLM hallucination, while real, is tolerable at training scale and that the cost savings from avoiding code synthesis would offset the performance degradation.

• No SFT on seed chains: Each ScaleEnv task comes with a verified seed tool chain—a correct reference solution (Section 4.2.1). The paper does not test whether supervised fine-tuning on these seed chains (without RL exploration in the expanded environment) produces comparable or better performance. This is a critical missing baseline because the seed chains represent clean, correct demonstrations; if SFT on these chains matches or exceeds RL performance, the complexity of the full RL pipeline (with environment expansion, user simulation, and GRPO training) would be unjustified.

The consequence is that the paper demonstrates ScaleEnv works, but does not demonstrate that it works better than the alternatives it critiques. The improvement over the base model (Table 1) establishes that ScaleEnv training is better than nothing, not that it is better than other training approaches. For practitioners, the relevant comparison is not ScaleEnv vs. zero-shot base model, but ScaleEnv vs. the next best approach to training tool-use agents.

Evidence: The paper's main results (Table 1) compare Qwen3-SE models against base Qwen3 models and against other open-weights models (which are not trained on matched data). The ablation study (Tables 3, 4) compares ScaleEnv variants (with vs. without EV, rule-based vs. LLM-as-a-judge rewards) but does not compare against non-ScaleEnv training paradigms. The Domain Scaling Analysis (Figure 3) sweeps domain count within the ScaleEnv framework but does not include alternative frameworks at matched domain counts. The paper cites prior work on LLM-simulated environments (Liu et al., 2024; Li et al., 2025) and synthetic environments (Cai et al., 2025; Song et al., 2026) but provides no head-to-head comparisons.

Mitigation status: The paper does not acknowledge this as a limitation. It frames the work as proving that executable, verified environments can produce robust training, not as proving that they are superior to alternatives. The absence of alternative-training baselines limits the strength of the conclusions that can be drawn about ScaleEnv's relative value.

5. Evaluation Is Restricted to Two Benchmarks That Share the Same Interaction Paradigm as the Training Environments

All evaluation is conducted on τ²-Bench and VitaBench (Section 5.2). Both benchmarks follow the same interaction paradigm as ScaleEnv's synthesized training environments: multi-turn dialogue where an agent calls tools, receives structured results, interacts with a user, and must satisfy a verifiable goal. The paper explicitly characterizes the evaluation as Out-Of-Distribution in terms of domain semantics (confirmed by the t-SNE visualization in Figure 4) and interaction format (τ²-Bench requires adherence to textual policy documents not present during training). However, the underlying task structure—tool-mediated interaction with a stateful environment toward a verifiable goal—is shared between training and evaluation.

The consequence is that the generalization demonstrated in Table 1 may be narrower than the paper suggests. The agent has learned to operate in environments that share a common structure: call a tool, get a result, decide on the next tool, repeat until the goal state is reached. The transfer from synthesized "Job Seeking" to τ²-Bench "Airline" involves adapting to new tool names, schemas, and domain knowledge, but the fundamental interaction loop and reasoning pattern remain the same. It is unknown whether the learned capabilities would transfer to substantially different interaction paradigms—for example, web navigation (where tool calls are UI actions like clicking and typing), code execution (where the agent writes and runs code and debugs based on output), or physical reasoning tasks (where tool calls affect simulated physical states with continuous dynamics).

This limitation is important because one of the paper's contributions is claiming to demonstrate "robust zero-shot generalization" (Section 1) and "strong generalization capabilities" (Abstract). If the generalization is restricted to tasks that share the same structural paradigm as training, the term "generalist" (used in the title and throughout) overstates the demonstrated scope.

Evidence: The evaluation benchmarks (τ²-Bench and VitaBench) are described in Section 5.2. Both are multi-turn, tool-use dialogue benchmarks where the agent's success is determined by whether it correctly manipulates a structured environment state through a sequence of API-like function calls. The paper does not evaluate on benchmarks with different interaction paradigms. The t-SNE visualization (Figure 4) demonstrates semantic separation (different domains) but not structural separation (different interaction paradigms). The paper acknowledges that τ²-Bench introduces a "novel textual policy format" not present during training, but this is a variation in constraint representation, not a fundamental change in task structure.

Mitigation status: The paper does not discuss this as a limitation. It frames the OOD nature of evaluation primarily in terms of domain semantics and data formats (Section 5.2), not in terms of task structure. The conclusion states that "training on ScaleEnv-synthesized environments and tasks significantly boosts the performance of baseline models on unseen benchmarks, evidencing robust zero-shot generalization" (Section 6) without qualifying what types of unseen benchmarks this claim covers.

6. Hardest Tasks Show Near-Zero Absolute Performance Despite Relative Gains

The paper reports that ScaleEnv training improves performance across all evaluation domains (Table 1). However, on the most challenging tasks—particularly VitaBench's Cross domain—the absolute performance remains extremely low even after training. Qwen3-SE-8B achieves 3.0% on VitaBench Cross (up from 1.5% for the base model), and Qwen3-SE-32B achieves 10.8% (up from 5.3%). The paper frames these as significant improvements—"more than double the success potential" for the 32B model—but the absolute numbers mean the agent fails on 89–97% of tasks in this domain.

The VitaBench Cross domain is specifically designed to test "ambiguous user needs requiring proactive information retrieval and complex multi-step planning" (Section 5.2). The example given is that when a user says "I am sick," the agent must infer the latent intent to "recommend light food." This class of reasoning—inferring unstated user goals from conversational context and proactively taking information-gathering actions—is precisely the kind of capability that distinguishes advanced agents from basic tool executors. The very low absolute performance suggests that ScaleEnv's synthesized environments, despite their diversity, do not adequately cover this reasoning pattern.

The consequence is a capability boundary: ScaleEnv training improves performance on tasks where the reasoning challenge is which tools to call in what sequence given explicit or easily inferable goals, but does not appear to transfer effectively to tasks where the primary challenge is inferring latent intent from ambiguous natural language. This is not surprising—the seed tool chain synthesis process (Section 4.2.1) generates instructions grounded in specific tool sequences, which means the training distribution likely underrepresents tasks where the mapping from instruction to tool sequence is indirect or requires non-trivial inference. The paper does not analyze what specific types of tasks benefit most or least from ScaleEnv training, which would help practitioners understand where the method is applicable.

Evidence: Table 1 reports VitaBench Cross scores of 3.0% (Qwen3-SE-8B) and 10.8% (Qwen3-SE-32B). Table 2 reports Pass@4 on the same domain: 10.8% for Qwen3-SE-32B vs. 5.3% for the base model. The Pass@4 improvement confirms that the model has some capacity to produce correct solutions on these hard tasks (it succeeds on roughly 1 in 10 tasks given 4 attempts), but even with multiple attempts the failure rate exceeds 89%. The paper does not break down performance by task difficulty within individual domains, so it is unclear whether the Cross domain's low scores reflect uniformly hard tasks or a mixture where ScaleEnv helps on some subtypes but not others.

Mitigation status: The paper does not acknowledge this as a limitation. The VitaBench Cross results are reported alongside other domain results in Tables 1 and 2 without commentary about the absolute performance ceiling or the implications for the scope of learned capabilities. The Domain Scaling Analysis (Figure 3) shows Pass@4 on VitaBench but does not break down by individual VitaBench domains (Cross, Delivery, In-store, OTA), making it impossible to see whether additional domains improve performance on the hardest tasks or primarily improve easier tasks.

7. Implications and Future Directions

How This Work Changes the Landscape

ScaleEnv's primary contribution is not a single method but a reframing of what the agent training pipeline should look like. The paper's central argument—that environment synthesis must be structured around code execution verification rather than LLM text generation—challenges the prevailing intuition that LLMs can serve as oracles for environment simulation. This is not a paradigm shift in the sense of upending established theory, but it is a methodological correction with practical consequences for how RL-based agent training should be conducted.

The reframing is best understood through the specific failure mode the paper identifies and addresses. Prior approaches to environment generation treated correctness as an output property: generate an environment, then evaluate whether it's good. The LLM-simulated environment paradigm (Liu et al., 2024; Li et al., 2025) takes this to its logical extreme—"evaluation" is implicit in the LLM's generation, with no separate verification step. The synthetic environment paradigm (AutoForge, EnvScaler) adds verification but treats it as a filter, not as a construction mechanism. ScaleEnv's breakthrough is recognizing that in a domain where correctness is mechanically verifiable (code executes or it doesn't, database constraints hold or they don't), the generation process should be subordinated to the verification process. The LLM proposes; the execution environment decides. The Procedural Testing loop (Section 4.1.2) and the Dependency-Aware BFS expansion (Section 4.2.2) are instances of this principle, not ad hoc quality checks.

This reframing has several consequences for the research landscape:

It demotes LLM-based environment simulation from a scalable solution to a potentially dangerous shortcut. The paper's ablation on Executability Verification (Table 3) demonstrates that training on unverified environments degrades performance consistently across all evaluation domains. The mechanism is not subtle: unverified tools produce runtime failures that create "conflicting reward signals, preventing the policy from learning precise, logic-grounded decision-making." This finding, combined with the cited evidence on LLM hallucination in stateful contexts (Kadavath et al., 2022; Zhang et al., 2025), makes a strong empirical case that the convenience of LLM-based simulation is not worth the cost in training signal quality. Researchers working on agent training should now bear the burden of proof: if you propose an LLM-simulated environment, you must demonstrate that your specific simulation approach avoids the degradation documented in Table 3. The default assumption should shift toward execution-based verification.

It introduces environment diversity as a first-class scaling axis with empirical evidence that it matters independently of task quantity. The Environment Scaling Curve (Figure 3) is not a scaling law in the formal sense—four data points over a factor of 8 is too narrow to fit functional forms or make quantitative predictions—but it establishes a qualitative regularity that was not previously documented: at fixed total training compute, distributing tasks across more domains improves generalization, and the improvement has not saturated at 16 domains. This finding redirects attention from "how many training examples do we need?" to "how diverse should our training environments be?", which has implications for how organizations allocate data generation budgets. If the curve continues its upward trajectory at larger N, the economics of agent training shift decisively toward breadth over depth: given a fixed budget for environment creation, invest in many domains with modest task counts rather than exhaustively populating a few domains.

It provides a constructive definition of environment completeness (Entity Consistency and Interaction Completeness; Section 4.2) that is operational rather than aspirational. Prior work described these properties as desiderata; ScaleEnv provides an algorithm (Dependency-Aware BFS expansion) that guarantees them constructively. This has implications for how the field evaluates environment generation frameworks: rather than asking "is this environment good?", reviewers and practitioners can ask "does the construction process guarantee Entity Consistency and Interaction Completeness?"—a more precise and falsifiable criterion.

It partially reconciles the tension between scalability and reliability in environment generation. Prior approaches faced a trilemma: real-world environments are reliable but not scalable; LLM-simulated environments are scalable but unreliable; existing synthetic approaches (AutoForge, EnvScaler) are scalable and partially reliable but struggle with task-environment coherence. ScaleEnv demonstrates that all three desiderata can be satisfied simultaneously by making execution the gatekeeper at every pipeline stage. The cost—LLM inference for synthesis—is amortized across many training episodes and is likely negligible relative to the RL exploration cost, though the paper does not compute this tradeoff explicitly.

It sharpens the question of what capabilities RL in tool-use environments actually teaches. The t-SNE visualization (Figure 4) shows clear semantic separation between training and evaluation domains, and the consistent zero-shot improvements (Table 1) suggest that something general is being learned. But the paper does not identify what that "something" is. Is the agent learning meta-strategies for tool selection ("when I need information about an entity, look for a query tool that takes that entity's ID as input")? Is it learning state-tracking ("after I modify a record, subsequent queries should reflect the modification")? Is it learning error recovery ("if a tool call fails with a precondition error, I need to satisfy the precondition before retrying")? The absence of a mechanistic account of the learned capabilities means the generalization demonstrated in Table 1 remains an empirical observation awaiting explanation. This is not a weakness of the paper—it's a productive open question that the paper's framework makes newly tractable.

Research directions that become more attractive: Scaling environment diversity well beyond 16 domains to characterize the shape of the scaling curve (log-linear? power law? sigmoidal?) and identify the saturation point. Developing theoretical models of why environment diversity produces generalization, which could connect the empirical scaling curve to concepts from meta-learning, domain randomization, or information theory. Extending the execution-gated synthesis paradigm to domains beyond tool-use (e.g., code execution environments, physical simulation, multi-agent coordination).

Research directions that become less attractive: Using LLM-simulated environments for RL training without addressing the hallucination and state consistency problems documented in Table 3. Building larger, more complex tool-use benchmarks without corresponding environment generation frameworks—the paper suggests that benchmark construction is bottlenecked by the same synthesis challenges as training environment construction, and that automated generation with verification is the path forward rather than manual curation. Treating environment design as an artisanal, domain-expert-driven activity when automated synthesis with execution verification can produce 16 diverse, reliable domains from scratch.

Follow-Up Research This Work Enables

Characterizing the shape and saturation point of the Environment Scaling Curve beyond N=16. The paper's most tantalizing finding is that performance has not plateaued at 16 domains (Figure 3), but the functional form of the scaling relationship is unknown. A direct follow-up would synthesize 32, 64, or 128 domains using the same ScaleEnv pipeline (which the paper demonstrates can produce domains at a cost of ~546K tokens per domain foundation plus ~93.2K tokens per task; Table 9), train Qwen3-8B models on random subsets of increasing size while holding total task count fixed at 1024, and plot the extended curve. The key questions: Does the curve follow a power law, a logarithmic function, or an S-curve? At what N do marginal gains drop below, say, 1% of the base model's performance? If the curve plateaus below N=50, practitioners can target that number; if it continues rising at N=128, the implication is that environment diversity is vastly more important than currently appreciated. This experiment also tests a Boundary condition: does the model eventually suffer from interference or catastrophic forgetting as the number of training domains grows very large? The Domain Stability Analysis (Table 5) suggests that different domain subsets produce consistent improvements at N=4, but this says nothing about interference at N=64 or N=128.

Disentangling domain count from tasks-per-domain in the scaling relationship. The paper's Domain Scaling Analysis holds total task count fixed at 1024, which means domain count and tasks-per-domain are inversely coupled: at N=2, the model sees 512 tasks per domain; at N=16, it sees 64 tasks per domain. The observed improvement with increasing N could reflect (a) the benefit of domain diversity, (b) diminishing returns from high task counts per domain (512 tasks saturates the learning signal from a single domain, so moving to 64 tasks per domain across 16 domains actually provides more total learning signal), or (c) some interaction. A 2D experiment that independently varies domain count (N = 2, 4, 8, 16, 32) and tasks per domain (T = 16, 64, 256, 1024) would identify the Pareto frontier. The hypothesis consistent with the paper's framing: at low N, increasing T provides benefits up to a domain-specific saturation point, beyond which additional tasks from the same domain add negligible value. At high N, even small T per domain is sufficient because the diversity across domains provides the primary learning signal. The experiment would also reveal whether the optimal allocation of a fixed total task budget is to maximize N (spread tasks as thinly as possible across many domains) or to balance N and T—a question with direct implications for how to allocate ScaleEnv's synthesis budget.

Head-to-head comparison of ScaleEnv-trained agents against agents trained via RL in LLM-simulated environments at matched task count and domain diversity. The paper's argument against LLM-simulated environments is principled (hallucination, state inconsistency) and supported by the EV ablation (Table 3, which shows that removing execution verification degrades performance), but there is no direct comparison between ScaleEnv and the approach it critiques. A strong follow-up would: (1) take the exact same 16 domains and 1024 task specifications produced by ScaleEnv; (2) train a separate model via RL where tool execution and state updates are handled by prompting an LLM (e.g., GPT-5.1 or Qwen3-235B) to simulate the environment's responses, using the same GRPO setup and hyperparameters; (3) compare zero-shot performance on τ²-Bench and VitaBench. The LLM-simulated baseline would likely suffer from exactly the failure modes the paper predicts (inconsistent state, hallucinated tool outputs), but quantifying the performance gap would either validate the paper's strong stance or reveal that LLM-based simulation, despite its flaws, is "good enough" at training scale—either outcome advances our understanding. A nuanced variant would test different LLM simulators (stronger vs. weaker models, with vs. without chain-of-thought state tracking prompts) to see whether simulation quality can be improved enough to close the gap.

Testing whether ScaleEnv-trained capabilities transfer to structurally different interaction paradigms. The paper demonstrates generalization across domain semantics (e.g., from synthesized "Job Seeking" to τ²-Bench "Airline") and across superficial format changes (τ²-Bench's policy document constraint), but the underlying interaction paradigm—tool-mediated dialogue with structured API calls and verifiable goal states—is shared between training and evaluation. Does the learned capability transfer to substantially different paradigms? Concrete targets: (1) Web navigation benchmarks (e.g., WebArena, Mind2Web) where "tools" are UI actions and "state" is the DOM, testing whether the agent has learned general state-tracking and action sequencing strategies or paradigm-specific heuristics. (2) Code execution tasks (e.g., SWE-bench, HumanEval with interactive debugging) where "tools" are code execution and debugging operations, testing whether the agent can adapt its tool-use strategies to an environment where actions produce complex, unbounded outputs. (3) Multi-agent coordination tasks where "tools" are communication actions and "state" includes other agents' beliefs, testing whether the agent's learned exploration strategies extend to partially observable settings with strategic interaction. A negative result (no transfer) would scope the paper's generalization claim to tool-use paradigms and motivate research on what additional training diversity is needed for broader transfer. A positive result would substantially strengthen the claim that ScaleEnv teaches genuinely general reasoning strategies.

Ablating reward mechanism components to identify which are load-bearing. The paper's rule-based reward evaluator (Section 4.1.1) introduces a three-tier classification of database columns: Exempt Fields (ignored in comparison), Hard Constraints (exact matching), and Semantic Alignment (fuzzy semantic matching). This design is motivated by principled arguments (reward hacking avoidance, computational efficiency) but the components are not individually tested. A clean ablation would train separate models with: (1) Hard Constraints only (exact matching on all non-exempt fields, no semantic matching), (2) Hard + Semantic matching (the paper's configuration), (3) LLM-as-a-Judge only, and (4) a hybrid that uses rule-based rewards for Hard Constraints fields and LLM-based rewards for Semantic Alignment fields. The comparison would reveal whether semantic matching for text fields actually improves training (or whether exact matching on text is adequate because agents quickly learn to reproduce ground-truth phrasing), whether the benefit over LLM-as-a-Judge comes primarily from avoiding reward hacking or primarily from computational efficiency, and whether the optimal reward design is domain-dependent. The result would inform practitioners building similar systems about which components are worth implementing versus which add complexity without measurable benefit.

Training a difficulty estimator to replace the 2048-sample oracle in difficulty estimation. This paper does not use difficulty estimation, so this direction applies the paper's framework to a related problem. But a logical extension within the ScaleEnv framework: can the synthesis pipeline generate environments of calibrated difficulty by controlling the complexity parameters (distractor density, graph expansion depth, number of available tools)? The paper mentions that distractor density is "dynamically scaled according to predefined task complexity" (Section 4.2.1) but does not operationalize this scaling function or validate that it actually produces tasks of varying difficulty. An experiment would define a difficulty metric (e.g., length of the seed chain, number of distractors, graph density of the expanded subgraph), synthesize tasks across a range of these parameters, and evaluate whether a trained model's success rate correlates with the parameters. If the correlation is strong, the synthesis pipeline can produce targeted difficulty levels for curriculum learning—start RL training on easy tasks (short chains, few distractors) and gradually increase difficulty. This would address a limitation the paper does not discuss: the current synthesis produces tasks of uncontrolled difficulty, which may waste training compute on tasks that are trivially easy or impossibly hard for the current policy.

Practical Applications and Downstream Use Cases

Cost-efficient training data generation for enterprise tool-use agents. Organizations deploying LLM agents for internal tool use (e.g., an HR agent that can query employee records, submit time-off requests, and schedule interviews via internal APIs) face the same environment scarcity problem the paper addresses: they need diverse, reliable training environments that exercise their specific tools, but manually constructing hundreds of training scenarios is expensive and error-prone. ScaleEnv's pipeline could be adapted: given the organization's actual API specifications (tool schemas) and database schemas, the Procedural Testing mechanism would generate verified tool implementations and test cases, and the Graph Expansion pipeline would produce diverse task instances with distractors and edge cases. The paper's Domain Scaling Analysis (Figure 3) suggests that 16+ diverse task distributions yield substantial generalization improvements; for an enterprise with 3–5 internal tools, the framework could generate far more task diversity than manual curation at a fraction of the cost. The synthesis cost (~546K tokens per domain, ~93.2K tokens per task; Table 9) represents a few dollars of LLM inference per domain—orders of magnitude cheaper than hiring domain experts to design training scenarios. The rule-based reward mechanism (Section 4.1.1) is particularly valuable here because it enables automated, deterministic evaluation of agent performance without human judgment, making large-scale RL or iterative SFT pipelines feasible.

Accelerated RL training for open-source agent models. The paper demonstrates that training Qwen3-8B and Qwen3-32B via GRPO on ScaleEnv environments produces consistent zero-shot gains across unseen benchmarks (Table 1), with the 8B model showing improvements of +5.7 to +12.5 points on τ²-Bench domains and +2.5 to +9.0 points on VitaBench domains. This has direct implications for the open-source agent community. Currently, training competitive open-source tool-use agents requires either expensive human demonstration data or access to proprietary APIs for environment interaction. ScaleEnv provides a fully automated alternative that produces verifiable training environments from scratch. An open-source team with access to a moderately strong LLM for synthesis (the paper uses Deepseek-V3.2, GLM-4.7, GPT-5.1, and Qwen3-32B; Section 5.1) could replicate the pipeline, generate 16+ domains, and train their own agent models via the GRPO recipe in Appendix C. The 48-step training with batch sizes of 1024–2048 (Section 5.1) is computationally modest by modern standards—feasible on a small cluster of GPUs. The Environment Scaling Curve (Figure 3) provides an empirical basis for allocating synthesis resources: if the goal is maximizing generalization, invest in domain breadth (more domains) rather than task depth (more tasks per domain), with the caveat that the curve has only been demonstrated up to N=16.

Benchmark construction for tool-use agent evaluation. The paper's OOD evaluation setup—training on synthesized domains, evaluating on independently constructed benchmarks (τ²-Bench, VitaBench)—demonstrates that ScaleEnv's synthesized environments are sufficiently realistic that training on them transfers to human-designed benchmarks. This suggests a different use case: rather than training agents, use ScaleEnv to generate evaluation benchmarks for testing agent capabilities. The framework produces tasks with known ground-truth solutions (the seed chains), verifiable reward signals (the rule-based evaluator), and controlled complexity (via distractor density and graph expansion parameters). A benchmark constructed this way would have several advantages over manually curated benchmarks: it could be arbitrarily large (thousands of tasks across dozens of domains), it would have guaranteed solvability (each task's seed chain is verified by execution), it would resist contamination (new domains can be synthesized on demand), and it would support fine-grained capability analysis (varying task complexity along specific dimensions to identify agent strengths and weaknesses). The t-SNE visualization (Figure 4) provides a methodology for verifying that synthesized evaluation domains are semantically distinct from training domains, addressing a key concern in benchmark construction.

When to Prefer This Method

The paper does not articulate an explicit tradeoff matrix against named alternative training approaches (e.g., "use ScaleEnv when X, use LLM-simulated environments when Y"). It positions itself against prior work categorically—arguing that execution-based verification is necessary for reliable RL training—rather than conditionally. The ablation in Table 3 demonstrates that removing execution verification degrades performance, but this is a within-framework comparison, not a comparison against alternative frameworks. The paper does not provide data on how ScaleEnv-trained agents compare to agents trained via SFT on human demonstrations, RL in LLM-simulated environments at matched scale, or fine-tuning on real-world API interactions. Without such comparisons, a conditional decision rule would be speculative rather than evidence-based.

The paper does, however, imply conditions under which ScaleEnv is applicable versus inapplicable, based on the framework's design and limitations. From the synthesis pipeline's requirements (Section 4.1), ScaleEnv is applicable when the target domain can be characterized by structured tool schemas with verifiable pre/post-conditions, when database states can be represented as typed records with integrity constraints, and when task success can be evaluated by comparing final database states against ground truth. It is inapplicable—or at least untested—when tasks require open-ended natural language generation without structured state representations, when domain knowledge for tool schema synthesis exceeds the LLM's pretraining knowledge, or when the interaction paradigm differs fundamentally from tool-mediated dialogue (e.g., continuous control, real-time strategy, physical reasoning with noisy sensors). These boundary conditions follow from the framework's design but are not empirically validated in the paper.