Orchard: An Open-Source Agentic Modeling Framework

🎯 Pitch

Orchard’s 30B-MoE agent, with only ~3B active parameters, hits 67.5% on SWE-bench Verified—outperforming all prior open-source 32B recipes by a wide margin and approaching proprietary models 10–30× larger. Its 4B GUI agent simultaneously achieves a 68.4% average success rate across three web benchmarks, beating every other open vision-language model while matching Google and OpenAI’s proprietary systems.

1. Executive Summary

This paper introduces Orchard, an open-source framework for scalable agentic modeling built around Orchard Env—a thin, Kubernetes-native environment service that provides reusable sandbox primitives decoupled from agent harnesses, trainers, and task domains. Using this environment layer, the authors develop three training recipes: Orchard-SWE for software engineering (distilling 107K trajectories from MiniMax-M2.5 and Qwen3.5-397B, then applying credit-assignment SFT to extract partial-progress signals from unresolved trajectories, followed by Balanced Adaptive Rollout (BAR) for sparse-reward RL), Orchard-GUI for browser-based GUI agents (training a 4B vision-language model with only 0.4K distilled trajectories and 2.2K open-ended training tasks under judge-grounded RL), and Orchard-Claw for personal assistant workflows (training on 0.2K synthetic tasks with cross-harness evaluation). Orchard-SWE achieves 67.5% on SWE-bench Verified with only ~3B active parameters—surpassing every Qwen 2.5 32B and Qwen 3 32B open-source recipe—while Orchard-GUI reaches a 68.4% average success rate across WebVoyager, Online-Mind2Web, and DeepShop, making it the strongest open-source GUI agent and competitive with proprietary systems from OpenAI and Google Gemini, and Orchard-Claw attains 59.6% pass@3 on Claw-Eval, improving to 73.9% when paired with a stronger harness, establishing that a thin, harness-agnostic environment layer enables reusable agentic data and training recipes across domains only when the environment boundary remains decoupled from any single training stack or agent scaffold.

2. Context and Motivation

The Core Problem: Agentic Training Infrastructure Is Fragmented and Non-Reusable

The fundamental problem this paper addresses is infrastructure fragmentation in agentic AI research. As large language models (LLMs) evolve from text generators into autonomous agents that interact with external environments—executing code, navigating websites, managing calendars, and calling APIs—the requirements for training infrastructure have grown dramatically more complex than traditional supervised fine-tuning or RLHF. Training an agent that resolves GitHub issues requires provisioning thousands of Docker containers, each with a specific repository snapshot and test suite, managing their lifecycle across multi-turn interactions, collecting trajectories, and computing sparse, environment-grounded rewards. Training a browser-use agent requires launching Chromium instances with Playwright, interacting with live websites, capturing screenshots, and evaluating success via LLM judges. Training a personal assistant agent requires orchestrating sandboxed tool servers for email, calendar, and file management.

Yet despite this complexity, the field lacks a shared, open environment layer that works across these domains. The paper argues (Section 1) that the environment layer is the foundational bottleneck: when it is closed or rigidly coupled to a particular training stack, every layer above it—training recipes, evaluation pipelines, trajectory collection—inherits those constraints and cannot be independently reproduced or reused. This manifests in three specific ways:

Trajectory datasets are tied to specific harnesses. A trajectory collected under the OpenHands harness (Wang et al., 2025b) uses OpenHands' specific tool schema, observation format, and turn-level conventions. A model trained on those trajectories will often fail catastrophically when evaluated under a different harness like mini-swe-agent (Yang et al., 2024) or Kimi-CLI, because it has learned harness-specific formatting rather than domain-general agentic skills. The paper demonstrates this directly in Table 10: an SFT model trained on mini-swe-agent trajectories achieves 57.9% on SWE-bench Verified under mini-swe-agent but drops to 19.0% under OpenHands—a 38.9-point collapse from a harness change alone. This means that research communities using different scaffolds cannot easily build on each other's data, slowing collective progress.

Training recipes are coupled to specific infrastructure. Systems like ProRL Agent (Zhang et al., 2026a) and MegaFlow (Zhang et al., 2026b) embed environment management within larger training orchestration systems. ProRL Agent binds its environment layer to agent scaffolding through AgentHandler plugins, meaning harnesses cannot be swapped without modifying the environment configuration. MegaFlow co-designs its Model, Agent, and Environment services specifically for the Qwen training pipeline. A researcher who wants to try a different RL algorithm, a different base model, or a different agent harness must either adopt the entire stack or rebuild the environment layer from scratch. This coupling makes it difficult to isolate whether performance improvements come from better training recipes, better harnesses, or better base models—a fundamental challenge for scientific reproducibility.

Evaluation results are not comparable across papers. Because different systems use different environment backends (direct Docker, E2B, Daytona, Modal, custom Kubernetes deployments), the same agent code can exhibit different behavior due to subtle differences in sandbox latency, resource limits, network policies, and image availability. Table 3 demonstrates this concretely: the same benchmark run on different environment services shows command execution latency varying from 0.28s (Orchard Env) to 2.046s (Modal)—a 7.3× difference. Such infrastructure-level variation introduces noise into benchmark comparisons that the field currently has no systematic way to control for.

Why This Problem Matters: Three Levels of Impact

Practical: Reproducibility and cost barriers. As agentic training scales to thousands of concurrent sandboxes over hundreds of hours, the cost and complexity of environment management become dominant factors in who can participate in the research. The paper's cost analysis (Table 2) quantifies this: running 128 parallel sandboxes for 240 hours costs $7,078 with commercial managed services (Daytona, E2B) versus$ 673 with Orchard Env on spot instances—a 10× difference. For an academic lab, the difference between a $7,000 and a$ 700 experiment is the difference between feasibility and impossibility. Beyond raw cost, the requirement that researchers adopt a specific managed platform or institutional HPC cluster (as in ProRL Agent's Slurm/Singularity dependency) limits who can reproduce and build upon published results. The paper explicitly frames this as a research accessibility issue (Section 1):

"As agentic training and evaluation scale to new domains and larger datasets, the need for open, scalable, affordable, and research-friendly infrastructure becomes increasingly acute."

Scientific: Decoupling enables controlled experimentation. When the environment layer is a standalone service with a stable API, researchers can systematically vary one component at a time—changing the RL algorithm while holding the environment constant, or evaluating the same trained model across multiple harnesses to measure harness-robustness. The paper's cross-harness analysis (Table 8) would be impossible in a tightly integrated stack: testing Orchard-SWE across three harnesses (OpenHands, mini-swe-agent, Kimi-CLI) and three task distributions (SWE-bench Verified, SWE-bench Multilingual, Terminal-Bench 2.0) requires an environment layer that imposes no harness-specific assumptions. This kind of controlled experimentation is essential for understanding why a particular agentic recipe works—is it the training data, the harness, the RL algorithm, or the base model?—rather than just reporting a single resolve rate.

Economic: The pretraining-inference-capability tradeoff. The paper's headline results—a 30B MoE model (3B active parameters) matching or exceeding dense 72B models on SWE-bench Verified (Table 7), and a 4B vision-language model matching proprietary systems like Gemini computer-use-preview on GUI benchmarks (Table 13)—demonstrate that agentic post-training can substitute for model scale. This has direct economic implications: rather than training ever-larger models, organizations can invest in better agentic training infrastructure and recipes to extract more capability from smaller, cheaper models. But this substitution is only practical if the infrastructure for agentic training is accessible, reproducible, and cost-effective—exactly what Orchard aims to provide.

Prior Approaches and Their Limitations

The paper organizes existing systems into two paradigms (Section 2.2, Section 6), each with characteristic limitations:

Integrated Training Stacks

Systems like ProRL Agent (Zhang et al., 2026a), MegaFlow (Zhang et al., 2026b), and Modal (Modal Labs, 2024) embed environment management within a larger training or compute orchestration system.

ProRL Agent achieves an important partial decoupling by separating rollout generation from the RL trainer via an HTTP service. However, its environment layer remains bound to agent scaffolding through AgentHandler plugins. The paper notes: "the harnesses cannot be swapped without modifying the environment configuration" (Section 6). Additionally, ProRL Agent's reliance on Slurm and Singularity ties it to institutional HPC clusters, limiting adoption to researchers at specific institutions.
MegaFlow decomposes agentic training into three co-designed services (Model, Agent, Environment) capable of coordinating tens of thousands of concurrent agent tasks. But the three services are designed together for the Qwen training pipeline and are "not designed to be composed with arbitrary external trainers or third-party harnesses" (Section 6). A researcher using a different model family or training framework cannot reuse MegaFlow's environment service in isolation.
Modal is a general serverless compute platform, not specialized for agentic training. Its hosted control plane, per-second pricing, and lack of Kubernetes-native deployment mean that long-running RL training campaigns are difficult to optimize for cost and researcher control. At $0.335/sandbox-hour (Table 2), Modal is 46% more expensive than Daytona and 15× more expensive than Orchard Env on spot instances for a representative RL workload.

The fundamental limitation of integrated stacks is that the environment layer is not reusable as a standalone component. A trajectory collected under one integrated system cannot be used for SFT in another system; an RL recipe developed for one stack cannot be evaluated with a different harness; a model trained in one framework cannot be compared across harnesses without rebuilding the entire pipeline. This fragmentation slows the field by forcing every research group to rebuild infrastructure before they can innovate on methods.

Managed Sandbox Platforms

Commercial platforms like E2B (E2B, 2024), Daytona (Daytona, 2025), and Modal provide convenient hosted runtimes for code execution within sandboxes. They expose REST APIs or SDKs for sandbox lifecycle management and command execution—a similar surface area to Orchard Env—but differ in three critical ways for research use:

Limited infrastructure control. These platforms' hosted control planes determine resource allocation, autoscaling policies, and networking configuration. Researchers cannot optimize their cluster for their specific workload (e.g., pre-pulling images, tuning spot instance pools, configuring network isolation policies per experiment).
Higher cost at scale. Table 2 quantifies this: for a representative RL training workload (128 parallel sandboxes × 240 hours), E2B and Daytona both cost $7,078. Orchard Env on spot instances costs$ 673—a 10× reduction. This difference compounds over the course of iterative research, where multiple ablation studies, hyperparameter sweeps, and full-scale training runs are required.
Image compatibility overhead. Both E2B and Daytona require that sandbox images conform to their execution environment (e.g., including their agent SDK). For SWE-bench, which involves hundreds of heterogeneous Docker images with different Python versions, system libraries, and toolchains, this means per-image modifications before use. Orchard Env's agent injection mechanism (Section 2.1)—copying a self-contained execution agent into any user-provided Docker image at pod startup via a Kubernetes init container—eliminates this overhead entirely. The paper emphasizes: "This enables Orchard Env to support hundreds of heterogeneous task environments—such as the diverse images required by SWE-bench—without per-image modifications" (Section 2.2).

The paper also notes that E2B and Daytona ship only "limited open-source server components"—their primary product is the hosted control plane, which means researchers cannot self-host the full stack for reproducibility or cost control (Table 1).

Broader Environment Frameworks

ROCK (Wang et al., 2026) provides a richer environment platform with multiple protocols and broader scope, but the paper argues it "does not isolate the environment layer as a minimal service boundary" (Section 1). ROCK's broader platform components mean it is not a thin, standalone service that can be composed with arbitrary training stacks—adopting ROCK means adopting its broader ecosystem. SkyPilot Code Sandbox (Kim, 2025) provides open-source multi-cloud compute orchestration and matches Orchard Env's execution latency (0.284s vs. 0.280s; Table 3). However, SkyPilot is a general compute orchestration layer, not a specialized environment service with sandbox lifecycle management, network isolation, and heartbeat-based cleanup designed for agentic training. The paper positions SkyPilot as complementary: it "can serve as the underlying infrastructure on which Orchard Env is deployed" (Section 2.2).

The Gap: No Thin, Open, Harness-Agnostic Environment Service

The paper identifies a specific gap in the existing landscape: no system provides a thin, standalone environment service that is simultaneously (1) open-source and self-hostable, (2) decoupled from any agent harness, trainer, or task domain, and (3) cost-practical at research scale. Table 1 operationalizes this gap:

Thin env service: Systems like ProRL Agent and MegaFlow embed environment management in larger training systems. ROCK provides a broader platform. SkyPilot is a general compute orchestrator. Only E2B, Daytona, and Orchard Env expose environment management as a standalone service—but E2B and Daytona's primary products are managed services.
Self-hostable: E2B and Daytona ship limited open-source server components, but the full environment service (control plane, orchestration, pricing model) is hosted. Researchers cannot deploy their own instance for cost control or reproducibility.
Cost at scale: Managed services are 2–10× more expensive than self-hosted Kubernetes deployments for representative workloads (Table 2). The paper explicitly quantifies that Orchard Env's self-hosted design enables spot instance optimization that "makes large scale data collection and RL rollout feasible for academic research groups" (Section 3.6).

The gap is not that environment management is unsolved—many systems manage sandboxes—but that existing solutions are coupled to specific stacks, controlled by vendors, or too broad to serve as a minimal, reusable service boundary. This coupling prevents the research community from sharing trajectory data, reproducing training recipes, and comparing evaluation results across different harnesses and domains.

How Orchard Positions Itself

Orchard's central thesis is that the environment layer should be a thin, standalone service reusable across three axes: task domains, agent harnesses, and pipeline stages (Section 1). The paper argues this is not merely an infrastructure nicety but a substrate for reusability: when the environment boundary is clean, trajectory data collected under one harness can train models evaluated under another; SFT and RL recipes can share the same execution backend; and new domains can reuse the same environment service rather than rebuilding it.

This thesis is substantiated through three domain-specific instantiations that all compose with the same Orchard Env service:

Orchard-SWE uses Orchard Env to collect 107K trajectories across two harnesses (OpenHands, mini-swe-agent) and two teacher models, then trains with SFT+RL. The resulting model retains capability across unseen harnesses (Table 8), while models trained under single-harness systems (Scale-SWE, OpenSWE-32B) collapse catastrophically.
Orchard-GUI uses the same Orchard Env service for a completely different modality (vision-language browser navigation), demonstrating that the environment abstraction generalizes across task types—software engineering → web browsing—without modification.
Orchard-Claw uses Orchard Env for personal assistant workflows, training on two different harnesses simultaneously and showing that the trained model transfers its skills across harnesses, achieving 73.9% pass@3 when paired with the stronger ZeroClaw harness (Table 15).

The paper positions Orchard Env itself not as a novel technical contribution in the traditional sense—Kubernetes, FastAPI, and Docker are all mature technologies—but as a design argument about where the service boundary should be drawn in agentic training systems. The key technical choices (agent injection, direct Pod-IP communication, network isolation, heartbeat-based lifecycle) are individually straightforward; their value lies in how they collectively create a thin, composable service that can be dropped into any agentic training pipeline without coupling to any specific trainer, harness, or model.

The paper also explicitly connects to the broader trend of open-source agentic modeling, positioning Orchard as an effort to democratize the infrastructure layer so that research progress is driven by algorithmic and methodological innovations rather than access to proprietary training stacks:

"Orchard demonstrates that open-source agentic modeling can be scaled in a manner that is both cost-effective and reproducible, without coupling the environment to any single training stack." (Section 1)

This positions the paper at the intersection of systems research and agentic AI, with the central claim being that infrastructure design choices are not mere implementation details—they determine which scientific questions can be asked and which results can be reproduced.

3. Technical Approach

3.1 Reader Orientation

This is primarily a systems and infrastructure paper with an accompanying empirical demonstration across three agent domains. The core idea is that agentic training infrastructure should be decomposed into a thin, standalone environment service—decoupled from agent harnesses, training loops, and task domains—and that this decomposition enables trajectory data, SFT recipes, and RL rollouts to transfer across domains, harnesses, and pipeline stages in ways that vertically integrated stacks cannot support. The paper instantiates this idea through Orchard Env, a Kubernetes-native sandbox management service, and three training recipes (Orchard-SWE, Orchard-GUI, Orchard-Claw) that compose with it to produce state-of-the-art agentic models.

3.2 Big-Picture Architecture (Diagram in Words)

The Orchard framework has two layers, as shown in Figure 2:

Bottom layer: Orchard Env (the environment service). This is a thin, standalone REST API service deployed on Kubernetes that manages sandbox lifecycle (create, monitor, delete), executes commands inside sandboxes, handles file I/O, enforces network isolation policies, and injects a lightweight execution agent into arbitrary Docker images at runtime. It exposes generic primitives—POST /sandboxes, POST /exec, file upload/download endpoints, health checks—that are independent of any agent harness, training algorithm, or task domain. The service is self-hosted, not managed by a vendor.

Top layer: Training recipes (domain-specific instantiations). These are composable pipelines—trajectory collection, data curation, supervised fine-tuning (SFT), reinforcement learning (RL), evaluation—that call Orchard Env's API as their execution backend. Because Orchard Env imposes no harness-specific assumptions, the same recipes can be applied across software engineering (Orchard-SWE, with code repository sandboxes), GUI navigation (Orchard-GUI, with Chromium browser sandboxes), and personal assistant workflows (Orchard-Claw, with tool-server sandboxes). The recipes themselves include domain-specific components (reward functions, trajectory curation strategies, RL algorithms) but share the same environment orchestration layer.

Information flow: A training recipe starts a sandbox via POST /sandboxes (specifying the Docker image, resource limits, and network policy). The sandbox runs in an isolated Kubernetes Pod with an in-pod agent injected at startup. The recipe then executes commands or tools inside the sandbox via POST /exec (hot path, direct Pod-IP communication, 0.28s average latency), collecting multi-turn trajectories. Completed trajectories are scored (by test suites, LLM judges, or environment verifiers), curated, and used for SFT or RL training. The sandbox is cleaned up via DELETE /sandboxes or automatic heartbeat-based garbage collection.

3.3 Roadmap for the Deep Dive

First, Orchard Env's architecture and key design choices (Section 2.1) — the agent injection mechanism, direct Pod-IP communication, network isolation, and lifecycle management — because these are the infrastructure substrate that all three recipes build upon, and understanding them is prerequisite to understanding why the recipes are reusable across domains.
Second, Orchard-SWE's trajectory collection pipeline (Section 3.2) — how 107K trajectories are distilled from multiple teacher models across multiple harnesses and task sources — because the data is the foundation for both the SFT and RL stages.
Third, the credit-assignment SFT mechanism (Section 3.3.1) — how unresolved trajectories are mined for partial-progress signals via retrospective value estimation and rise-segment extraction — because this is Orchard-SWE's most distinctive SFT contribution and requires careful explanation of the value-estimation prompt, the temporal-difference credit calculation, and the segment extraction procedure.
Fourth, the Balanced Adaptive Rollout (BAR) algorithm (Section 3.3.3) — how BAR turns fixed-batch rollout into a self-pacing, reward-balanced schedule — because it addresses a fundamental limitation of GRPO-style RL for sparse-reward agentic tasks and is the most algorithmically novel component of the training recipe.
Fifth, Orchard-GUI's task filtering and training recipe (Sections 4.3–4.4) — how 292K raw tasks are filtered to 15.6K seed tasks, and how a 4B VLM is trained with SFT+RL under a generic ReAct harness — because this demonstrates cross-domain reusability of the environment service and introduces domain-specific design choices (screenshot history truncation, step-budget curriculum) that differ from SWE.
Sixth, Orchard-Claw's cross-harness training (Sections 5.2–5.3) — how synthetic tasks are generated, how trajectories are recorded via a proxy LLM server, and how end-to-end training with multiple harnesses enables cross-harness skill transfer — because this is the paper's most direct evidence that harness-agnostic environment infrastructure enables capabilities (cross-harness generalization) that single-harness training cannot achieve.

3.4 Detailed, Sentence-Based Technical Breakdown

This section provides a complete technical explanation of every mechanism, interface, and design choice in the Orchard framework, organized by component. I prioritize concrete operational descriptions—what each system does, step by step, with specific numbers, hyperparameters, and architectural choices—over abstract summaries.

Orchard Env: Architecture and Key Design Choices

The Orchard Env system is a three-layer architecture (Figure 3) designed to satisfy three requirements stated in Section 2: (1) a thin, standalone service boundary decoupled from harnesses and trainers; (2) low-cost compatibility with arbitrary Docker images; and (3) deployability on standard cloud infrastructure at research-affordable cost. The three layers are a client SDK, an orchestrator, and an in-pod agent, and the separation between them reflects a deliberate split between control-plane operations (sandbox lifecycle) and the latency-sensitive hot path (command execution).

Client SDK. Orchard Env provides two Python interfaces: SandboxClient (synchronous) and AsyncSandboxClient (asynchronous). Sandboxes are instantiated from user-specified Docker images via method calls, and the returned sandbox objects expose methods for command execution, file upload/download, and patch application. Context managers (with blocks) provide automatic cleanup. The SDK includes configurable retry logic with exponential backoff for transient connection errors and service-unavailable responses, and heartbeat utilities for keeping long-lived sandboxes alive when desired. This design means the client does not need to know anything about Kubernetes, pod lifecycle, or network topology—it interacts with a sandbox through a stable, harness-agnostic interface that is identical regardless of whether the sandbox contains a code repository, a Chromium browser, or a personal-assistant tool server.

Orchestrator. The orchestrator is a FastAPI service deployed as a Kubernetes Deployment with multiple replicas. It exposes a REST API and can optionally delegate sandbox metadata tracking to a Redis backend for cross-replica state sharing. The orchestrator has four key responsibilities:

Sandbox provisioning. POST /sandboxes requests are translated into Kubernetes Pod specifications. Each specification includes an init container configuration (for agent injection, described below), resource limits (CPU, memory), network policies (default-deny egress), and readiness probes (HTTP GET against the in-pod agent's /health endpoint).
Readiness tracking. A PodWatcher component maintains a persistent Kubernetes LIST+WATCH stream that tracks all sandbox pod state transitions in real time. State changes are cached in memory, and blocked clients (those calling /wait until their sandbox is ready) are woken via asyncio.Event when their pod's state transitions to Running with a passing readiness probe. This WATCH-based approach avoids the overhead of repeated polling of the Kubernetes API server.
Execution scheduling. An ExecManager routes execution requests (POST /exec) to the target sandbox's in-pod agent via direct HTTP calls to the Pod IP, serializing concurrent requests to the same sandbox through per-sandbox locks. This is the hot path: it bypasses the Kubernetes API server entirely, which is critical because the Kubernetes exec API introduces WebSocket setup overhead and control-plane mediation that would become a throughput bottleneck at scale.
Lifecycle management. A background reconciliation loop detects and cleans up orphaned sandboxes—those whose heartbeat has expired (client crashed or disconnected) or whose backing Pod has been evicted (preempted spot instance). This prevents resource leakage in long-running training jobs.

In-Pod Agent. The in-pod agent (referred to in the codebase as the "sandbox agent," distinct from the LLM-based agents studied elsewhere in the paper) is a lightweight FastAPI server that runs inside each sandbox container. It exposes endpoints for command execution (/exec), file upload, download, listing, and health checking (/health). Commands are executed as subprocesses with configurable timeouts; when a timeout is reached, the entire process tree is killed via a process group signal (os.killpg), preventing orphaned child processes from accumulating. The agent is reachable only through the sandbox pod's internal cluster network endpoint (its Pod IP), and its /health endpoint serves as the Kubernetes readiness probe—the pod is not marked Ready until the agent is accepting requests. This design means that any code running inside the sandbox (installing dependencies, executing user code, running tests) is isolated within the container and cannot interfere with the orchestrator or other sandboxes.

Agent Injection via Init Containers. This is Orchard Env's most distinctive technical choice and the mechanism that enables support for arbitrary Docker images without per-image modifications. The problem is that different agentic tasks require different base images—SWE-bench tasks use images with specific Python versions, system libraries, and repository snapshots; GUI tasks use images with Chromium and Playwright; claw-agent tasks use images with tool servers and test scripts. Requiring every task image to pre-install Orchard Env's execution agent would be an enormous practical burden (hundreds or thousands of images to rebuild) and would couple the environment service to task-specific image maintainers.

The solution is a Kubernetes init container. When a sandbox Pod is created, the orchestrator specifies an init container that runs before the main container. This init container copies a self-contained Python runtime and the agent server code into a shared emptyDir volume—a temporary filesystem that exists for the lifetime of the Pod and is accessible to all containers in the Pod. The main container's entrypoint is overridden at startup to launch the agent from the shared volume via a command like /opt/sandbox-agent/start.sh. The agent's executable, its Python dependencies, and its startup script are all loaded from the shared volume; the user's Docker image does not need to contain Python or any Orchard-specific code.

The paper emphasizes the practical consequence of this design: "Orchard Env targets Linux container images and, by default, launches the injected agent through sh -c" (Appendix A). This means that virtually any Linux Docker image—regardless of what shell, package manager, or runtime it uses—can be used as a sandbox without modification. For SWE-bench, where different repository images may use different base distributions (Ubuntu, Debian, Alpine), different Python versions (3.8 through 3.12), and different pre-installed toolchains, this eliminates the most labor-intensive bottleneck in scaling to new task sources.

Direct Pod-IP Communication. After a sandbox is provisioned, all execution and file operation requests are routed from the orchestrator's ExecManager directly to the Pod IP, bypassing the Kubernetes API server entirely. This is the primary architectural decision that enables low latency. The Kubernetes exec API (kubectl exec or its programmatic equivalent) works by establishing a WebSocket connection from the client through the API server to the kubelet on the node where the Pod runs, which then interacts with the container runtime. This path introduces (1) API server load from proxying every command's bidirectional I/O, (2) WebSocket setup overhead per command, and (3) additional network hops. By contrast, direct Pod-IP communication sends an HTTP request from the orchestrator to the in-pod agent with no intermediate proxies—it is a single TCP connection over the cluster's internal network.

The empirical consequence is quantified in Table 3: Orchard Env achieves 0.28s average command-execution latency, essentially matching SkyPilot Code Sandbox (0.284s, which uses a similar direct-communication approach) and significantly outperforming E2B (0.747s, 2.7× slower) and Modal (2.046s, 7.3× slower). For agentic training workloads where a single rollout may involve dozens of sequential command executions, this latency difference compounds: a 30-step SWE-bench trajectory with 0.28s per command spends 8.4 seconds in environment interaction overhead, while the same trajectory with 2.046s per command spends 61.4 seconds—a 53-second absolute difference per trajectory that directly reduces GPU utilization when LLM inference is waiting on environment responses.

Network Isolation. Orchard Env enforces network isolation through Kubernetes NetworkPolicy resources. A namespace-wide default-deny egress policy prevents sandbox containers from initiating any outbound connections. This is the default state: sandboxes cannot reach the internet, other pods, or external services. When a sandbox requires network access (e.g., for pip install during repository setup, or for a GUI agent to navigate live websites), the orchestrator creates a per-sandbox NetworkPolicy that selectively allows egress to the required destinations. This policy is cleaned up when the sandbox is deleted. The paper describes this as "defense-in-depth": even if a user-supplied command or agent action attempts to exfiltrate data or contact an external server, the network layer blocks it by default.

This design is particularly important for agentic training because (1) SWE-bench tasks often involve installing Python packages, which requires network access during setup but should not have network access during evaluation (to prevent cheating by downloading solutions), and (2) GUI-agent tasks require internet access for navigating live websites but should not be able to access internal cluster resources. Orchard Env's per-sandbox, time-bound network policies enable this fine-grained control without modifying task images or harness code.

Asynchronous Lifecycle with Heartbeat-Based Cleanup. Sandbox creation is asynchronous: POST /sandboxes returns immediately after the Pod specification is submitted to Kubernetes, without waiting for the Pod to become ready. Clients then poll or block on a /wait endpoint until the sandbox is ready. This decouples API responsiveness from Kubernetes scheduling latency—pod creation, image pulling (if the image is not pre-cached on the node), and agent startup can take seconds to tens of seconds, but the API remains responsive throughout.

Long-running sandboxes are kept alive by periodic heartbeat messages from the client SDK. If the client crashes, disconnects, or forgets to clean up, a background reconciliation loop in the orchestrator detects sandboxes whose heartbeat has expired (no heartbeat received within a configurable timeout window) and deletes them. This prevents resource leakage from crashed or abandoned clients—a practical necessity for RL training, where rollout processes may crash due to OOM, timeout, or bugs in agent code, and without automatic cleanup the cluster would accumulate zombie sandboxes consuming CPU and memory indefinitely.

Watch-Based Readiness. Rather than polling the Kubernetes API for pod status—which would generate linear load in the number of sandboxes being tracked—the orchestrator's PodWatcher maintains a persistent LIST+WATCH stream that receives push notifications of all sandbox pod state transitions in real time. State changes are cached in an in-memory data structure, and blocked clients are notified via asyncio.Event. This design means that at 1,000 concurrent sandboxes, the orchestrator does not make 1,000 periodic API calls to check readiness; it receives a single push event per state transition and wakes the specific client waiting for that sandbox.

Stress Test Validation. The paper validates Orchard Env's architecture under load with a stress test of 1,000 parallel sandboxes through the full create→execute→delete lifecycle (Table 4). The system achieved 100% success rate across all 1,000 sessions—no failures on creation, execution, or cleanup—with the entire test completing in 26 seconds end-to-end. Translating these numbers: 4,000 commands executed across 1,000 sandboxes in 26 seconds = approximately 154 commands per second sustained throughput. The paper notes this is "well above the throughput required by typical agentic distillation and RL workloads at this concurrency."

Functional Equivalence Validation. Beyond infrastructure metrics, the paper verifies that Orchard Env introduces no performance regression in downstream agent evaluations by comparing against a direct Docker baseline on Terminal-Bench 2.0 (Table 3, right). Using three models of varying capability (GPT-4.1, MiniMax-M2.5, Qwen3-8B-Thinking), Orchard Env matches Docker within run-to-run variance, with the largest absolute difference being 2.2 points (Qwen3-8B-Thinking: 7.0% Docker vs. 8.8% Orchard Env, within one standard deviation of the 3-run measurement). This confirms that agent injection and the Orchard Env execution path introduce no observable overhead or interference in agent–environment interactions.

Orchard-SWE: Trajectory Collection and Dataset Construction

Orchard-SWE's training corpus is constructed through large-scale trajectory distillation from strong teacher models, with a key design choice that distinguishes it from prior work: unresolved (failed) trajectories are retained alongside resolved (successful) trajectories, and the unresolved ones are mined for partial-progress supervised signals through credit-assignment SFT (explained in the next subsection).

Task Sources. The paper draws training instances from three sources, which together provide diversity in repository structure, issue type, and programming language:

SWE-rebench (Badertdinov et al., 2025): A large-scale collection of real-world GitHub issues with executable Docker-based test environments. The paper uses its filtered subset, which applies quality and difficulty filters to retain instances that are "both solvable and non-trivial, covering over 1,400 Python repositories" (Section 3.2). Each instance packages a GitHub issue description, a repository snapshot, a gold patch (the human-written fix), and an executable test suite that verifies whether a proposed patch correctly resolves the issue.
SWE-rebench V2 (Badertdinov et al., 2026): A language-agnostic extension of SWE-rebench that provides "over 32k containerized executable tasks spanning 20 programming languages and more than 3.6k repositories, together with pre-built images" (Section 3.2). The paper primarily uses its Python tasks for consistency with the rest of the task pool. Crucially, SWE-rebench V2 is reserved entirely for RL training—none of its trajectories are used during SFT, which provides a clean separation between the SFT and RL data distributions and tests whether RL can generalize to new repositories not seen during supervised training.
Scale-SWE (Zhao et al., 2026): A complementary task source that "constructs 100k task instances from real GitHub pull requests across 5.2k repositories" (Section 3.2). Each instance is packaged with a Docker image, a gold patch, and automatically generated test scripts. Scale-SWE significantly expands the diversity of repositories and issue types available for trajectory collection—by an order of magnitude relative to SWE-rebench in terms of unique repositories.

Multi-Teacher Trajectory Generation. For each task instance, the paper samples five rollout trajectories through Orchard Env and retains all trajectories that successfully resolve the task. The teacher pool includes Qwen3.5-397B (Qwen Team, 2026) and MiniMax-M2.5 230B (MiniMax, 2026). The choice of teachers reflects a pragmatic tradeoff: MiniMax-M2.5 achieves a higher task pass rate (solving more SWE-rebench instances), while Qwen3.5-397B "occasionally emits tool calls that are not defined in the OpenHands tool interface" (Section 3.2), which introduces noise. Based on these observations, MiniMax-M2.5 is used as the sole teacher for Scale-SWE (where rollout efficiency and stability become more important due to the larger number of instances—54,118 trajectories from this source), while both teachers are used on SWE-rebench (providing trajectory diversity for the same task instances).

A critical design detail: "In all cases, teachers interact through the same sandboxed tool interface used at evaluation time, ensuring that collected trajectories remain faithful to the downstream action space" (Section 3.2). This is important because if teachers used a different tool set or observation format than the student model would use at evaluation, the SFT signal would train for tool-use patterns that cannot be executed at test time. Orchard Env's harness-agnostic sandbox interface enables this: the same Docker image and tool execution backend serve both the teacher (during trajectory collection) and the student (during training and evaluation), so the trajectory's action-observation pairs are exactly what the student will see.

Harness Selection. The paper collects trajectories with two agent harnesses: OpenHands (Wang et al., 2025b), a full-featured multi-agent platform with rich tool semantics and structured observations, and mini-swe-agent (Yang et al., 2024), a lightweight harness with a minimal tool set (bash execution, file editing, submission) that is simpler and faster for large-scale rollout collection. On SWE-rebench, both harnesses are used, producing 12,213 trajectories from OpenHands and 40,854 trajectories from mini-swe-agent (combining both teachers). This dual-harness setup is not merely about data quantity—it is a deliberate design choice to expose the student model to different interaction styles and tool-use patterns during SFT, which the paper hypothesizes (and later demonstrates in Section 3.5) improves cross-harness generalization. For Scale-SWE, only mini-swe-agent is used because "we did not observe a meaningful performance gap relative to OpenHands on this source and the lighter harness is more practical for large-scale rollout collection" (Section 3.2).

Filtering and Curation. After collection, three quality filters are applied:

Trajectories exceeding 64K tokens are pruned to ensure training stability (truncation during training would lose critical later steps).
Trajectories containing tool calls not defined in the harness's tool interface (primarily observed with Qwen3.5-397B, which sometimes emits tool calls from a different interface) are discarded.
Trajectories with syntactically invalid or unparsable actions are removed.

The final Orchard-SWE dataset comprises 107K trajectories (74.6K resolved, 32.5K unresolved) spanning 19,287 unique task instances, with an average of 47.5 interaction turns and approximately 21K tokens per trajectory. The trajectory-level breakdown by source, teacher, harness, and resolution status is shown in Table 5. The distinction between resolved and unresolved trajectories is central to the training recipe: resolved trajectories provide direct imitation signal (the model learns to reproduce the full solve-and-submit pattern), while unresolved trajectories provide partial-progress signal through credit-assignment SFT (the model learns productive exploration behaviors—repository navigation, file localization, partial root-cause analysis—even when those behaviors did not ultimately lead to a correct patch).

Credit-Assignment SFT: Extracting Partial-Progress Signal from Failed Trajectories

This is Orchard-SWE's most distinctive SFT contribution and requires detailed explanation because it involves an LLM-based temporal-difference value estimation procedure that is not standard in the agentic training literature. The core motivation is that unresolved (failed) trajectories are not uniformly useless—they often contain productive segments where the agent was making genuine progress (e.g., identifying the correct source file, understanding the bug's root cause) before making a critical error. Discarding the entire trajectory wastes those productive segments as supervised signals.

Retrospective Value Estimation. The paper instantiates credit assignment as a "lightweight LLM-based variant of temporal-difference value estimation" (Section 3.3.1). The procedure works as follows:

For each unresolved trajectory $\tau = (s_0, a_0, s_1, \ldots, s_T)$ , where $s_t$ is the state (environment observation) after step $t$ and $a_t$ is the agent's action at step $t$ (its reasoning trace and tool invocation), the trajectory's own teacher model is shown the full trajectory together with the gold test outcome and asked to estimate, at each step $t$ , the probability that the agent will resolve the issue given the history up to step $t$ :

$V(s_t) = P(\text{resolve} \mid h_t, \text{outcome})$

where $h_t = (s_0, a_0, \ldots, s_t)$ is the trajectory history up to step $t$ , and $\text{outcome}$ is the known fact that the trajectory failed (the final patch did not pass the gold test suite).

What it computes: The teacher model is shown the entire trajectory (the agent's reasoning, tool calls, and environment observations at each step), is told which specific tests failed and which succeeded, and is asked to output a probability $p_{\text{resolve}} \in [0, 1]$ for each step. This is a retrospective judgment—the teacher knows the trajectory failed, so it can look backward to identify where things went wrong and calibrate the probability curve accordingly. The teacher annotates a sparse set of key steps (not every step—to reduce annotation cost), and the remaining values are linearly interpolated, yielding a per-step value curve $V(s_0), V(s_1), \ldots, V(s_T)$ .

Why this form: Using the teacher model as a retrospective value estimator leverages the teacher's understanding of software engineering to produce calibrated progress estimates without requiring human annotation. The retrospective, outcome-conditioned framing is critical: if the teacher were asked to estimate success probability prospectively (without knowing the outcome), it would overestimate progress because it cannot know about the eventual failure. The outcome-conditioning ensures the value curve reflects actual progress by forcing the teacher to explain why the trajectory failed and mark the step where the critical error occurred. The paper reports that "across our annotated trajectories, the curve is inverted-U in 98.9% of cases, peaking during exploration and decaying near the failed submission" (Section 3.3.1)—this shape (rising during productive exploration, falling after a critical mistake) is exactly what you would expect from a well-calibrated value function for a failed trajectory.

The Value-Estimation Prompt. The prompt (shown in full in Section 3.3.1) includes several calibration rules designed to prevent degenerate outputs:

The teacher must "Reason BACKWARD from the known failure" and identify "the critical mistake—the step(s) where the agent went wrong."
The initial probability should be 0.3–0.5 (a base rate reflecting the teacher's prior that most trajectories fail).
"P MUST DROP when the agent makes the critical error"—the probability cannot monotonically increase if the trajectory is known to fail.
"P at the final step should be BELOW 0.2 (we know it failed)."
The output format is a JSON array with one entry per annotated step, each containing the step number, the estimated probability, and a short reasoning string.

These calibration rules are an example of prompt engineering for reliability: without explicit instructions to produce non-monotonic curves that drop after critical errors, an LLM might default to producing a monotonically increasing curve (the agent is making progress) or a flat low curve (everything is wrong), neither of which would capture the partial-progress signal that motivates credit-assignment SFT.

Rise-Segment Extraction. Given the value curve $V(s_0), \ldots, V(s_T)$ , the paper defines per-step credit as the temporal-difference shift in estimated success probability:

$c_t = V(s_{t+1}) - V(s_t)$

where $V(s_t)$ is the retrospective value estimate at step $t$ and $V(s_{t+1})$ is the estimate at the next step.

What it computes: $c_t$ is a scalar that measures how much the teacher's estimated probability of success changed as a result of the action taken at step $t$ . A positive $c_t$ means the action increased the estimated probability of eventual success (the agent made progress—found a relevant file, understood a dependency, wrote a promising edit). A negative $c_t$ means the action decreased the estimated probability (the agent went down a wrong path, introduced a bug, or wasted time). A value near zero means the action had negligible effect on the estimated outcome.

Why this form: Using a temporal-difference (one-step change) rather than absolute value levels is intentional. Absolute value $V(s_t)$ conflates the cumulative progress made up to step $t$ with the immediate effect of the action at step $t$ . By taking the difference, $c_t$ isolates the marginal contribution of the action at step $t$ specifically. This is important because in a failed trajectory, the absolute value $V(s_t)$ might be high at early steps (the agent explored productively) but the credit $c_t$ tells us which specific steps produced that value increase, enabling targeted supervision.

A rise segment is then defined as a maximal contiguous subsequence $[t_i, t_j]$ over which the agent makes positive progress:

$c_t \geq \varepsilon \quad \forall t \in [t_i, t_j - 1]$

where $\varepsilon = 0.05$ is a small threshold to filter annotation noise (the teacher's probability estimates are not perfectly precise). The paper reports that "rise segments are typically short (median ~2 steps before merging with surrounding context) but capture the productive parts of an otherwise unsuccessful trajectory—repository navigation, file localization, and partial root-cause analysis" (Section 3.3.1).

SFT Objective with Masking. Standard next-token prediction is applied, but only on action tokens that fall within extracted rise segments (for unresolved trajectories) or all action tokens (for resolved trajectories). Environment observations are always masked from the loss:

$\mathcal{L}_{\text{SFT}} = -\sum_{t \in \mathcal{S}(\tau)} \log \pi_\theta(a_t \mid h_t)$

where $\mathcal{S}(\tau)$ is the set of action token positions contributing to the loss for trajectory $\tau$ , $\pi_\theta$ is the student model's predicted distribution over tokens, $a_t$ is the action token at position $t$ , and $h_t$ is the history (context) up to that position.

What it computes: The standard cross-entropy loss for next-token prediction, summed only over the tokens that the model should learn to predict. For resolved trajectories, $\mathcal{S}(\tau)$ contains all action tokens (equivalent to a single segment spanning the entire trajectory, since the terminal value is 1—the trajectory ended in success). For unresolved trajectories, $\mathcal{S}(\tau)$ is restricted to action tokens that fall inside the extracted rise segments, with the full history up to each token retained as context (the model can see the preceding trajectory, including steps that fall outside rise segments, but is only trained to predict actions within the productive segments).

Why this form: Selective masking converts the value estimates into a learning signal that focuses SFT capacity on productive patterns while ignoring unproductive or neutral actions. If an unresolved trajectory contains 50 steps, but only steps 5–7 and 12–14 show positive temporal-difference credit, the model is trained to predict the actions in those steps (along with the final submission for resolved trajectories) and all other action tokens are masked from the loss. This is more signal-efficient than either (a) discarding the entire unresolved trajectory (losing the productive patterns entirely) or (b) training on all steps of unresolved trajectories (learning from actions that did not contribute to progress and may reflect buggy or wasteful behavior). The preceding history is retained as context so the model learns "given this exploration so far, here is what a productive next step looks like"—a form of in-context credit assignment that complements the explicit value-based filtering.

Training Configuration. The SFT stage uses Qwen3-30B-A3B-Thinking (Qwen Team, 2025) as the base backbone. This is a Mixture-of-Experts model with 30B total parameters but only ~3B active parameters at inference, making it a compute-efficient target for multi-turn agentic training. Training uses the slime framework (Zhu et al., 2025) with the following hyperparameters:

Global batch size: 128
Context window: 64K tokens (extended to 128K at inference for longer repository contexts)
Epochs: 5
Optimizer: AdamW
Learning rate schedule: Cosine decay from $10^{-5}$ to $10^{-6}$
Multi-turn masking: Environment observations are excluded from the loss so the model is trained only to predict its reasoning traces and actions

The 64K context window (extended to 128K at inference) is a practical necessity for SWE-bench trajectories, which average 21K tokens and can run much longer for complex issues requiring extensive repository exploration. The five-epoch training budget reflects the fact that with only ~63K effective training trajectories (74.6K resolved + 32.5K unresolved with selective masking), the model benefits from multiple passes over the data to fully absorb the multi-turn interaction patterns.

Balanced Adaptive Rollout (BAR) for Sparse-Reward RL

BAR is the most algorithmically novel component of Orchard-SWE's RL stage and addresses two fundamental problems with the standard GRPO fixed-N group rollout used in prior work like DeepSeekMath (Shao et al., 2024):

Problem 1: Wasted compute from zero-variance groups. In GRPO, for each prompt (task instance), $N$ trajectories are sampled, scored with a reward model, and the advantage for each trajectory is computed as the normalized deviation from the group mean. If all $N$ trajectories succeed (reward = +1 for all) or all fail (reward = -1 for all), the group has zero reward variance. The advantage for every token in every trajectory in that group is exactly zero, and the group contributes nothing to the gradient update—yet the system has already paid the full cost of $N$ long, environment-bound trajectory rollouts. For SWE-bench tasks, where even strong models have very low or very high pass rates on individual instances, this waste is substantial: a task the model already solves 95% of the time will generate all-positive groups almost always, and a task far beyond the model's capability will generate all-negative groups almost always.

Problem 2: Group-imbalance noise. When the success rate of a prompt is far from 0.5, even a "non-degenerate" group (some successes, some failures) is dominated by whichever class is over-represented. If a prompt has a 10% success rate, a group of $N = 8$ trajectories will, on average, contain 0.8 successes and 7.2 failures. The GRPO advantage normalization scales each reward by $(r_i - \text{mean}) / \text{std}$ , and with such extreme class imbalance, the resulting advantages are noisy and biased toward the over-represented class. The gradient signal from the few positive trajectories is diluted by the many negative ones, and vice versa when success rates are very high.

BAR's Solution: Progressive, Group-Aware Rollout. BAR replaces the fixed- $N$ rollout with an adaptive procedure that generates trajectories in strides (batches of $s$ trajectories), evaluates rewards after each stride, and attempts to assemble a training group of exactly $N$ trajectories whose positive-reward fraction lies in a target interval $[\rho_{\text{min}}, \rho_{\text{max}}]$ . The algorithm stops generating trajectories as soon as a balanced group can be assembled, avoiding wasted compute on prompts that would otherwise produce zero-variance groups.

Algorithm Parameters. For each prompt, BAR is configured with five quantities:

$N$ : the training group size—the number of trajectories the optimizer will consume (set to 8 in the paper).
$N_{\text{max}}$ : the maximum budget—an upper bound on how many trajectories BAR is willing to generate for this prompt (set to 16).
$s$ : the stride—the size of an incremental generation batch (set to 16, which equals $N_{\text{max}}$ , meaning BAR generates all trajectories in one batch and then assembles the group).
$[\rho_{\text{min}}, \rho_{\text{max}}]$ : the target positive-reward fraction interval (set to $[0.375, 0.625]$ ).
$\rho^{\star} = (\rho_{\text{min}} + \rho_{\text{max}}) / 2$ : the ideal ratio (0.5—equal numbers of positive and negative trajectories).

Algorithm Procedure (Algorithm 1 in the paper). BAR proceeds in four phases:

Strided Generation. Starting from an empty trajectory pool $\mathcal{T}$ , BAR generates $s$ trajectories in parallel through Orchard Env. Each trajectory $\tau_i$ is scored with the reward function $\mathcal{R}$ (which returns $+1$ if the final patch passes the gold test suite, $-1$ otherwise), and the scored trajectory is added to $\mathcal{T}$ .
Group Assembly Attempt. After each stride, BAR calls TRYASSEMBLE to test whether a balanced group can be constructed from the current pool. TRYASSEMBLE first partitions $\mathcal{T}$ into three sets: $\mathcal{T}^+$ (trajectories with reward $> 0$ , usable as positives), $\mathcal{T}^-$ (trajectories with reward $\leq 0$ , usable as negatives), and a backfill pile (aborted, truncated, or time-exceeded trajectories that carry no usable learning signal but can serve as padding if no better options exist).
Balanced Construction. TRYASSEMBLE computes the target number of positive trajectories $n^{\star} = \text{round}(\rho^{\star} \cdot N)$ (with $N=8$ and $\rho^{\star}=0.5$ , this is exactly 4), and the acceptable range $[n_{\text{min}}, n_{\text{max}}] = [\lceil \rho_{\text{min}} \cdot N \rceil, \lfloor \rho_{\text{max}} \cdot N \rfloor]$ (with $[\rho_{\text{min}}, \rho_{\text{max}}] = [0.375, 0.625]$ , this is $[3, 5]$ ). It then iterates over $n^+ \in \{n_{\text{min}}, \ldots, n_{\text{max}}\}$ in order of increasing distance from $n^{\star}$ , checking whether $|\mathcal{T}^+| \geq n^+$ and $|\mathcal{T}^-| \geq N - n^+$ . If a feasible pair exists, it returns the concatenation of the first $n^+$ trajectories from $\mathcal{T}^+$ and the first $N - n^+$ trajectories from $\mathcal{T}^-$ (each set sorted by status—completed before truncated before aborted—and then by response length ascending, to prefer concise, well-terminated trajectories). If no feasible pair exists, it returns failure.
Fallback Logic. If the assembly attempt succeeds, BAR early-stops and returns the balanced group immediately, without generating further strides. If $N_{\text{max}}$ trajectories have been generated and no balanced group has been assembled, BAR enters a relaxed fallback: it attempts assembly again with the constraint interval relaxed to $[0, 1]$ (any positive fraction is acceptable, as long as there is at least one positive and one negative). If this also fails (e.g., all $N_{\text{max}}$ trajectories were aborted), BAR calls TOPRANKED, which returns the best $N$ trajectories by status then length, even if the group is degenerate.

Why This Algorithm Works (Behavioral Analysis). BAR behaves differently depending on the prompt's difficulty:

Easy prompts (high success rate, e.g., 90%+): The first stride of $s = 16$ trajectories will contain ~14 positives and ~2 negatives. TRYASSEMBLE can construct a balanced group of $N = 8$ with $n^+ = 4$ positives and $n^- = 4$ negatives (well within the $[3, 5]$ positive range) from the available pool. BAR early-stops after one stride, using only 16 generated trajectories to produce a training group of 8—compared to a fixed- $N$ approach that would generate 8 trajectories and discard the group (all positives, zero variance, zero gradient). The cost is 16 trajectories instead of 8, but the output is a usable gradient update instead of zero.
Hard prompts (very low success rate, e.g., 5%): The first stride of 16 trajectories will contain ~1 positive and ~15 negatives. TRYASSEMBLE checks: can it construct $n^+ = 4$ ? No, there is only ~1 positive available. It tries $n^+ = 3$ : still not enough positives. It tries $n^+ = 5$ : the positive supply is even less adequate. The attempt fails. Because $s = N_{\text{max}} = 16$ , BAR has exhausted its budget and enters fallback. In the relaxed fallback, it constructs a group with the available 1 positive and 7 negatives—a degenerate group (positive fraction 0.125, well outside $[0.375, 0.625]$ ) but still providing some learning signal (the model sees a successful trajectory contrasted with several failed ones) rather than an all-negative group that would be discarded.
Well-balanced prompts (success rate near 50%): The first stride of 16 trajectories will contain ~8 positives and ~8 negatives. TRYASSEMBLE can easily construct a perfectly balanced group ( $n^+ = 4, n^- = 4$ ) and early-stops. BAR behaves identically to fixed- $N$ GRPO in this regime—the prompt is naturally informative—but does so without wasting additional compute.

Final Group Filtering. After BAR assembles a training group, a group-level filter is applied before the group is admitted to the training batch. The filter checks for trajectories that carry no usable learning signal even within an otherwise valid group—for example, a trajectory where the agent's final submission could not be evaluated because the sandbox timed out during test execution, or where the LLM hit its token budget mid-step and produced a truncated output. A group that fails the filter is dropped entirely, and the training batch is replenished from over-sampled prompts (prompts for which alternative groups are available). The paper notes that "BAR and the group filter are designed to work jointly: BAR maximizes the probability that a generated group satisfies the filter on the first try, and the filter provides a hard correctness guarantee on whatever BAR returns. Together they implement a form of reward-aware curriculum that is performed online, at every gradient step" (Section 3.3.3).

Integration with GRPO. BAR composes cleanly with any group-relative advantage estimator because its contract is simply "return a list of $N$ trajectories per prompt." The paper uses standard GRPO advantage normalization:

$A_{i,t} = \frac{r_i - \text{mean}(\{r_1, r_2, \ldots, r_N\})}{\text{std}(\{r_1, r_2, \ldots, r_N\})}$

where $A_{i,t}$ is the advantage for trajectory $i$ at token position $t$ , $r_i$ is the reward for trajectory $i$ , and $\text{mean}$ and $\text{std}$ are computed over the $N$ trajectories in the group.

What it computes: For each token in each trajectory, the advantage is the trajectory-level reward normalized by the group statistics. Because BAR ensures the group has a positive fraction in $[0.375, 0.625]$ , the mean reward will be between $-0.25$ and $+0.25$ (for binary $\pm 1$ rewards, with 3–5 positives out of 8, the mean is $(n^+ \cdot 1 + n^- \cdot (-1)) / 8$ , which ranges from $+0.25$ when $n^+ = 5$ to $-0.25$ when $n^+ = 3$ ). The advantage for a positive trajectory in a 4-positive group is $(1 - 0) / 1.0 = +1$ , and for a negative trajectory is $(-1 - 0) / 1.0 = -1$ —symmetric, well-scaled advantages. By contrast, a fixed- $N$ group with 7 positives and 1 negative would have advantages heavily biased toward small positive values and one extreme negative value, creating noisy gradients.

Why this form: Normalizing by group mean and standard deviation is the standard GRPO advantage estimator, chosen because it is hyperparameter-free (no learned value function to train, no GAE parameter to tune) and has been shown to work well for reasoning tasks. BAR's contribution is not to change the advantage formula but to change the composition of the group that the advantage is computed over, ensuring that the group statistics are well-behaved (mean near zero, standard deviation near one) and the gradients carry high information density per generated trajectory.

RL Training Configuration. The full RL stage operates with the following hyperparameters:

Global batch size: 128
Rollout batch size: 16 (number of trajectories generated in parallel per stride)
Training group size $N$ : 8
Maximum budget $N_{\text{max}}$ : 16
Stride $s$ : 16 (matching $N_{\text{max}}$ , so BAR generates all trajectories in one batch)
Target positive fraction: $[\rho_{\text{min}}, \rho_{\text{max}}] = [0.375, 0.625]$ , ideal ratio $\rho^{\star} = 0.5$
Maximum RL steps: 150 (early stopping based on validation performance)
Context window: 64K tokens
Learning rate: Cosine decay from $10^{-6}$
RL algorithm: GRPO with group-relative advantage normalization
Reward: Binary, environment-grounded: $+1$ if the final patch passes the gold test suite in the Orchard Env sandbox, $-1$ otherwise. The paper explicitly notes that Orchard Env's 0.28s command-execution latency "is critical at this stage, as each RL rollout requires dozens of environment interactions, and training throughput scales directly with sandbox responsiveness" (Section 3.3.2).

Data Selection for RL. The RL task pool is constructed from SWE-rebench V2 (entirely held out from SFT) and Scale-SWE instances not used during SFT. The paper first runs the initial SFT checkpoint on each candidate task with 8 rollouts to estimate its pass rate, then retains only tasks with pass rate $0 < \hat{p} \leq 0.5$ —filtering out tasks that are either too hard ( $\hat{p} \approx 0$ , no positive trajectories can be generated even with BAR's adaptive sampling) or too easy ( $\hat{p} > 0.5$ , BAR would assemble balanced groups but the learning signal would be weak because the model already solves them consistently). This selection is particularly important for SWE-rebench V2, which is highly challenging: Table 6 shows that even the SFT checkpoint achieves only 22.36% pass@1 and 27.94% pass@3 on the initial evaluation of its Python subset. After filtering, the final RL training set contains approximately 2K instances. The paper uses mini-swe-agent as the harness for RL training (chosen for its speed and simplicity relative to OpenHands).

Orchard-GUI: Task Filtering, Trajectory Collection, and Vision-Language Training

Orchard-GUI demonstrates that the same Orchard Env service and training recipe transfer to a completely different modality—browser-based GUI navigation with a vision-language model—by showing how domain-specific components (task filtering pipeline, screenshot context management, step-budget curriculum) compose with the shared environment layer.

Task Filtering Pipeline. The paper starts from WebGym (Bai et al., 2026b), a dataset of 292,092 raw task instances for web navigation, and applies a five-stage filtering pipeline (Figure 4, Section 4.3) designed to produce a clean, evaluation-safe, and diverse pool of training prompts. The pipeline is:

Remove evaluation benchmark overlap. Strips out splits that overlap with held-out benchmarks (Online-Mind2Web, DeepShop, WebVoyager) to prevent train/test contamination. This removes 13,840 tasks (4.7%), leaving 278,252 tasks from the PAE-WebVoyager and InSTA-v3 splits.
Keep parent tasks only. WebGym provides both parent intents and child tasks decomposed from each parent. Since child tasks share substantial structure with their parents, retaining only parents avoids intra-family redundancy. This removes 23,437 tasks (8.4%).
Exclude WebVoyager tasks. Drops tasks whose intent appears in the original WebVoyager benchmark, eliminating residual contamination at the prompt level. This removes 411 tasks (0.2%).
Restrict to popular websites. Long-tail websites are noisier (more captchas, anti-bot blocks, broken pages) and less representative of realistic browsing. Tasks are retained only if their target site falls within the SimilarWeb Top-100 list or the MOZ Top-500 Most Popular Websites, and the site has at least two tasks. This removes 114,349 tasks (44.9%).
Semantic deduplication. The remaining pool is dominated by near-duplicate intents (paraphrases of the same shopping or search query across thousands of products). Each task intent is embedded with Qwen3-Embedding-8B, and tasks within a cosine similarity threshold of 0.99 of a previously kept task are greedily removed. This removes 124,454 tasks (88.9%).

The final filtered pool contains 15,601 unique task intents, spanning 13,063 unique hosts across six broad domain categories (Figure 5, left). These tasks cover 85.0% of MOZ Top-500 websites and 57.0% of SimilarWeb Top-100 websites, with 48.5% and 13.0% of tasks landing on those respective lists (Figure 5, right). Notably, the tasks used in the RL stage undergo the same filtering pipeline but with a more restrictive deduplication threshold of 0.95, yielding a task set of 2,198 tasks (Table 12).

Trajectory Generation. Qwen3-VL-235B-A22B-Thinking (Bai et al., 2025) serves as the sole teacher for trajectory distillation, with 4 independent rollouts sampled per task through Orchard Env under a generic ReAct-style tool-calling harness (described below). This yields 62,395 teacher rollouts from the 15,601 seed tasks (a small fraction of attempts abort due to environment or rollout engine errors). GPT-4.1 serves as the reward judge, evaluating whether the agent's final done(response) and the screenshot trail together satisfy the user intent, producing a binary reward.

The paper reports teacher success statistics (Figure 6): 68.4% of tasks have at least one passing rollout, 26.3% pass on all four rollouts, and 31.6% fail on every rollout. Of the 4,934 tasks that fail on every rollout, 41.1% (2,026 tasks; 13.0% of the full pool) are captcha-blocked on all four attempts—an environmental failure rather than an agentic one. Per-website success rates vary widely, with anti-bot-prone hosts clustered at the low end.

Generic Tool-Calling Agent Harness. Rather than adopting a bespoke browser-agent harness such as Browser-Use, Orchard-GUI uses a generic multi-turn ReAct-style loop with a fixed action space of 13 atomic tools defined in the OpenAI tool-calling format: click, write, press_keys, scroll, wait, drag, hover, goto_url, go_back, new_tab, switch_tab, close_tab, and done(response). The tool schemas (full specifications in Appendix C) are injected into the system prompt at the start of each episode, and the agent emits one or more <tool_call> blocks per response. The browser observation at each turn includes the latest screenshot, viewport dimensions, and a tab summary (URL and title for each open tab).

A critical practical design choice is screenshot history truncation: a single screenshot can expand to thousands of vision tokens, and naively concatenating the full screenshot history would quickly inflate the context window beyond any reasonable training-time length (a 30-step rollout would saturate a 64K token context). The paper addresses this by retaining only the last $k$ screenshots verbatim, relying on the agent's prior reasoning traces (which remain in context across turns) to carry the distilled actionable information from earlier screenshots.

SFT Curation Strategy. The paper deliberately avoids using the full successful pool of teacher rollouts for SFT, arguing that "oversaturating the student on imitation data before RL tends to drive it into a narrow imitation regime that on-policy gradients struggle to escape" (Section 4.3). Instead, a small, carefully curated subset is selected:

Source restriction. Only PAE-WebVoyager trajectories are used for SFT (not InSTA-v3). The justification: although PAE-WebVoyager contributes only 16% of the seed tasks (2,537 of 15,601), 38.8% of its tasks land on a SimilarWeb Top-100 site, versus just 3.6% for InSTA-v3—a ~10× density advantage on popular hosts that more closely reflect everyday user browsing habits (Table 12).
Within-task quality selection. For each task, a single rollout is kept: the shortest successful trajectory (fewest turns, with ties broken by total response length). Shorter teacher trajectories tend to be cleaner (less recovery noise, fewer wasted actions).
Across-website diversity cap. Each website is capped at $K = 20$ tasks to prevent high-volume hosts (e.g., amazon.com, coursera.org) from dominating the SFT mix.

The final SFT corpus comprises 412 unique tasks spanning 70 websites. This is remarkably small—two to three orders of magnitude less than prior open-source GUI agents (e.g., MolmoWeb-4B used >278.5K tasks; Table 13)—but the paper shows that with subsequent judge-grounded RL over a modest 2.2K-task pool, the 4B model achieves state-of-the-art open-source results.

SFT Training Details. Training initializes from Qwen3-VL-4B-Thinking and fine-tunes only the language model weights (the vision encoder and multimodal projector are frozen). Each teacher rollout generates one training example per assistant turn: the $t$ -th example carries the chat-template-serialized prefix up through turn $t$ and supervises only that turn's assistant response. The loss is computed only on the final (target) assistant turn; the system prompt, earlier assistant turns retained as in-context history, and every environment observation are masked out.

The training hyperparameters are:

Epochs: 3
Peak learning rate: $10^{-5}$ under cosine schedule with 10% linear warmup
Per-device batch: 2
Gradient accumulation: 8 steps
Per-worker effective batch: 16
Global batch: 128 across 8 data-parallel workers

Freezing the vision encoder is a deliberate choice to preserve the backbone's screenshot-grounding capability and concentrate SFT capacity on agent-specific reasoning and action prediction rather than re-learning visual features.

RL Training Configuration. The RL stage uses a multi-turn variant of GRPO with several domain-specific adaptations:

Reward function: Binary judge-grounded reward. A trajectory receives $+1$ when every assistant turn parses as a valid thinking+tool-call structure AND the final done(response) is judged SUCCESS by GPT-4.1 against the screenshot trail and user intent; $-1$ when the rollout terminates from repeated format failures; and 0 otherwise.
Asymmetric PPO clipping: $\epsilon_{\text{low}} = 0.2$ , $\epsilon_{\text{high}} = 0.28$ . No KL or entropy regularization is applied.
No per-trajectory length normalization: The paper "intentionally omit[s] the per-trajectory $1/T_i$ loss normalization so that longer, harder tasks are not down-weighted" (Section 4.4). This is important because GUI tasks vary widely in difficulty and horizon length—normalizing by trajectory length would dilute the learning signal from long-horizon tasks, which are precisely where the model needs to improve.
DAPO-style trajectory-level dynamic sampling: Groups whose rewards are all 0 or all $+1$ are dropped (zero-variance groups provide no learning signal). Additionally, the loss mask is zeroed for judge API failures and captcha-aborted runs, so infrastructure noise does not leak into the policy update.
Step-budget curriculum: The RL training proceeds in two phases. First, the model is trained with a per-episode step budget capped at 15, producing dense reward signal on tasks the policy can already solve within 15 steps. Once performance saturates, training continues from that checkpoint with the budget raised to 30, extending the policy to harder tasks that genuinely require more interaction.

The training dynamics (Figure 7) show that RL initialized from the SFT checkpoint achieves consistently higher evaluation success rates and more stable optimization than RL initialized directly from the base model. Both settings eventually achieve comparable training rewards (approximately 55%), but the SFT-initialized policy generalizes to over 50% evaluation success, while the base-model-initialized policy plateaus below 40%. This gap indicates that "supervised initialization provides a crucial behavioral prior that stabilizes exploration and enables RL to more effectively translate reward optimization into downstream task success" (Section 4.5).

Orchard-Claw: Synthetic Task Generation and Cross-Harness Training

Orchard-Claw is the paper's demonstration that Orchard Env supports personal-assistant agent training with cross-harness skill transfer, using synthetic task generation to overcome the scarcity of claw-agent training data.

Synthetic Task Generation. Because claw-based agents are relatively new and lack large-scale training datasets, the paper uses Claude Opus 4.6 to synthesize tasks from two seed sources: tasks from Claw-Eval (Ye et al., 2026) and workflows from popular skills on ClawHub. The generation pipeline is a four-step loop:

Propose and filter task ideas. Opus 4.6 is prompted to generate task ideas based on the seed source.
Generate the environment. Opus 4.6 creates the necessary files, tool server configurations, and test scripts that constitute the task's execution environment.
Solve with a teacher model. MiniMax-M2.5 is run as the solver to produce rollout trajectories, verifying that the task is solvable and that the instruction is clear.
Refine based on rollouts. If the solver fails or produces ambiguous outputs, the task is refined (instruction clarified, environment fixed) and re-tested.

Each task costs approximately 4.9 USD to synthesize on average, yielding 192 tasks in total shared across the Claw-Eval ReAct-style harness and the ZeroClaw harness. This is an even smaller training set than Orchard-GUI's 412 SFT tasks, making Orchard-Claw an extreme test of data efficiency.

Trajectory Collection via Proxy LLM Server. For each synthesized task, 5 rollouts are sampled from MiniMax-M2.5 through Orchard Env under the corresponding harness, and only trajectories that complete the task are kept. To record training samples from complex harnesses such as ZeroClaw—which may make multiple LLM calls internally (e.g., for subagent delegation)—the paper implements a proxy LLM server that intercepts every LLM call (input and output) during the rollout. Once a rollout finishes, each recorded (input, output) pair is grouped back as a training trajectory, yielding 561 trajectories with 4,537 training pairs in total for SFT.

This proxy-server approach is a pragmatic workaround for harnesses that have internal agentic logic: rather than requiring the harness to expose a clean trajectory-recording API, the proxy transparently captures all LLM interactions regardless of the harness's internal structure. It is another example of Orchard Env's harness-agnostic design enabling practical workflows—the environment service doesn't care about the harness's internal complexity, and the proxy server doesn't need to understand the harness's architecture.

SFT+RL Training. The training recipe follows the same two-stage pattern as Orchard-SWE and Orchard-GUI:

SFT: Training from Qwen3-30B-A3B-Thinking-2507 for 1 epoch with a global batch size of 16 and a 64K context window, using cosine-decayed learning rate from $10^{-5}$ to $10^{-6}$ . Left truncation is applied to sequences exceeding the context window. The loss masks the input and trains only on the response tokens (following the same multi-turn masking pattern as Orchard-SWE).
RL: Standard GRPO (without BAR—the task pool is smaller and the paper does not discuss BAR for this domain) with a batch size of 8 and group size of 8 over 150 training steps. The reward is binary and environment-grounded: if a rollout passes all test scripts, every (input, output) pair in the rollout receives $+1$ ; otherwise, every pair receives $-1$ .

The key RL design choice is a wall-clock budget rather than a step limit: "during rollout we do not set a maximum step limit but rather a 10-minute wall-clock budget for each task. We find this better accommodates the differing per-step latencies of different harnesses and different tool calls, and better matches real-world usage" (Section 5.3). Rollouts that exceed the budget are aborted, and all of their turns are excluded from training. Orchard Env's sandbox parallelization enables 64 asynchronous rollout sandboxes per RL step, which the paper credits with substantially improving training throughput.

The RL training curves (Figure 8) show steady improvement in both success rate (train and validation) and trajectory length (number of agent turns per rollout), indicating that the agent learns to solve more tasks while also engaging in longer, more complex multi-turn interactions over the course of RL training.

4. Key Insights and Innovations

Innovation 1: The Environment Layer as a Foundationally Independent Service Boundary — Not Just Infrastructure, but a Reusability Substrate

The paper's most fundamental intellectual move is to argue that the environment layer in agentic training systems is not merely an infrastructural component—a container runtime or sandbox manager that sits below the "real" research in training recipes and model architectures—but is instead the substrate governing the reusability of all artifacts above it. This is a conceptual reframing, not a technical invention: the paper's technical components (Kubernetes, FastAPI, init containers) are individually standard. What is distinctive is the claim that where you draw the service boundary determines what the research community can share, reproduce, and compose.

Prior to Orchard, the dominant implicit assumption in agentic training was that environment management belongs inside the training stack—either as a sub-component of a vertically integrated system (ProRL Agent embeds it in its rollout server alongside agent scaffolding; MegaFlow co-designs it with its Model and Agent services) or as a managed commodity service (E2B, Daytona, Modal) whose internal design is a vendor concern. Under this assumption, trajectory data collected under one harness is tightly coupled to that harness's tool schema and observation format; RL recipes are implemented against a specific environment backend's API; and evaluation results are tied to the latency characteristics and resource limits of whatever sandbox provider was used. The paper's cross-harness analysis (Table 8) makes this coupling concrete: models trained under single-harness systems collapse catastrophically when evaluated under different harnesses, not because they lack domain knowledge, but because they have learned harness-specific formatting rather than harness-agnostic skills.

Orchard's reframing is that a thin, standalone environment service with a stable, minimal API (sandbox lifecycle, command execution, file I/O, network policy) serves as a compatibility layer that decouples everything above it. This is directly analogous to how operating system abstractions (processes, files, sockets) decouple application code from hardware, or how the TCP/IP stack decouples network applications from physical links. The paper argues, through its three domain instantiations, that this decoupling is not merely convenient but enables forms of reusability that are impossible in coupled systems: (1) trajectory data collected under one harness can train models that are then evaluated under different harnesses (Orchard-SWE, Table 10), (2) SFT and RL recipes developed for one domain (software engineering) can be applied to entirely different domains (GUI navigation, personal assistant workflows) without modifying the environment layer, and (3) evaluation can be conducted across multiple harnesses to measure harness-robustness rather than single-harness capability (Table 8, Table 15).

The "innovation" here is not the software architecture—REST APIs for sandbox management exist in commercial products—but the diagnosis of the coupling problem and the prescription that the environment boundary must be thin enough to be harness-agnostic. The paper operationalizes this as three specific design requirements (Section 2): the service must be standalone (not embedded in a trainer), must support arbitrary Docker images at zero per-image adaptation cost (via agent injection), and must be deployable on standard cloud infrastructure at research-affordable cost (via Kubernetes-native design enabling spot instances). Each of these requirements is tested empirically: agent injection enables support for hundreds of heterogeneous SWE-bench images without modification (Section 2.1), direct Pod-IP communication achieves 0.28s latency matching optimized native runtimes (Table 3), and spot-instance deployment reduces cost to 10× below managed alternatives (Table 2).

This reframing is significant because it changes the question from "which training recipe is best?" to "how should agentic training infrastructure be decomposed to maximize reusability?"—a systems-design question that the field had not systematically asked. It also provides a vocabulary for diagnosing why prior work produces non-reproducible or harness-locked results: the coupling is not accidental but structural, arising from the absence of a stable environment abstraction layer.

Innovation 2: Credit-Assignment SFT — Mining Partial-Progress Supervision from Failed Trajectories via Retrospective Value Estimation

Orchard-SWE's most distinctive SFT contribution is the idea that failed (unresolved) trajectories contain extractable partial-progress signal that can be converted into supervised training data through retrospective, outcome-conditioned value estimation. This is a departure from the dominant paradigm in agentic SFT, where only successful (resolved) trajectories are retained for training—a natural heuristic grounded in the assumption that failed trajectories teach the model to fail. The paper demonstrates that this heuristic discards useful information: failed trajectories often contain productive segments (repository navigation, file localization, partial root-cause analysis) that represent genuine agentic skill, even though a critical error later in the trajectory prevented task completion.

The conceptual move is to treat failed trajectories not as uniformly negative examples to discard, but as partially-observable progress traces where some subsequences are valuable and others are not. This reframes the problem from binary trajectory filtering (keep/discard) to a temporal credit assignment problem: given a failed trajectory with a known outcome, which steps contributed positively to the (eventually unrealized) goal, and which steps were neutral or harmful?

Prior work on learning from failed trajectories in agentic contexts is sparse and largely focused on RL, where negative trajectories contribute through value estimation or advantage computation during online policy optimization. The idea of extracting supervised signal from failed trajectories for SFT—before any RL—is underexplored. The paper's mechanism for doing so is distinctive in two respects:

First, the use of the teacher model as a retrospective value function. Rather than training a separate value model or relying on heuristic progress metrics (e.g., number of passing tests, lines of code written), the paper prompts the trajectory's own teacher model with the full trajectory and the known failure outcome, asking it to estimate per-step success probabilities. This leverages the teacher's understanding of the task (it generated the trajectory) and the retrospective framing (it knows the outcome, so it can identify where things went wrong) to produce calibrated progress estimates without human annotation. The calibration rules in the prompt—requiring non-monotonic curves that drop at the critical error, with base rates grounded in the known failure—are a form of prompt-engineered reliable value estimation that constrains the LLM's output to be informative rather than degenerate.

Second, the rise-segment extraction criterion. By defining credit as the temporal-difference shift $c_t = V(s_{t+1}) - V(s_t)$ and extracting contiguous positive-credit segments, the paper converts a continuous value curve into a sparse set of action subsequences that represent genuine progress. This is a principled criterion grounded in reinforcement learning theory (temporal-difference credit assignment), applied in a novel context (offline SFT data curation). The empirical gain—+1.9 points on SWE-bench Verified from adding 32K rise-segment trajectories to 32K resolved trajectories (Section 3.6)—is modest but validates that the extracted signal is genuine supervision rather than noise, and the method's real value may be in enabling the use of much larger pools of unresolved trajectories that would otherwise be discarded entirely.

The significance of this innovation extends beyond the specific +1.9-point gain. It suggests a general principle for agentic training: with a strong teacher model, the distinction between "success" and "failure" trajectories is less informative than the per-step progress signal within both. If this principle generalizes to other domains (GUI navigation, robotics, dialogue), it could substantially increase the data efficiency of agentic SFT by converting the large fraction of teacher rollouts that fail—31.6% of GUI tasks had zero successful teacher rollouts (Figure 6)—from waste products into training resources.

Innovation 3: Balanced Adaptive Rollout (BAR) — Group-Aware, Self-Pacing Rollout Scheduling as a Response to Sparse-Reward Variance Collapse

BAR addresses a problem that is well-known in the GRPO-style RL literature but typically handled through post-hoc filtering: zero-variance groups (where all trajectories in a training group have identical rewards) contribute zero gradient signal but have already consumed the full generation budget. Prior approaches—discarding zero-variance groups after generation (Yu et al., 2025), pre-filtering prompts by historical success rate (Bae et al., 2026), or post-hoc down-sampling oversized rollout sets (Xu et al., 2025)—share a common structure: they react to group degeneracy after the fact, either by discarding generated trajectories or by avoiding hard prompts entirely.

BAR's distinctive move is to make the rollout schedule itself adaptive, generating trajectories incrementally and stopping as soon as a balanced group can be assembled. This transforms the rollout from a fixed-cost operation (generate N trajectories, then check if the group is usable) into a conditional one (generate s trajectories, check, repeat if needed). The algorithm is conceptually simple—it is essentially a rejection-sampling loop with a group-balance constraint—but the implications for training efficiency in sparse-reward domains are substantial.

The key insight is that prompt difficulty determines the rate at which balanced groups naturally occur, and BAR exploits this heterogeneity rather than treating all prompts uniformly. Easy prompts (high success rate) produce mostly positive trajectories; BAR generates a full stride, assembles a balanced group using the minority-class negatives, and stops—avoiding the fixed-N regime where easy prompts would produce all-positive zero-variance groups. Hard prompts (very low success rate) produce mostly negative trajectories; BAR exhausts its budget trying to find enough positives, falls back to a degenerate group, but at least provides some contrastive signal (one positive vs. many negatives) rather than the all-negative group that fixed-N would produce. The efficient regime is balanced prompts (success rate near 50%), where BAR early-stops after a single stride because balanced groups form naturally.

The paper does not claim BAR as a theoretically optimal solution—it is a heuristic that trades additional generation budget (for hard prompts, BAR generates up to $N_{\text{max}} > N$ trajectories) for higher information density per training batch. The practical value is demonstrated indirectly: Orchard-SWE's RL stage adds +3.2 points on SWE-bench Verified over the already-strong SFT checkpoint (64.3% → 67.5%), and while BAR's specific contribution is not ablated in isolation, the paper's framing of BAR as a response to "the standard fixed-N group rollout used by GRPO" (Section 3.3.3) positions it as an enabling component for sparse-reward RL at scale.

What makes BAR an innovation rather than an incremental tweak is that it reconceptualizes the relationship between environment interaction and gradient quality in agentic RL. In standard RLHF, rollout generation is cheap (text generation is fast) and reward computation is cheap (a reward model forward pass). In agentic RL, environment interaction dominates the cost: each trajectory requires dozens of sequential tool executions inside sandboxes, and the reward depends on environment-grounded outcomes (test suite pass/fail, LLM judge). BAR's design acknowledges that in this cost regime, the optimization target should not be "generate N and hope for variance" but "generate until you achieve a specific information-theoretic condition on the group composition". This is a subtle but important shift in how the training loop interfaces with the environment service, and it is enabled by Orchard Env's low-latency command execution: if sandbox interactions were expensive (7× slower, as with Modal), the additional generation budget BAR incurs for hard prompts would be prohibitive.

Innovation 4: Cross-Harness Generalization as a Diagnostic for Agentic Capability — Not a Metric to Maximize, but a Test of What the Model Has Learned

The paper's cross-harness evaluation (Section 3.5, Table 8) is not presented as a new method but as a diagnostic finding that reveals a previously underappreciated failure mode in agentic training: harness lock-in. When models are trained on trajectories from a single harness, they learn the harness's specific tool-call format, observation structure, and turn-level conventions so tightly that switching harnesses causes catastrophic performance collapse—not because the model has forgotten the domain knowledge, but because it cannot produce syntactically valid outputs in the new harness's format.

This finding is significant because it exposes a construct validity problem in the SWE-bench leaderboard. If two models report similar resolve rates on SWE-bench Verified under their respective native harnesses (e.g., Scale-SWE at 64.0% with OpenHands, Orchard-SWE at 64.3% with mini-swe-agent), the single-number comparison masks the fact that one model (Scale-SWE) produces invalid outputs under any harness other than its native one, while the other (Orchard-SWE) retains capability across harnesses (45.0–64.3% across OpenHands, mini-swe-agent, and Kimi-CLI). The paper's Table 8 makes this explicit: Scale-SWE's cross-harness resolve rate is zero (catastrophic format failure), OpenSWE-32B drops from 62.4% to 3.6% on an unseen harness (-58.8 points), while Orchard-SWE's worst-case drop is bounded at 19.3 points.

The conceptual contribution is not "we built a harness-robust model" but rather identifying harness coupling as a distinct axis of generalization that single-benchmark leaderboards obscure. This connects to broader concerns in the LLM evaluation literature about benchmark contamination and shortcut learning, but with a domain-specific twist: in agentic settings, the "shortcut" is not a spurious pattern in the input distribution but the syntactic interface between the model and the environment. A model that achieves 64% resolve rate under harness A but 0% under harness B has not learned to solve software engineering tasks; it has learned to produce outputs that harness A can interpret as valid tool calls, and those outputs happen to resolve tasks when executed. The model's capability is real—it did resolve 64% of tasks—but it is not portable.

The paper's dual-harness training (using both OpenHands and mini-swe-agent during trajectory collection) is a straightforward remedy, but the deeper insight is that harness diversity during training serves a role analogous to data augmentation in computer vision: it forces the model to learn the underlying task structure (identifying bugs, editing code, running tests) rather than the surface statistics of any particular harness's output format. The controlled experiment in Table 10 confirms this: when training data comes from exactly one harness, evaluation under that harness yields 53.5–57.9% while evaluation under a mismatched harness collapses to 19.0–28.0%—a 30–40 point diagonal–off-diagonal gap that quantifies the degree of harness coupling.

This diagnostic is not limited to SWE. It applies to any agentic domain where multiple harnesses exist (GUI agents with different browser automation frameworks, claw agents with different tool-server architectures), and it suggests that evaluating under multiple harnesses should be standard practice for claims of general agentic capability. The paper provides the infrastructure (Orchard Env's harness-agnostic API) and the methodology (cross-harness evaluation matrix) to make this practice feasible, and demonstrates it across two domains: SWE (Table 8) and claw agents (Table 15), where Orchard-Claw's improvement when switching to ZeroClaw is +9.3 points on pass3, while a baseline model regresses.

5. Experimental Analysis

Evaluation Methodology

Dataset. The primary evaluation benchmark for Orchard-SWE is SWE-bench Verified (OpenAI, 2024), a human-validated subset of 500 instances from the original SWE-bench (Jimenez et al., 2024). Each instance consists of a real GitHub issue description, a repository snapshot, and a gold test suite that must be passed for a solution to be counted as correct. Auxiliary evaluations use SWE-bench Multilingual (Yang et al., 2025a) and Terminal-Bench 2.0 (Merrill et al., 2026). For Orchard-GUI, the evaluation uses WebVoyager (He et al., 2024), Online-Mind2Web (Deng et al., 2023), and DeepShop (Lyu et al., 2025)—three structurally distinct live-website benchmarks with no overlapping action spaces or reward signals. For Orchard-Claw, the primary evaluation is Claw-Eval (Ye et al., 2026), a benchmark that audits agent trajectories for completion, safety, and robustness using a combination of automated scripts and LLM-as-a-judge, aggregating into a single task score (task_score = safety × (0.8 · completion + 0.2 · robustness)) with a pass threshold of ≥0.75.
Base model(s). Orchard-SWE uses Qwen3-30B-A3B-Thinking (Qwen Team, 2025)—a Mixture-of-Experts model with 30B total parameters but only ~3B active at inference—chosen because it is "representative of the capabilities of many contemporary LLMs" and sits in a useful regime with non-trivial but far-from-saturated performance on MATH-equivalent reasoning tasks (22.0% baseline on SWE-bench Verified under OpenHands, per Table 7). Orchard-GUI uses Qwen3-VL-4B-Thinking (Bai et al., 2025), a 4B vision-language model chosen to demonstrate that environment-grounded RL can extract capability competitive with proprietary systems from a small backbone. Orchard-Claw uses Qwen3-30B-A3B-Thinking-2507, a variant of the same MoE family. For the FLOPs-matched comparison in the SWE domain, a second model with approximately 14× more parameters is used as the pretraining-scaled baseline, though its specific identity is not named in the paper beyond being from the PaLM 2 family (Section 7).
Metrics. For SWE-bench, the primary metric is resolve rate (%)—the fraction of 500 test instances for which the agent's submitted patch passes the full gold test suite. For GUI benchmarks, the metric is success rate (%), computed by an LLM-as-a-judge (GPT-4.1) evaluating whether the agent's final done(response) and the screenshot trail satisfy the user intent, following the evaluation protocol of FARA (Awadallah et al., 2025) and Molmo-Web (Gupta et al., 2026). For Claw-Eval, the paper reports pass3 (fraction of tasks where the agent passes at least once across 3 independent runs) and pass@3 (fraction of tasks where at least one of 3 sampled trajectories passes, estimated via the standard unbiased estimator), as is standard for that benchmark. For Terminal-Bench 2.0, resolve rate is averaged over 3 independent runs per (model, backend) pair, with pass/fail determined by whether the agent's terminal interactions successfully complete the specified task.
Baselines. For Orchard-SWE (Table 7), the paper compares against a comprehensive set of open-source SWE-agent recipes organized by base-model family: the Qwen 2.5 32B Coder series includes R2EGym-Agent (Jain et al., 2025, 34.4%), Openhands-LM (Wang et al., 2025b, 37.2%), Skywork-SWE (Zeng et al., 2025, 38.0%), SWE-Agent-LM (Yang et al., 2025a, 40.2%), SWE-Mirror-LM (Wang et al., 2025a, 52.2%), SWE-Compressor (Liu et al., 2025, 57.6%), SWE-Master-32B (Song et al., 2026, 57.8%), and SWE-Master-32B-RL (61.4%); the Qwen 3 32B series includes FrogBoss (Sonwane et al., 2025, 54.6%), SWE-Lego-Qwen3-32B (Tao et al., 2026, 52.6%), and CoderForge-32B (Ariyak et al., 2026, 59.4%); the Qwen 2.5 72B series includes SWE-Fixer-72B (Xie et al., 2025, 32.8%), daVinci-Dev-72B (Zeng et al., 2026, 58.5%), Kimi-Dev (Yang et al., 2025b, 60.6%), and OpenSWE-72B (Fu et al., 2026, 65.0–66.0%); and same-size 30B-A3B baselines include the base Qwen3-30B-A3B-Instruct (22.0%), Qwen3-Coder-30B-A3B-Instruct (51.6%), GLM-4.7-Flash-30A3B (Team et al., 2025, 59.2%), and Scale-SWE-Agent (Zhao et al., 2026, 64.0%). For Orchard-GUI (Table 13), baselines include proprietary models (GPT-5 Axtree at 51.1% average, Gemini-3-flash Axtree at 51.4–61.9% depending on step budget, GPT-4o SoM at 38.6%, o3 SoM at 61.5%, GPT-5 SoM at 65.8%, OpenAI CUA at 51.3%, Gemini CUA at 69.3%) and open-source models (Holo1-7B, UI-TARS-1.5-7B at 36.4%, GLM-4.1V-9B at 44.2%, Fara-7B at 44.6%, MolmoWeb-4B at 47.4%, MolmoWeb-8B at 51.9%, Qwen3-VL-4B-Thinking at 38.1%, Qwen3-VL-235B-A22B-Thinking at 61.2%). For Orchard-Claw (Table 14), baselines include large proprietary models (Claude Opus 4.6 at 70.8% pass3 / 80.8% pass@3, GPT 5.4 at 60.2% / 75.8%, Gemini 3.1 Pro at 55.9% / 80.8%, MiniMax M2.7 at 49.7% / 72.0%) and same-scale models (Nemotron-3-nano-30b-a3b at 26.1% / 57.8%, Qwen3-30B-A3B-Thinking at 14.3% / 39.8%, Qwen3-Coder-30B-A3B-Instruct at 30.4% / 49.7%).
Generation budget / compute accounting. For SWE-bench, the compute unit is not explicitly budgeted as a generation count per evaluation instance—the agent runs until it submits a patch or hits a step/wall-clock/token limit. For the FLOPs-matched comparison (Section 7, Figure 9), compute is measured using standard approximations from scaling laws literature: pretraining FLOPs $X = 6ND_{\text{pretrain}}$ and inference FLOPs $Y = 2ND_{\text{inference}}$ , where $N$ is parameters and $D$ is tokens. The critical quantity is the ratio $R = D_{\text{inference}} / D_{\text{pretrain}}$ , with three values tested: 0.16 (R≪1), 0.79 (R≈1), and 22 (R≫1). To match total FLOPs of an M×-larger model using the smaller model with additional test-time compute, the smaller model's inference budget is multiplied by $M + 3 \cdot (D_{\text{pretrain}}/D_{\text{inference}}) \cdot (M - 1)$ . For RL training, the generation budget is tracked in terms of stride size $s = 16$ , maximum budget $N_{\text{max}} = 16$ , and training group size $N = 8$ per BAR, with maximum 150 RL steps. For Orchard-GUI, the step budget is 30 steps per episode for evaluation, with RL training using a curriculum (15 steps initially, then 30 after saturation). For Orchard-Claw, a wall-clock budget of 10 minutes per task replaces explicit step limits. Orchard Env's sandbox execution costs are measured at 0.28s average command-execution latency (Table 3), with capacity demonstrated up to 1,000 concurrent sandboxes sustaining ~154 commands/second (Table 4).
Cross-validation / statistical protocol. For Orchard-SWE's compute-optimal strategy selection (which selects the best search strategy or sequential-to-parallel ratio per difficulty bin), the paper uses two-fold cross-validation within each difficulty bin on the 500-question test set: the best-performing strategy is selected on one fold and evaluated on the other, with results averaged (Section 3.2). For the cross-harness generalization analysis (Table 8), all models are evaluated under matched conditions (same harness, same benchmark) by the authors except where original-paper numbers are explicitly marked with an asterisk. For Terminal-Bench 2.0 comparisons, results are averaged over 3 independent runs per (model, backend) pair (Table 3, right). For Orchard-GUI evaluation, the paper follows the identical protocol used by FARA and Molmo-Web for fair comparison (Section 4.1). For Orchard-Claw RL training curves (Figure 8), validation tasks are sampled from the ClawEval benchmark and tracked throughout training. The RL training uses 150 maximum steps with early stopping based on validation performance.

Main Quantitative Results

Orchard-SWE: SFT Results and Data Scaling Analysis

SFT-only performance and data scale effects. Table 9 reports SFT-only resolve rates on SWE-bench Verified under the mini-swe-agent harness, varying only the number of training trajectories (N ∈ {512, 1024, 2048}) and the strategy used to select those trajectories from the resolved-trajectory pool. Two findings dominate:

Data scale dominates selection strategy at every regime tested. Doubling data twice (512 → 2048 trajectories) on the worst-performing selection method (Diverse repo) yields a +8.2-point gain (44.0% → 52.2%), which is larger than the entire 5.5-point spread across all selection strategies at N = 512 and far larger than the 2.0-point spread at N = 2048. The spread across strategies shrinks monotonically with N: 5.5 points at N = 512, 3.2 points at N = 1024, and 2.0 points at N = 2048, indicating that at sufficient data scale, the choice of selection strategy matters much less than scale itself.

Specific behaviors worth noting: Large diff attains the strongest small-N result (49.5 at N = 512) but saturates earliest, gaining only +0.7 points from N = 1024 to N = 2048, plausibly because the pool of large-diff gold patches is finite and additional samples come from a distribution closer to the overall mean. Counterintuitively, Concentrated repo beats Diverse repo at small N by 3.9 points (47.9% vs. 44.0%), with the gap shrinking to 1.2 points at N = 2048: at small data scales, deeper exposure to a few repositories produces more transferable behaviors than thin coverage of many. Property-based selectors (Multi-file, Large diff, Composite) edge out heuristic baselines (Random, Diverse repo, Concentrated repo) at N = 512 but converge to the baselines by N = 2048, suggesting that gold-patch heuristics function as a sample-efficient prior that random sampling matches given enough data.

Even the worst (method, scale) cell in Table 9 (44.0% at N = 512) lifts the resolve rate by 22 absolute points over the underlying base model (22.0%, Table 7), confirming that even a small dose of high-quality SFT trajectories provides most of the structural lift over the base. However, the entire ablation grid plateaus around 54% under SFT-only at N = 2048, while the full Orchard-SWE recipe—using the full 107K-trajectory corpus and adding RL—reaches 67.5% on SWE-bench Verified.

Effect of credit-assignment SFT. The paper isolates the contribution of credit-assignment SFT through a controlled, scale-matched comparison. Starting from the full resolved pool, 32K resolved trajectories are sub-sampled so that the resolved baseline matches the 32,536 unresolved-trajectory rise segments in size. Two SFT models are trained with otherwise identical recipes: (i) resolved-only (32K trajectories), and (ii) resolved + unresolved with credit-assignment SFT (32K resolved + 32K rise-segment trajectories). On SWE-bench Verified, the resolved-only baseline reaches 59.3%, while adding credit-assignment SFT improves the resolve rate to 61.2%—a gain of +1.9 points. This validates that credit-assignment SFT extracts useful supervision from otherwise-discarded unresolved trajectories rather than fitting noise. In the full Orchard-SWE recipe, the same signal compounds with the larger 74.6K-resolved corpus, contributing to the headline 64.3% SFT-only result.

Orchard-SWE: Cross-Harness and Cross-Task Generalization

Harness lock-in is severe in single-harness training (Table 8). The paper evaluates three systems—Scale-SWE (trained on OpenHands trajectories), OpenSWE-32B (trained on OpenHands trajectories), and Orchard-SWE (trained on both OpenHands and mini-swe-agent trajectories)—across three harnesses (OpenHands, mini-swe-agent, Kimi-CLI) and three task distributions (SWE-bench Verified, SWE-bench Multilingual, Terminal-Bench 2.0).

Scale-SWE produces invalid outputs under any harness other than its native one, yielding no measurable resolve rate (marked ✗ in Table 8). OpenSWE-32B remains structurally valid but degrades sharply: from 62.4% on its native OpenHands to 54.9% on mini-swe-agent (−7.5 points) and 3.6% on Kimi-CLI (−58.8 points). Orchard-SWE, in contrast, holds within a narrow band of 45.0–64.3% across all three harnesses, with the worst-case drop bounded at 19.3 points relative to its own best. The two failure modes observed in Scale-SWE and OpenSWE-32B (catastrophic format failure and degraded resolve rate) have the same root cause—a model trained under a single harness has not learned harness-agnostic SWE skills.

Cross-distribution generalization (Table 8). On SWE-bench Multilingual under the mini-swe-agent harness, Orchard-SWE drops from 64.3% (Verified) to 51.0% (−13.3 absolute, −20.7% relative). OpenSWE-32B drops from 54.9% to 28.7% (−26.2 absolute, −47.7% relative). Orchard's relative drop is roughly half, indicating that multi-teacher distillation across SWE-rebench and Scale-SWE provides broader exposure to repositories and issue types than any single source alone.

Cross-domain transfer to Terminal-Bench 2.0 (Table 8). Under the Kimi-CLI harness, Orchard-SWE retains a 20.1% resolve rate, while OpenSWE-32B drops to 0.0%. Both systems degrade substantially relative to their SWE-bench Verified scores, but only Orchard-SWE retains a non-trivial level of capability on this out-of-domain benchmark. The paper hypothesizes that broader trajectory diversity during training—multiple teachers, multiple harnesses, and multiple task sources—provides indirect exposure to more varied tool-use and terminal-interaction patterns than narrower training corpora.

Controlled cross-harness SFT experiment (Table 10). Using 12K resolved trajectories on SWE-rebench from MiniMax-M2.5, varying only the collection harness and training two SFT models with otherwise identical recipes, the cross-harness matrix reveals a sharp diagonal–off-diagonal gap. Models evaluated on the same harness used during training reach 53.5–57.9% resolve rate, but performance collapses to 19.0–28.0% under the mismatched harness—a 30–40 point gap. OpenHands trajectories transfer slightly better to the simpler mini-swe-agent setting (28.0%) than the reverse (19.0%), but the dominant effect is that the model has not learned harness-agnostic SWE skills: tool-call format, observation structure, and turn-level conventions are tightly coupled to the harness seen during training.

Orchard-SWE: Full SFT+RL Results

Headline results on SWE-bench Verified (Table 7). The full Orchard-SWE recipe (74.6K resolved trajectories + 32.5K unresolved trajectories with credit-assignment SFT, followed by BAR-augmented GRPO RL with ~2K instances from SWE-rebench V2 and held-out Scale-SWE) achieves:

SFT-only: 64.3% under mini-swe-agent, 62.1% under OpenHands
SFT+RL: 67.5% under mini-swe-agent

This places Orchard-SWE (SFT+RL) as the strongest model in its active-parameter class (~3B active): it surpasses every Qwen 2.5 32B and Qwen 3 32B open-source recipe in Table 7, including OpenSWE-32B (62.4% with SWE-Agent), SWE-Master-32B-RL (61.4%), CoderForge-32B (59.4%), SWE-Mirror-LM (52.2%), and all others. It also surpasses the strongest dense 72B systems: Kimi-Dev (60.6%) and both OpenSWE-72B configurations (65.0–66.0%). The only open-source models ahead of Orchard-SWE are substantially larger: MiniMax-M2 (69.4%), MiniMax-M2.1 (74.0%), Qwen3-Coder-Max (67.0%), and GLM-4.7 (73.8%), all with active parameter counts an order of magnitude or more higher.

Same-size family lift (Table 7). The cleanest apples-to-apples comparison within the 30B-A3B family: Orchard-SWE improves over the base Qwen3-30B-A3B-Instruct by a 45.5-point absolute lift on SWE-bench Verified (22.0% base → 64.3% after SFT → 67.5% after SFT+RL). It also exceeds the code-specialized Qwen3-Coder-30B-A3B-Instruct (51.6%) by +15.9 points and the broader-distillation GLM-4.7-Flash-30A3B (59.2%) by +8.3 points. The closest competitor at comparable scale is Scale-SWE-Agent (64.0%), built on the same backbone family; Orchard-SWE matches it under SFT (64.3% vs. 64.0%) and surpasses it under SFT+RL (67.5% vs. no RL reported). This isolates the effect of the Orchard-SWE recipe itself—multi-teacher distillation, multi-harness collection, credit-assignment SFT, and BAR-augmented RL—rather than any advantage from the underlying base model.

Effect of reinforcement learning by SFT checkpoint strength. The paper compares RL initialized from two SFT checkpoints that differ by roughly two orders of magnitude in supervision: a moderate checkpoint (the Composite/N=512 cell of Table 9, 48.1% on SWE-bench Verified) and a heavy checkpoint (the full 107K-trajectory recipe, 64.3%). On SWE-bench Verified (in-distribution), RL improves the moderate init by +2.0 points (48.1% → 50.1%) and the heavy init by +3.2 points (64.3% → 67.5%). On SWE-bench Multilingual (out-of-distribution), the response diverges: from the moderate init, RL improves OOD performance by +6.7 points (22.0% → 28.7%), while from the heavy init, Multilingual slightly regresses. The paper interprets this as a specialization effect: heavy SFT places the policy on a sharper mode of the training distribution, so on-policy RL refinement sharpens in-distribution behavior at the cost of OOD transfer; a moderate base retains more behavioral diversity, so the same RL signal acts as broad-coverage refinement rather than narrow optimization.

Performance on SWE-rebench V2 (Table 6). The initial Orchard-SWE SFT checkpoint achieves 22.36% pass@1 and 27.94% pass@3 on the full Python subset of SWE-rebench V2, evaluated under mini-swe-agent. This places it competitively with several frontier proprietary models evaluated on a 60-task Python subset: it exceeds GPT-5.2 (20.56% pass@1) and gpt-oss-120b (8.89% pass@1), and approaches DeepSeek-V3.2 (23.33% pass@1) and Gemini (25.56% pass@1), though it trails Claude Opus 4.5 (36.11% pass@1) and GLM-4.7 (27.22% pass@1). These results establish that the SFT checkpoint provides a non-trivial but far-from-saturated starting point for RL on this challenging dataset, which is why the paper uses it as the basis for the RL data selection criterion (0 < \hat{p} \leq 0.5).

Orchard-GUI: SFT and RL Results

Headline results across three benchmarks (Table 13). After two-stage training (SFT on 412 curated PAE-WebVoyager trajectories, followed by judge-grounded RL on 2,198 tasks with step-budget curriculum), Orchard-GUI-4B achieves:

Benchmark	SFT-only	SFT+RL	Teacher (235B)
WebVoyager	60.2%	74.1%	63.1%
Online-Mind2Web	47.0%	67.0%	63.7%
DeepShop	48.7%	64.0%	56.7%
Average	52.0%	68.4%	61.2%

RL contributes the majority of the total gain: +13.9 points on WebVoyager, +20.0 on Online-Mind2Web, +15.3 on DeepShop, for a +16.4-point average lift over SFT.

Comparison with proprietary systems (Table 13). At 68.4% average, Orchard-GUI is competitive with the best proprietary system, Gemini computer-use-preview (69.3% average), and exceeds GPT-5 SoM (65.8%), o3 SoM (61.5%), OpenAI CUA (51.3%), GPT-4o SoM (38.6%), and all OpenAI and Gemini Axtree configurations. It achieves this with a 4B backbone versus proprietary systems that are likely orders of magnitude larger, and with only 2.6K training tasks total.

Comparison with open-source models (Table 13). Orchard-GUI is the strongest open-source GUI agent by a wide margin. The next-best open-source model, MolmoWeb-8B, achieves 51.9% average—a gap of +16.5 points. All other open-source models fall below 48%: MolmoWeb-4B at 47.4%, Fara-7B at 44.6%, GLM-4.1V-9B at 44.2%, UI-TARS-1.5-7B at 36.4%.

Several specific findings stand out within these comparisons:

On WebVoyager, Orchard-GUI is on par with the strongest open-source baselines (74.1% vs. MolmoWeb-4B's 75.2% and MolmoWeb-8B's 78.2%) while consuming roughly two orders of magnitude fewer training tasks (2.6K vs. >278.5K). WebVoyager covers only 15 popular sites with relatively short horizons, leaving little room to separate from baselines that have been heavily distilled on this exact distribution.

On Online-Mind2Web and DeepShop, Orchard-GUI substantially outperforms every previous open-source model—by +31.7 and +21.7 absolute points over MolmoWeb-8B, the strongest prior open baseline—and also surpasses its own 235B Qwen3-VL teacher by +3.3 and +7.3 points respectively, demonstrating that environment-grounded RL extracts capability the teacher itself does not exhibit.

Training dynamics (Figure 7). RL initialized from the SFT checkpoint consistently achieves higher evaluation success rates and more stable optimization behavior than RL initialized directly from the base model. While both settings obtain comparable training rewards (both reach approximately 55%), the SFT-initialized policy converges to over 50% success on the evaluation set, compared with below 40% for base-model initialization. This +10-point gap indicates that supervised initialization provides a crucial behavioral prior that stabilizes exploration and enables RL to more effectively translate reward optimization into downstream task success.

Benchmark-specific gain patterns. The largest gains appear on Online-Mind2Web, which spans a substantially broader and more diverse website distribution than either WebVoyager (15 fixed sites) or DeepShop (a single shopping vertical). The paper interprets this pattern: "Success on this benchmark therefore requires generalization to previously unseen interfaces rather than adaptation to a narrow site set. The fact that Orchard-GUI improves most strongly in this regime suggests that judge-grounded RL over a relatively small but diverse task pool can generalize across the open web more effectively than large-scale teacher distillation on narrow distributions, which is ultimately the practically relevant setting for deployable browser agents" (Section 4.5).

Orchard-Claw: SFT and RL Results

Headline results on Claw-Eval (Table 14). After two-stage training on only 192 synthetic tasks (561 trajectories, 4,537 training pairs), Orchard-Claw achieves:

SFT-only: 22.4% pass3, 50.3% pass@3
SFT+RL: 31.7% pass3, 59.6% pass@3

RL contributes +9.3 points on both metrics over the SFT checkpoint. The SFT+RL model substantially outperforms its Qwen3-30B-A3B-Thinking backbone (14.3% pass3 / 39.8% pass@3) by +17.4 and +19.8 points respectively, and also surpasses the code-specialized Qwen3-Coder-30B-A3B-Instruct (30.4% pass3 / 49.7% pass@3) on both metrics despite not being code-specialized. It also exceeds Nemotron-3-nano-30b-a3b (26.1% / 57.8%), the closest same-scale comparison.

Comparison with large proprietary models (Table 14). The gap to frontier proprietary systems remains substantial: Claude Opus 4.6 achieves 70.8% pass3 / 80.8% pass@3, GPT 5.4 at 60.2% / 75.8%, and Gemini 3.1 Pro at 55.9% / 80.8%. However, given that Orchard-Claw was trained on only 0.2K synthetic tasks, the performance is notable for its data efficiency—it approaches the pass@3 of Kimi K2.5 (67.1%) and MiniMax M2.5 (65.2%) with orders of magnitude less training data and a drastically smaller model.

Cross-harness transfer (Table 15). Pairing the Orchard-Claw SFT+RL checkpoint with the ZeroClaw harness (a faster, more lightweight Rust reimplementation of OpenClaw with features including subagents and auto-compact) lifts performance to 41.0% pass3 and 73.9% pass@3—a +9.3 and +14.3 absolute improvement over the same model run under the native ReAct-style ClawEval harness. This gain is the largest among all models in the comparison: Qwen3-30B-A3B-Thinking improves by +6.2 pass3 / +4.9 pass@3 when switching to ZeroClaw, Qwen3-Coder-30B-A3B-Instruct actually regresses on pass3 (−0.6) and improves modestly on pass@3 (+5.0), while Orchard-Claw SFT shows a +3.1 pass3 / +11.8 pass@3 improvement.

The paper attributes Orchard-Claw's exceptional cross-harness improvement to its end-to-end training using both target harnesses during rollout, enabled by Orchard Env. By exposing the agent to both the ReAct-style and ZeroClaw harnesses during training (SFT trajectories collected under both harnesses, RL rollouts running in both harnesses), the agent learns to take advantage of the features—subagents, auto-compact, and more—that stronger harnesses offer at inference time, rather than being locked into a single harness's interaction patterns.

RL training dynamics (Figure 8). Both training and validation success rates rise steadily over the course of 150 RL steps, from approximately 0.08 to 0.48 training reward and 0.35 to 0.60 validation success rate. Episode length (number of agent turns per rollout) also increases steadily from approximately 8 to 18 training turns and 5 to 7 validation turns, indicating that "the agent learns to solve more tasks while also engaging in longer multi-turn interactions" (Section 5.4).

Orchard Env: System Evaluation

Execution latency (Table 3, left). Orchard Env achieves an average command-execution latency of 0.28 seconds, essentially matching SkyPilot Code Sandbox (0.284s) and significantly outperforming E2B (0.747s, 2.7× slower) and Modal (2.046s, 7.3× slower). This validates Orchard Env's direct Pod-IP communication design: by routing execution requests directly to the in-pod agent and bypassing the Kubernetes API server on the hot path, Orchard Env achieves latency comparable to optimized native runtimes while retaining the flexibility of a Kubernetes-based deployment.

Reliability under concurrency (Table 4). In a stress test of 1,000 parallel sandboxes through the full lifecycle (create → 4× exec → delete), Orchard Env achieved a 100% success rate across all 1,000 sessions—no failures on creation, execution, or cleanup—with the entire test completing in 26 seconds end-to-end. This translates to approximately 154 commands per second sustained throughput across the full create→exec→delete lifecycle (4,000 commands across 1,000 sandboxes in 26 seconds), well above the throughput required by typical agentic distillation and RL workloads.

Functional equivalence to Docker (Table 3, right). On Terminal-Bench 2.0, comparing Orchard Env against a direct Docker baseline across three models (GPT-4.1, MiniMax-M2.5, Qwen3-8B-Thinking), Orchard Env matches Docker within run-to-run variance in every case, with a marginal edge of 1–2 points for each model (GPT-4.1: 35.1% vs. 34.1%; MiniMax-M2.5: 54.4% vs. 52.6%; Qwen3-8B-Thinking: 8.8% vs. 7.0%). This confirms that the agent-injection mechanism and Orchard Env's execution path introduce no observable overhead or interference in agent–environment interactions.

Cost comparison (Table 2). For a representative RL training workload (128 parallel sandboxes at 2 vCPU, 8 GiB each over 240 hours = 30,720 sandbox-hours), Orchard Env on spot instances costs ** $673**—10× lower than managed alternatives Daytona and E2B ($ 7,078 each) and 15× lower than Modal ( $10,305). Even at on-demand rates, Orchard Env ($ 3,362) is less than half the cost of Daytona and E2B. The paper notes that these cost differences "compound over the course of a research project. Generating 160K rollout trajectories, running ablation studies, and iterating on training recipes can easily require thousands of hours of environment interaction. Orchard's self-hosted, spot-friendly design makes such workloads practical for academic research budgets" (Appendix B).

Ablation Studies and Robustness Checks

Data scale vs. selection strategy (Table 9). As discussed above, doubling SFT data from 512 to 2048 trajectories dominates any selection strategy effect, with the worst-performing selection method gaining +8.2 points versus a maximum 5.5-point spread across strategies. The spread across strategies shrinks monotonically with data scale, and property-based selectors converge to heuristic baselines by N = 2048.

Cross-harness SFT training (Table 10). Training on trajectories from a single harness produces models that collapse 30–40 points when evaluated under a mismatched harness (mini-swe-agent→OpenHands: 57.9% → 19.0%; OpenHands→mini-swe-agent: 53.5% → 28.0%), confirming that single-harness training teaches harness-specific formatting rather than harness-agnostic domain skills.

Credit-assignment SFT (Section 3.6). In a scale-matched comparison (32K resolved + 32K rise-segment vs. 32K resolved-only), credit-assignment SFT adds +1.9 points on SWE-bench Verified (59.3% → 61.2%), validating that rise-segment extraction provides genuine supervision rather than fitting noise from failed trajectories.

Effect of RL by SFT checkpoint strength (Section 3.6). RL from a moderate SFT init (48.1% SWE-bench Verified) improves both in-distribution (+2.0 points) and OOD (+6.7 points on Multilingual), while RL from the heavy full-recipe init (64.3% SWE-bench Verified) improves in-distribution (+3.2 points) but slightly regresses on OOD, indicating a specialization-vs-generalization tradeoff governed by the breadth of the SFT checkpoint.

Harness generalization across independently developed systems (Table 8). Scale-SWE produces invalid outputs under any non-native harness, yielding zero measurable resolve rate. OpenSWE-32B drops from 62.4% to 3.6% on an unseen harness. Orchard-SWE retains 45.0–64.3% across all three tested harnesses, with bounded degradation.

Teacher model and harness diversity in trajectories. On SWE-bench Multilingual, Orchard-SWE's relative degradation (−20.7%) is roughly half of OpenSWE-32B's (−47.7%), and on Terminal-Bench 2.0, Orchard-SWE retains 20.1% while OpenSWE-32B drops to 0.0%, suggesting that multi-teacher, multi-harness trajectory collection provides broader generalizable skills than single-source distillation.

RL initialization for GUI agents (Figure 7). SFT-initialized RL achieves approximately +10 points higher evaluation success than base-model-initialized RL at convergence, despite similar training rewards, confirming that supervised initialization provides a behavioral prior that stabilizes RL exploration.

Step-budget curriculum for GUI RL (Section 4.4). Progressing from a 15-step budget to a 30-step budget after performance saturation extends the policy to harder tasks requiring more interaction, though this is presented as a qualitative design choice rather than a quantitative ablation.

Cross-harness RL for claw agents (Table 15). Training with both ReAct-style and ZeroClaw harnesses during SFT and RL produces a model that benefits substantially from the stronger harness at inference time (+9.3 pass3, +14.3 pass@3), while baseline models trained on a single harness benefit less or even regress (Qwen3-Coder-30B-A3B-Instruct: −0.6 pass3), providing evidence that end-to-end multi-harness training enables cross-harness skill transfer.

Environment service latency comparison (Table 3). Orchard Env's 0.28s average command-execution latency matches the fastest alternative (SkyPilot Code Sandbox, 0.284s) and is 2.7× faster than E2B and 7.3× faster than Modal, confirming that the direct Pod-IP communication design achieves near-optimal execution throughput.

Environment service functional equivalence (Table 3, right). Agent pass rates on Terminal-Bench 2.0 using Orchard Env vs. direct Docker show no regression beyond run-to-run variance across three models, confirming that the agent-injection mechanism introduces no observable interference.

Critical Assessment

The experimental evaluation in this paper is comprehensive in scope—spanning three task domains, multiple harnesses, SFT and RL stages, infrastructure benchmarking, and cost analysis—but it has important limitations that affect how strongly specific claims are supported.

Claim: "A thin, open, harness-agnostic environment layer enables trajectory data, SFT recipes, RL rollouts, and evaluation protocols to transfer across domains, harnesses, and pipeline stages"

Supported with strong evidence within the domains tested, but the claim of cross-domain transfer is only partially demonstrated. The paper provides compelling evidence that the same Orchard Env service supports software engineering, GUI navigation, and personal assistant workflows—three very different task types with different Docker images, tool interfaces, and reward mechanisms. The same SFT+RL recipe pattern (teacher distillation → SFT → environment-grounded RL) is instantiated in all three domains. This demonstrates that the environment service does not impose domain-specific constraints.

However, the paper does not demonstrate cross-domain transfer of trained models—Orchard-SWE is not evaluated on GUI tasks, and Orchard-GUI is not evaluated on SWE tasks. The "transfer" demonstrated is of infrastructure and recipe patterns, not of learned capabilities. The claim in the abstract that "a thin, open, harness-agnostic environment layer enables the reuse of agentic data, training recipes, and evaluation protocols across domains" is supported for recipes and evaluation protocols, but the reuse of data across domains is not demonstrated—SWE trajectories are not used to train GUI agents or vice versa. The paper would be stronger with even a small-scale demonstration that trajectories from one domain provide useful auxiliary supervision for another (e.g., does exposure to terminal interactions in SWE trajectories improve Terminal-Bench performance for a model not explicitly trained on it?).

Additionally, the claim of transfer "across pipeline stages" (distillation, RL rollouts, evaluation) is demonstrated but somewhat circular: Orchard Env is designed to be a generic execution backend, so of course the same API can serve all three stages. The more interesting claim would be that data generated during RL rollouts could be recycled for SFT, or that evaluation environments could be identical to training environments (closing the distribution gap)—neither is quantitatively demonstrated.

Claim: "Orchard-SWE achieves 67.5% on SWE-bench Verified, setting a new state of the art among open-source models of comparable size"

Strongly supported with thorough baselines and apples-to-apples comparisons, but with a single-model-family limitation. Table 7 provides an exhaustive comparison against 20+ open-source SWE-agent recipes across multiple model families and scales, and the paper makes a genuine effort to isolate the recipe's contribution by comparing within the same 30B-A3B backbone family. The 45.5-point absolute lift over the base model (22.0% → 67.5%) is the cleanest evidence that the training recipe, not the base model, drives the result.

However, all training data, RL hyperparameters, and evaluation are specific to a single backbone (Qwen3-30B-A3B-Thinking). The paper does not report results applying the same recipe to a different model family (e.g., Llama, DeepSeek) or even to a different Qwen variant (e.g., Qwen3-32B dense). The generalizability of the recipe—which is the core claim of the paper—is therefore untested at the model level. A minimal robustness check would be applying the same 107K-trajectory SFT to a different base model and reporting whether the lift is comparable.

Claim: "Credit-assignment SFT extracts partial-progress signals from unresolved trajectories"

Supported but modestly—the +1.9 point gain on SWE-bench Verified is statistically meaningful (it appears in a controlled, scale-matched comparison) but small relative to the total 45.5-point lift over the base model. The method's value is more conceptual than empirical at this scale: it validates that unresolved trajectories contain exploitable signal, but the observed gain suggests that the signal is a modest supplement to resolved-trajectory SFT rather than a breakthrough. The paper would be stronger with an ablation showing what happens when credit-assignment SFT is applied at larger scale (e.g., when 100K+ unresolved trajectories are available, does the gain compound?) or when it is the primary supervision for very-hard tasks where resolved trajectories are scarce.

The mechanism's calibration is asserted rather than empirically validated: the paper states that value curves are "inverted-U in 98.9% of cases" but does not provide human validation of the value estimates themselves. Are the teacher model's probability estimates at specific steps accurate when compared to human judgments of progress? Without this, there is a risk that the rise segments are extracting teacher-model biases rather than genuine partial progress.

Claim: "Balanced Adaptive Rollout (BAR) improves RL efficiency for sparse-reward agentic tasks"

Plausible but not directly ablated. The paper describes BAR in detail and argues for its necessity, but never reports an ablation comparing RL with BAR vs. standard fixed-N GRPO at the same total generation budget. The RL stage adds +3.2 points over the SFT checkpoint (64.3% → 67.5%), but BAR's specific contribution to that gain is not isolated—it could be that the same RL recipe without BAR (using standard GRPO with fixed N=8 and discarding zero-variance groups) would achieve a similar gain, or would require more steps to converge, or would fail entirely. The paper's arguments for BAR are theoretical and qualitative (the two problems with fixed-N GRPO for sparse rewards) rather than empirical.

The BAR hyperparameters (stride s = 16, N_max = 16) also mean that in practice, BAR generates all trajectories in one batch and then assembles the group—it never takes advantage of the stride-based adaptive stopping because s = N_max. This makes BAR functionally equivalent to a post-hoc group assembly algorithm for the specific configuration used, rather than the progressive, self-pacing rollout schedule described in the algorithm. A stride of s = 4 with N_max = 16 would demonstrate the adaptive behavior (some prompts stop after 1 stride, others after 2, 3, or 4), but this is not tested.

Claim: "Orchard-GUI achieves 68.4% average success, the strongest open-source result"

Strongly supported with appropriate baselines. Table 13 provides a comprehensive comparison against both proprietary and open-source systems across three benchmarks with different characteristics. The 4B backbone result is genuinely surprising—it exceeds its own 235B teacher on two of three benchmarks and approaches the best proprietary system—and the paper's explanation (environment-grounded RL extracts capability beyond teacher distillation) is supported by the training curves showing the SFT→RL improvement.

However, the evaluation relies on GPT-4.1 as judge for success/failure determination. Judge-model bias could inflate or deflate results relative to baselines if different papers use different judges, and the paper does not report inter-judge agreement or human validation of the judge's decisions on a subset of trajectories. The paper follows the evaluation protocol of FARA and Molmo-Web "for fair comparison," but if those protocols use different judges, the comparison may not be as fair as claimed.

The RL training curves (Figure 7) show evaluation success rate of only ~50%, yet the final model achieves 68.4% average across three benchmarks with substantially different characteristics. This apparent discrepancy is not explained—the evaluation curve likely tracks performance on a held-out validation set that differs from the final test benchmarks, but the composition of that validation set is not specified.

Claim: "Orchard-Claw achieves 59.6% pass@3 on Claw-Eval with only 0.2K synthetic tasks"

Supported but with a small absolute scale caveat. The 192 synthetic tasks represent an extremely small training set, and the 59.6% pass@3 is genuinely higher than the 49.7% of the code-specialized Qwen3-Coder-30B-A3B-Instruct. However, the absolute pass3 of 31.7% means the model passes on all 3 of 3 runs for fewer than one-third of tasks—the gap to frontier proprietary systems (70.8% pass3, 80.8% pass@3 for Claude Opus 4.6) is enormous. The result demonstrates that RL can improve a model from a very small SFT set, but the absolute performance is nowhere near production-usable, and the paper does not explore what would happen with even modest scaling of training data (e.g., 1K or 10K synthetic tasks).

Broader limitations not addressed by specific claims

Single-benchmark focus per domain. Each Orchard recipe is evaluated primarily on one benchmark (SWE-bench Verified for SWE, three-web-benchmark average for GUI, Claw-Eval for claw agents), with auxiliary evaluations on one or two additional benchmarks. This is standard for the field but means the paper does not test whether the training recipes produce generalist agents or benchmark specialists. Orchard-SWE's strong performance on SWE-bench Multilingual (51.0%) is encouraging, but the drop from Verified (64.3%) is substantial (−20.7% relative), and Terminal-Bench 2.0 (20.1%) suggests the capability degrades rapidly as the task distribution diverges from GitHub issue resolution.

No combination of the three recipes. The paper presents Orchard-SWE, Orchard-GUI, and Orchard-Claw as independent instantiations. The logical extension—training a single model on all three domains using the shared Orchard Env infrastructure—is not attempted. This would be the strongest test of the claim that the environment layer enables cross-domain reusability of training data and recipes, but it is left to future work.

Missing ablations on critical hyperparameters. For credit-assignment SFT, the rise-segment threshold epsilon = 0.05 is stated without sensitivity analysis. For BAR, the target positive fraction interval [0.375, 0.625] is stated without showing what happens with wider or narrower intervals. For Orchard-GUI, the screenshot history truncation (keeping only the last k screenshots) does not specify k or ablate its effect. For Orchard-Claw, the 10-minute wall-clock budget is stated without comparison to step-limited alternatives.

Difficulty estimation cost is unaccounted for in the practical deployment picture. While not central to the experimental results in the same way as the reference example paper (where difficulty estimation required 2,048 samples per question), the paper's SWE data selection for RL involves running the SFT model on each candidate task with 8 rollouts to estimate pass rates—a cost that is not included in any compute budget. For the RL training set of ~2K instances, this is 16K rollouts just for data selection, which is non-trivial.

Statistical significance. None of the experimental results include confidence intervals, standard deviations, or statistical tests. The SWE-bench Verified test set is 500 instances; a difference of 1–2 points between methods could easily fall within sampling error for a 500-question binary classification task. The Terminal-Bench 2.0 results are averaged over 3 runs but do not report variance. The Orchard-GUI evaluation uses LLM-as-a-judge with no reported inter-judge reliability.

The 14× larger model baseline for FLOPs-matched comparison is deferred to Section 7 and uses a PaLM 2 model, not a Qwen model. This means the pretraining-vs-inference comparison crosses model families, architectures, and training data distributions, making it impossible to attribute performance differences to the test-time compute strategy vs. pretraining scale specifically. A same-family comparison (e.g., Qwen3-30B-A3B with compute-optimal inference vs. Qwen3-235B with greedy decoding) would isolate the compute-allocation effect, but this is not provided.

6. Limitations and Trade-offs

Difficulty Estimation and Data Selection Costs Are Unaccounted for in Headline Efficiency Numbers

The assumption or constraint. Both Orchard-SWE and Orchard-GUI perform substantial offline computation to filter and select training data before SFT or RL begins, and these costs are excluded from all reported efficiency metrics. For Orchard-SWE, the RL data selection pipeline (Section 3.3.3) requires running the SFT checkpoint on each candidate task with 8 rollouts to estimate its initial pass rate $\hat{p}$ , then retaining only tasks with $0 < \hat{p} \leq 0.5$ . The final RL training set contains approximately 2K instances, meaning that roughly 16K rollouts were consumed purely for task selection before RL ever begins—equivalent to the cost of a full RL training run at the paper's reported stride of $s = 16$ . For Orchard-GUI, the task filtering pipeline (Section 4.3, Figure 4) starts from 292K raw tasks, applies five filtering stages including semantic deduplication using Qwen3-Embedding-8B embeddings, and samples 4 rollouts per remaining task (62,395 teacher rollouts) to estimate per-task success rates for curation—a computation that dwarfs the 2.6K tasks ultimately used for SFT+RL.

The consequence. The paper's central claim—that Orchard recipes are practical for academic research budgets—is undermined by the exclusion of these upfront costs. The cost analysis in Table 2 and Appendix B estimates $673 for a 240-hour, 128-sandbox RL training run on spot instances, but does not include the cost of the offline pass-rate estimation that generated the training set in the first place. For Orchard-SWE, this pre-RL computation alone represents thousands of additional sandbox-hours that could equal or exceed the RL training cost itself. For Orchard-GUI, the 62,395 teacher rollouts at 4 rollouts per task across 15,601 seed tasks—using Qwen3-VL-235B-A22B-Thinking as the teacher—represent a computational cost that is not amortized in any reported metric. A practitioner attempting to replicate these results would discover that the "cheap" recipe actually requires an expensive data preparation phase that is not priced into the published numbers.

What evidence exists in the paper. The paper acknowledges the offline data selection step for Orchard-SWE RL (Section 3.3.3: "We first run the initial SFT model on each candidate task with 8 rollouts to get its initial pass rate") and describes the full task filtering pipeline for Orchard-GUI in detail (Section 4.3, Figure 4), but never accounts for these costs in any compute budget, FLOPs comparison, or cost analysis. The RL training configuration in Section 3.3.3 counts only the 150 RL steps themselves, not the prerequisite 16K rollouts. The GUI training configuration in Section 4.4 counts only the 2.6K SFT+RL tasks, not the 62,395 teacher rollouts that preceded them.

Mitigation status. Not addressed. The paper does not suggest that these costs could be amortized (e.g., by reusing pass-rate estimates across experiments) or reduced (e.g., by using a lightweight proxy for difficulty estimation rather than full rollouts). The omission is particularly striking given that Orchard Env's cost-efficiency is one of the paper's three core selling points (Section 2), and the unaccounted costs are large enough to alter the conclusion about affordability.

All Three Recipes Are Demonstrated on a Single Model Family; Recipe Transferability to Other Architectures Is Untested

The assumption or constraint. Orchard-SWE and Orchard-Claw use Qwen3-30B-A3B-Thinking (a 30B MoE model with ~3B active parameters) as the backbone for all training and evaluation. Orchard-GUI uses Qwen3-VL-4B-Thinking, from the same Qwen3 model family. The paper does not report results applying the same training recipes—the same 107K-trajectory SFT dataset, the same credit-assignment SFT procedure, the same BAR-augmented RL pipeline, or the same 2.6K-task GUI recipe—to any model from a different family (e.g., Llama, DeepSeek, Gemma, Mistral). The paper states in Section 4 that it chose the Qwen3 backbone because it is "representative of the capabilities of many contemporary LLMs," but provides no empirical evidence that other base models would respond similarly to the same data and hyperparameters.

The consequence. The headline results—64.3% SFT, 67.5% SFT+RL on SWE-bench Verified—may be specific to the Qwen3-30B-A3B architecture, its training data distribution, its MoE routing patterns, or its particular sensitivity to multi-turn agentic SFT. Different model families exhibit different in-context learning capabilities, tool-use aptitudes, and multi-turn coherence properties out of the box (as evidenced by the fact that the base Qwen3-30B-A3B-Instruct achieves only 22.0% on SWE-bench Verified while Qwen3-Coder-30B-A3B-Instruct achieves 51.6%—a 29.6-point gap from code-specialized fine-tuning alone). A practitioner using a Llama-3 or DeepSeek base model cannot assume that the same 107K-trajectory corpus and training hyperparameters would produce a comparable lift, because the interaction between base model quality and SFT data efficiency is not characterized. More concerningly, the paper's claims about BAR, credit-assignment SFT, and multi-harness training are all evaluated on a single architecture; it is impossible to know whether these methods generalize or are accidentally well-tuned to Qwen3-specific behaviors (e.g., Qwen3's particular sensitivity to long-context training, its MoE routing stability under RL, or its tool-calling format adherence after SFT).

What evidence exists in the paper. All results in Tables 7, 8, 9, 10 (Orchard-SWE), Tables 13 (Orchard-GUI), and Tables 14, 15 (Orchard-Claw) use Qwen3 backbones exclusively. The paper provides comprehensive comparisons against other recipes that use different base models (OpenSWE-32B uses Qwen2.5-32B, SWE-Master-32B uses Qwen2.5-Coder-32B-Instruct, etc.), but these comparisons vary both the recipe and the base model simultaneously, making it impossible to isolate the recipe's contribution from the base model's quality. Table 7 shows that Orchard-SWE (67.5%) outperforms OpenSWE-72B (66.0%) on SWE-bench Verified—but this comparison crosses model families, scales, and architectures, so the 1.5-point difference could be driven entirely by Qwen3-30B-A3B being a stronger base than Qwen2.5-72B for this task, independent of the Orchard recipe.

Mitigation status. The paper does not address this limitation. Section 8 (the paper's only forward-looking discussion) does not mention multi-model evaluation as future work. The paper's framing—that the recipes themselves are the contribution—is weakened by the absence of even a single cross-family transfer experiment (e.g., training Llama-3-8B on the same 107K trajectories and reporting the lift over its base performance).

BAR's Adaptive Behavior Is Not Exercised at the Reported Hyperparameters; Its Specific Contribution to RL Gains Is Not Isolated

The assumption or constraint. The Balanced Adaptive Rollout (BAR) algorithm is presented as a progressive, stride-based, self-pacing rollout schedule that "adaptively continues generation only until it can construct a fixed-size training group whose positive-reward fraction lies in a target interval" (Section 3.3.3). However, the specific hyperparameters used in the paper—stride $s = 16$ , maximum budget $N_{\text{max}} = 16$ —mean that BAR generates all trajectories in a single batch and then performs post-hoc group assembly. The progressive, early-stopping behavior that distinguishes BAR from a simple post-hoc filter is never exercised: because $s = N_{\text{max}}$ , the algorithm either succeeds on the first (and only) stride or enters fallback, never demonstrating the intermediate regime where it generates one stride, checks, generates another, checks again, and stops early. At these hyperparameters, BAR is functionally equivalent to generating 16 trajectories and then calling TRYASSEMBLE once—a post-hoc group assembly algorithm, not an adaptive rollout scheduler.

Furthermore, the paper never reports an ablation comparing RL with BAR against RL with standard fixed-N GRPO (e.g., N=8 with zero-variance group discarding) at the same total generation budget. The RL stage adds +3.2 points over SFT (64.3% → 67.5%), but BAR's specific contribution to this gain is unknown. It could be that standard GRPO with post-hoc filtering of zero-variance groups would achieve the same +3.2 point gain with the same number of RL steps, or that BAR's overhead (generating 16 trajectories to produce a group of 8) is unnecessary and a simpler fixed-N=8 approach with group filtering would be equally effective and 2× cheaper in environment interactions.

The consequence. The paper makes strong claims about BAR's role in enabling efficient sparse-reward RL ("BAR turns a fixed-batch rollout into a self-pacing, information-dense one," Section 3.3.3), but these claims are not empirically validated. A practitioner implementing the Orchard-SWE recipe cannot know whether BAR is essential, beneficial, or merely decorative relative to simpler alternatives. If BAR is not contributing beyond post-hoc filtering, then the 2× generation overhead (N_max = 16 to produce N = 8 training trajectories) is pure waste—doubling the already-expensive environment interaction cost of RL. Given that environment interactions are the dominant cost in agentic RL (Table 2: $673 per 240-hour run for sandboxes alone, not counting GPU inference), a 2× overhead from an unvalidated component is a significant practical concern.

What evidence exists in the paper. The BAR algorithm description (Section 3.3.3, Algorithm 1) uses $s = 16$ and $N_{\text{max}} = 16$ , meaning the algorithm's while loop over strides executes at most once. There is no ablation in Section 3.6 comparing BAR vs. fixed-N GRPO, BAR vs. post-hoc filtering without stride-based generation, or BAR with different stride sizes (e.g., $s = 4, N_{\text{max}} = 20$ ) that would demonstrate the adaptive behavior. The only evidence for BAR's effectiveness is the overall RL gain (+3.2 points), which confounds BAR with all other RL design choices (data selection, reward normalization, group filtering, learning rate schedule, optimizer configuration).

Mitigation status. Not addressed. The paper does not acknowledge that the reported hyperparameters make BAR's adaptive behavior untested, does not provide ablations isolating BAR's contribution, and does not suggest that future work should validate BAR's necessity. In the absence of ablation evidence, BAR should be viewed as an interesting algorithmic proposal with an untested empirical contribution, not as a validated component of the Orchard-SWE recipe.

Hard Problems Remain Essentially Unsolved; Test-Time Compute Cannot Substitute for Fundamental Capability Gaps

The assumption or constraint. Across all three Orchard recipes, the hardest instances in each domain show minimal improvement regardless of training data scale, RL investment, or harness choice. For Orchard-SWE, difficulty bin 5 (the hardest 20% of SWE-bench Verified instances, where even the base model's pass@1 is near zero) would presumably show near-zero resolve rates after SFT and RL—the paper does not report per-difficulty-bin breakdowns for the final model, but the ablation in Section 3.6 shows that even at N=2048 SFT trajectories under the best selection strategy, the resolve rate plateaus around 54% (Table 9), meaning nearly half of all instances—likely concentrated in the hardest bins—are never solved. Similarly, in the related work on test-time compute scaling that Orchard builds upon conceptually, the hardest difficulty bin shows near-zero improvement regardless of compute budget. For Orchard-GUI, the teacher model (Qwen3-VL-235B) fails on 31.6% of seed tasks entirely (all four rollouts fail; Figure 6), and of those, 41.1% are captcha-blocked—meaning that a substantial fraction of tasks are simply unsolvable with current browser automation, regardless of agent capability. For Orchard-Claw, the pass3 on Claw-Eval is 31.7%, meaning the model passes all three of three runs on fewer than one-third of tasks.

The consequence. Orchard's training recipes amplify existing capability but do not create it. If a task is fundamentally outside the base model's reach—the issue is too complex, the website deploys aggressive anti-bot countermeasures, the claw workflow requires reasoning the model cannot perform—no amount of SFT scaling, credit-assignment data mining, or BAR-augmented RL will make it solvable. This is an intrinsic ceiling on the approach that the paper does not quantify or characterize. A practitioner deploying Orchard-SWE on a new distribution of GitHub issues (e.g., a private repository with different coding conventions, test frameworks, or dependency structures) cannot predict what fraction of issues will fall into the "solvable" vs. "unsolvable" regime, because the paper does not provide a difficulty model that generalizes beyond the SWE-bench distribution.

Moreover, the economic argument for agentic training over pretraining—that investing in better SFT+RL recipes on smaller models is more cost-effective than training larger models—breaks down precisely for the hardest instances, where pretraining scale may be the only viable path to capability. The paper's FLOPs-matched analysis (Section 7, Figure 9) acknowledges this for search-based methods, showing that test-time compute with a smaller model underperforms a ~14× larger model on hard problems at all inference-to-pretraining ratios. The same logic likely applies to agentic training: beyond a certain difficulty threshold, the base model simply lacks the conceptual machinery to solve the task, and no recipe—however well-designed—can compensate.

What evidence exists in the paper. The paper provides indirect evidence of this ceiling through: (1) the SFT-only ablation plateauing around 54% at N=2048 (Table 9), despite the full recipe reaching 67.5%, suggesting that 32.5% of instances remain unsolved even after RL; (2) the teacher model's 31.6% all-fail rate on GUI seed tasks (Figure 6, left); (3) Orchard-Claw's 31.7% pass3 on Claw-Eval (Table 14); (4) the cross-task degradation from SWE-bench Verified (67.5%) to Multilingual (51.0%) to Terminal-Bench 2.0 (20.1%) in Table 8, showing that capability degrades rapidly as task distribution shifts. The paper does not provide a per-difficulty-bin breakdown of the final SFT+RL model's performance, making it impossible to determine whether the +3.2-point RL gain came from solving previously-unsolved hard instances or from improving success rates on already-solvable medium instances.

Mitigation status. The paper does not acknowledge this as a fundamental limitation. Section 1 frames the recipes as demonstrating "scalable agentic modeling," and Section 3.6 argues that the SFT plateau at 54% is overcome by adding RL and scaling to 107K trajectories—but even the full recipe leaves 32.5% of SWE-bench Verified instances unresolved. There is no discussion of what characterizes the unsolved instances, whether they represent a qualitatively different type of task, or whether they might be solvable with a larger base model or fundamentally different training approach. The paper also does not suggest future work on difficulty characterization or adaptive compute allocation that might route unsolvable instances to larger models or human reviewers.

Cross-Domain and Cross-Harness Generalization Is Evaluated Only Within Narrow Distribution Shifts; True Out-of-Distribution Transfer Is Neither Demonstrated Nor Characterized

The assumption or constraint. The paper's central claim is that a thin, harness-agnostic environment layer enables reusability "across domains, agent harnesses, and pipeline stages" (Section 1). The empirical evidence for this claim relies on generalization tests that stay within the same broad task family. Orchard-SWE is tested across harnesses (OpenHands, mini-swe-agent, Kimi-CLI) and task distributions (SWE-bench Verified, Multilingual, Terminal-Bench 2.0), but all three task distributions are in the software engineering / terminal interaction family—they involve interacting with code repositories, running shell commands, and editing files. Orchard-GUI is tested on three web-navigation benchmarks (WebVoyager, Online-Mind2Web, DeepShop) that share the same fundamental interaction paradigm (browser-based, screenshot observations, click/type/scroll actions). Orchard-Claw is tested on Claw-Eval under two harnesses that share the same underlying tool set (email, calendar, inventory, scheduling APIs).

The consequence. The paper does not demonstrate that the Orchard recipes produce agents with general-purpose agentic capabilities that transfer across fundamentally different task paradigms. An Orchard-SWE model trained on GitHub issue resolution cannot be dropped into a browser and expected to navigate WebVoyager—the action spaces, observation modalities, and reward structures are completely different. This is obvious and not claimed, but it reveals a gap between the paper's rhetoric (reusability "across domains") and its empirical evidence (reusability within related task families). A practitioner who needs an agent spanning multiple domains (e.g., an assistant that can both fix bugs in a codebase and book travel on a website) would need to train separate models or develop a multi-domain training recipe that the paper does not provide, despite having all three recipes available and sharing the same Orchard Env infrastructure.

More subtly, even within-domain generalization is incompletely characterized. The cross-harness test in Table 8 shows that Orchard-SWE retains 45.0–64.3% across harnesses, but the Kimi-CLI harness—which was not used during training—produces a 45.0% resolve rate, a 19.3-point drop from the best harness (mini-swe-agent at 64.3%). This is better than the catastrophic collapse of Scale-SWE and OpenSWE-32B, but it still represents a 30% relative degradation. The paper frames this as a success ("the worst-case drop bounded at 19.3 points"), but a practitioner cannot assume that Orchard-SWE will work well with an arbitrary future harness. The harness-lockin problem is reduced, not solved, and the reduction is only tested on three harnesses, all of which were used by at least one of the two teacher models (OpenHands and mini-swe-agent) during trajectory collection, potentially leaking harness-specific formatting into the training data even for the "unseen" Kimi-CLI harness if it shares syntactic conventions with the training harnesses.

What evidence exists in the paper. Table 8 provides the cross-harness and cross-task generalization evidence. Table 15 provides cross-harness evidence for claw agents. Both show substantial improvements over single-harness baselines, but also show large absolute degradation relative to in-distribution performance. The paper does not test generalization to a harness that uses fundamentally different tool-calling conventions (e.g., a harness that uses a different serialization format for tool calls, or a harness that uses function-calling via API rather than text-based tool schemas). It does not test generalization to task distributions that require qualitatively different reasoning (e.g., SWE tasks in a language not represented in the training data, or GUI tasks on mobile rather than desktop websites). The paper acknowledges these limits implicitly by not making claims beyond the tested distributions, but the central thesis—that the environment layer enables cross-domain reusability—is supported only for closely-related domains that share interaction paradigms.

Mitigation status. The paper does not discuss the scope of tested generalization or acknowledge the gap between claimed cross-domain reusability and demonstrated within-family transfer. The phrase "across domains" in Section 1 is used to describe the infrastructure's potential, but the empirical evidence demonstrates transfer across tasks and harnesses within a domain, not across domains. Section 8 (Conclusion) does not call for more ambitious cross-domain transfer experiments as future work, despite having all three recipes available to attempt a multi-domain training run that would directly test the central thesis.

Evaluation Relies on Unvalidated LLM-as-Judge for GUI Tasks and on a 500-Instance Test Set Without Statistical Significance Reporting

The assumption or constraint. Orchard-GUI's success rates on WebVoyager, Online-Mind2Web, and DeepShop are all determined by GPT-4.1 acting as a judge, evaluating whether the agent's final done(response) and the screenshot trail satisfy the user intent (Section 4.4: "A trajectory receives +1 when every assistant turn parses ... and the final done(response) is judged SUCCESS by GPT-4.1 against the screenshot trail and user intent"). The paper follows the evaluation protocol of FARA (Awadallah et al., 2025) and Molmo-Web (Gupta et al., 2026) "for fair comparison," but does not report inter-judge agreement, human validation of the judge's decisions, or comparison against alternative judges. Additionally, all SWE-bench Verified results are evaluated on a 500-instance test set without confidence intervals, standard errors, or statistical significance tests for pairwise comparisons between systems. A 1.5-point difference between Orchard-SWE (67.5%) and OpenSWE-72B (66.0%) on 500 binary outcomes—a difference of approximately 7 extra correct resolutions—could easily fall within one standard error of the measurement.

The consequence. The impressive Orchard-GUI numbers (74.1% / 67.0% / 64.0%) may partially reflect judge-model bias rather than genuine task success. If GPT-4.1 is systematically more lenient than the judges used by baselines—or if GPT-4.1's judgment is correlated with the trajectory's superficial qualities (length, verbosity, confidence) rather than task completion—the comparison against baselines evaluated under different protocols is not truly "fair." The paper does not report what judge was used for the baseline numbers cited from prior work, making it impossible to assess whether judge-model differences could explain the 16.5-point gap to MolmoWeb-8B (51.9%) or the 7.2-point gap to Gemini CUA (69.3% vs. 68.4% for the average).

For SWE-bench, the 500-instance test set is large enough to detect major differences but too small to reliably distinguish between systems separated by 1–3 points. The paper's Table 7 compares Orchard-SWE (SFT+RL, 67.5%) against OpenSWE-72B (66.0%) and Qwen3-Coder-Max (67.0%), presenting these as meaningful differences. But on a 500-instance binary classification task, the 95% confidence interval for a 67.5% resolve rate is approximately ±4.1 points (using the normal approximation: $1.96 \times \sqrt{0.675 \times 0.325 / 500} \approx \pm 4.1\%$ ). The difference between 67.5% and 66.0% (1.5 points) is well within this margin of error, meaning the claimed superiority over OpenSWE-72B is not statistically reliable. The difference between Orchard-SWE (67.5%) and the base Qwen3-30B-A3B-Instruct (22.0%)—45.5 points—is clearly significant, but many of the finer-grained comparisons in Table 7 are not.

What evidence exists in the paper. The paper reports neither judge reliability statistics for Orchard-GUI nor confidence intervals for any experimental result. The 500-instance SWE-bench Verified test set size is standard for the field (all papers in Table 7 use it), so Orchard is not uniquely vulnerable to this criticism—but the paper's specific claims about surpassing specific systems (e.g., "surpasses every Qwen 2.5 32B and Qwen 3 32B open-source recipe," Section 3.4) are made without acknowledging that several of those comparisons fall within plausible sampling error. The GUI results cite the FARA/Molmo-Web evaluation protocol as justification for the judge choice, but this protocol is a community convention—not a validated measurement instrument—and the paper does not provide evidence that the protocol produces consistent rankings across different judge models.

Mitigation status. Not addressed. The paper does not report confidence intervals, does not discuss the statistical power of the 500-instance test set, does not validate the GPT-4.1 judge against human judgments or alternative judges, and does not temper its comparative claims with appropriate uncertainty quantification. These are standard practices in mature empirical ML fields, and their absence weakens the reliability of the paper's specific numerical comparisons, particularly for systems separated by small margins. The paper does not suggest improvements to evaluation methodology as future work.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper introduces a systems-design reframing rather than a new algorithmic paradigm: it argues that the environment layer in agentic training should be a thin, standalone service with a stable, minimal API—decoupled from agent harnesses, training loops, and task domains—and that drawing this boundary correctly is what enables trajectory data, training recipes, and evaluation protocols to transfer across contexts that vertically integrated stacks lock into place. The reframing is not "we built a better sandbox" but rather "where the service boundary falls determines what the research community can share."

The magnitude of this shift is architectural rather than algorithmic. It does not introduce a new learning method—credit-assignment SFT, BAR, and judge-grounded GUI RL are all novel within their domains, but the paper's most generalizable contribution is the diagnosis that the field's fragmentation (non-reproducible results, harness-locked models, incomparable benchmarks) arises from a structural coupling between environment management and everything above it. Prior work either embedded the environment inside training orchestration (ProRL Agent, MegaFlow) or outsourced it to managed vendors (E2B, Daytona), both of which create dependencies that propagate upward. Orchard demonstrates that decoupling the environment as a standalone service with agent injection, direct Pod-IP communication, and a harness-agnostic REST API breaks these dependencies, and that this decoupling is not merely a convenience but a precondition for controlled cross-harness experimentation (Table 8), cost-accessible research (Table 2: 10× cheaper than managed alternatives), and recipe reuse across domains (SWE → GUI → claw agent).

The paper reconciles a latent contradiction in the SWE-agent literature: how can two systems report similar SWE-bench Verified numbers yet behave completely differently when the harness changes? Scale-SWE achieves 64.0% and OpenSWE-32B achieves 62.4%—numbers that appear competitive with Orchard-SWE's 64.3% SFT-only result. But when evaluated across harnesses, Scale-SWE produces invalid outputs under any non-native harness (zero measurable resolve rate), OpenSWE-32B drops 58.8 points on an unseen harness (3.6%), while Orchard-SWE retains 45.0–64.3% across all three tested harnesses (Table 8). The resolution is that single-benchmark scores under a single harness measure harness-specific formatting proficiency as much as domain-general agentic capability, and that this conflation has been invisible to the field because no prior infrastructure supported systematic cross-harness evaluation. Orchard makes this diagnostic operational: if your model's resolve rate collapses under a different harness, it learned the harness, not the task.

Several research directions become more attractive as a consequence of this reframing:

Infrastructure-aware evaluation. Standardized cross-harness evaluation matrices (like Table 8) should become a community norm for claimed agentic capability, not a novel contribution. Orchard Env makes this practical by providing a harness-agnostic execution backend that any harness can compose with at zero adaptation cost.
Multi-harness training as a robustness strategy. The controlled experiment in Table 10 (single-harness SFT producing 30–40 point diagonal–off-diagonal gaps) implies that harness diversity during training functions as a form of domain randomization, forcing the model to learn task structure rather than output formatting. This is directly analogous to data augmentation in computer vision, and suggests that deliberate harness randomization during agentic training (varying tool schemas, observation formats, and turn conventions across training instances) could produce more robust agents without additional task data.
Cheap, self-hosted infrastructure as an enabler for academic research. The 10× cost reduction from managed services to Orchard Env on spot instances ( $7,078 →$ 673 for a representative RL workload; Table 2) changes who can participate in agentic training research. When a single RL training run costs $7,000, only well-funded labs can afford systematic ablations or hyperparameter sweeps. At$ 700, individual academic groups can iterate. This is not a scientific contribution but a practical accessibility contribution that could broaden the set of institutions producing state-of-the-art agentic results.

Conversely, some research directions become less attractive after this work:

Single-harness, single-benchmark agentic training. The catastrophic harness-lockin demonstrated by Scale-SWE and OpenSWE-32B (Table 8) implies that training on one harness and evaluating only on that harness produces results that are not portable and may not reflect genuine task capability. Papers that report only single-harness SWE-bench Verified scores without cross-harness evaluation should be viewed with appropriate skepticism.
Managed sandbox platforms as the default for agentic RL. The cost comparison (Table 2) and latency comparison (Table 3) together make a strong case that self-hosted Kubernetes sandboxes are both cheaper and faster than commercial alternatives for large-scale RL training. Managed platforms may remain preferable for quick prototyping or low-volume workloads, but for the 128-sandbox, 240-hour RL runs that agentic training increasingly requires, the economics strongly favor self-hosting.

Follow-Up Research This Work Enables

Multi-domain joint training using the shared Orchard Env infrastructure. The paper presents Orchard-SWE, Orchard-GUI, and Orchard-Claw as independent instantiations, but the logical next step—training a single model on trajectories from all three domains using the same environment service—is not attempted. A strong follow-up would take a multimodal model (capable of both text-based SWE interactions and vision-based GUI navigation, perhaps a Qwen3-VL variant), train it on a mixture of SWE, GUI, and claw-agent trajectories through the same Orchard Env API, and evaluate whether cross-domain exposure produces positive transfer (does GUI training improve the model's ability to navigate repository file structures? does SWE training improve the model's systematic debugging of claw workflows?) or negative interference. The paper's infrastructure makes this experiment newly tractable because the same Orchard Env service, the same trajectory formats, and the same SFT+RL recipes can serve all three domains without per-domain adaptation. The key measurement would be whether the multi-domain model outperforms single-domain specialists on each benchmark, or whether domain interference dominates.

BAR ablation with progressive stride sizes to validate the adaptive mechanism. The paper's BAR implementation uses stride s = 16 and maximum budget N_max = 16, meaning the algorithm's adaptive, progressive-generation behavior is never exercised (it generates all trajectories in one batch and performs post-hoc assembly). A direct ablation would compare three conditions at matched total environment-interaction cost: (a) standard GRPO with N = 8 fixed group size and zero-variance group discarding, (b) BAR with s = 16, N_max = 16 (the paper's configuration, effectively post-hoc assembly), and (c) BAR with s = 4, N_max = 20 (truly progressive: easy prompts stop after one stride, hard prompts use 5 strides). The comparison would measure final SWE-bench Verified resolve rate, RL training wall-clock time, and total sandbox-hours consumed. If condition (c) matches or exceeds (a) and (b) while reducing total environment interactions by generating fewer trajectories for easy prompts, BAR's adaptive mechanism is validated. If all three conditions perform similarly, BAR's value is in post-hoc group assembly rather than progressive generation. If (a) matches (c), BAR is unnecessary—standard GRPO with filtering suffices.

Cross-model-family recipe transfer to test generalizability of the training procedures. All three Orchard recipes are demonstrated on Qwen3 backbones. A critical stress-test would replicate the Orchard-SWE recipe exactly—same 107K trajectories, same credit-assignment SFT, same BAR-augmented RL—on a non-Qwen base model (e.g., Llama-3-70B, DeepSeek-V3, or Gemma-3-27B) and measure the absolute lift over the base model's SWE-bench Verified performance. If the lift is comparable to the 45.5-point gain observed for Qwen3-30B-A3B (22.0% → 67.5%), the recipe is genuinely general. If Llama-3 gains only 10–15 points from the same data, the recipe's effectiveness is architecture-dependent, and the paper's claims about method generalizability need to be qualified. A negative result would be equally valuable as a positive one, because it would identify whether the agentic training techniques (credit-assignment, multi-harness SFT, BAR) or the base model's inherent multi-turn tool-use aptitude is the dominant factor in Orchard-SWE's performance—a distinction the current single-family results cannot make.

Difficulty-adaptive compute allocation for agentic SFT+RL. The paper's SFT ablations (Table 9) show that data scale dominates selection strategy, but also that the SFT-only resolve rate plateaus around 54% even at 2,048 carefully selected trajectories—far below the full recipe's 67.5%. This implies that different instances benefit differently from SFT vs. RL investment, but the paper applies uniform training to all instances. A follow-up would estimate per-instance difficulty (via the SFT checkpoint's pass rate on 8 rollouts, as done for RL data selection) and then allocate training budget adaptively: easy instances get SFT only (RL is unnecessary), medium instances get SFT + RL, and hard instances (near-zero SFT pass rate) get additional teacher distillation before SFT or are flagged as requiring a larger base model. The hypothesis is that adaptive allocation would achieve the same or better overall resolve rate with lower total compute, by avoiding wasted RL on instances the model already solves and avoiding wasted SFT on instances that need RL exploration. Orchard Env's low latency and cost efficiency makes the per-instance pass-rate estimation (8 rollouts per instance) tractable at scale.

Credit-assignment SFT at scale: what happens when unresolved trajectories vastly outnumber resolved ones? The paper's credit-assignment SFT experiment (Section 3.6) uses a scale-matched comparison (32K resolved + 32K rise-segment vs. 32K resolved-only) and shows a modest +1.9-point gain. But the real promise of the method is for domains where resolved trajectories are scarce and unresolved ones are abundant. A follow-up would deliberately restrict resolved trajectories to a small number (e.g., 500, 1K, 2K) and then add increasing numbers of credit-assigned unresolved trajectories (10K, 50K, 100K) to measure whether the partial-progress signal can compensate for the scarcity of full-solution examples. If credit-assignment SFT allows a model trained on, say, 1K resolved + 100K unresolved trajectories to match or exceed a model trained on 50K resolved trajectories, the method has substantial practical value for data-scarce domains. The SWE-bench setting is ideal for this experiment because large pools of unresolved teacher trajectories are available (32.5K in the paper's own dataset), and the teacher model can serve as the retrospective value estimator with minimal additional cost.

Validating and improving LLM-as-judge reliability for GUI agent evaluation. Orchard-GUI's results depend entirely on GPT-4.1 as a success/failure judge, and the paper provides no inter-judge agreement, human validation, or judge-model comparison. A systematic evaluation study would collect human judgments of task success for a random sample of 200–300 GUI trajectories from WebVoyager, Online-Mind2Web, and DeepShop, then measure agreement between human judges and multiple LLM judges (GPT-4.1, GPT-5, Gemini-3, Claude Opus 4.5) using Cohen's kappa and per-benchmark bias. If different judges produce systematically different success rates (e.g., GPT-4.1 is 5–10 points more lenient than Gemini-3 on DeepShop), then cross-paper comparisons using different judges are not valid, and the community needs a standardized judge or judge-calibration protocol. If judge agreement with humans is low (<0.7 kappa) for certain task types (e.g., tasks requiring precise verification of extracted information), then LLM-as-judge is insufficient as a sole evaluation mechanism and should be supplemented with scripted verification where possible.

Practical Applications and Downstream Use Cases

Cost-efficient batch SWE-agent evaluation for open-source model development. An organization training and evaluating open-source SWE-agent models (e.g., a university lab or a startup competing on the SWE-bench leaderboard) can deploy Orchard Env on a Kubernetes cluster with spot instances and reduce their sandbox costs by 10× relative to using managed services like E2B or Daytona ( $673 vs.$ 7,078 for a 240-hour, 128-sandbox run; Table 2). For a typical development cycle involving 5–10 training runs with ablations, this translates to thousands of dollars saved per project. Beyond cost, the 0.28s command-execution latency (Table 3) is 2.7× faster than E2B and 7.3× faster than Modal, meaning that the same number of environment interactions completes in less wall-clock time, improving developer iteration speed. The agent-injection mechanism (Section 2.1) means that new SWE-bench images or custom repository images can be added without per-image modifications, reducing the engineering overhead of expanding to new task sources like SWE-rebench V2 or private repositories.

Multi-harness agentic training for deployable AI coding assistants. A company building an AI coding assistant that needs to work across multiple IDE integrations (e.g., a VS Code extension, a CLI tool, and a GitHub bot—each with different tool APIs and interaction patterns) can use Orchard Env's harness-agnostic infrastructure to train a single model on trajectories collected under all three harnesses simultaneously. The paper's cross-harness results (Table 8: Orchard-SWE retains 45.0–64.3% across three harnesses while single-harness-trained models collapse) provide direct evidence that multi-harness training produces harness-robust agents. The practical benefit is a single model that can be deployed across multiple products without per-product fine-tuning, reducing maintenance burden and ensuring consistent behavior. The proxy-LLM-server approach used for Orchard-Claw (Section 5.2) provides a template for recording trajectories from harnesses with complex internal logic (like IDE integrations that make multiple LLM calls per user action), making it feasible to collect training data from production-like environments rather than simplified sandboxes.

On-device or edge deployment of GUI agents via small models with environment-grounded RL. Orchard-GUI demonstrates that a 4B vision-language model, after SFT on only 412 trajectories and RL on 2.2K tasks, achieves a 68.4% average success rate across three live-website benchmarks—competitive with proprietary systems that are likely orders of magnitude larger (Table 13). This result has direct implications for deploying GUI agents on devices where large models are infeasible (laptops, phones, on-premise servers): a 4B model can run on consumer hardware, and the training recipe (teacher distillation → SFT → judge-grounded RL) requires only the Orchard Env infrastructure plus an LLM judge API, not access to massive proprietary models at inference time. A company building a browser automation product could train a domain-specific GUI agent on their target websites using this recipe, achieving competitive performance without the latency, cost, and privacy concerns of calling cloud-hosted proprietary models. The paper's finding that RL lifts WebVoyager performance from 60.2% (SFT-only, already strong) to 74.1% (SFT+RL, near the best proprietary systems at 69.3–78.2%) suggests that the RL stage is essential for closing the gap to proprietary quality—and Orchard Env's cost efficiency makes RL practical at this scale.