Computer Environments Elicit General Agentic Intelligence in LLMs

🎯 Pitch

LLMs given a bare virtual computer without training spontaneously install domain-specific tools, grep through 100K-token files, and write validation scripts—boosting math accuracy by over 24 points on AIME25—revealing that much of what we call 'reasoning' is really just tool-use waiting for an interface.

1. Executive Summary

This paper introduces LLM-in-Sandbox, a paradigm that grants LLMs access to a virtual computer—a code sandbox with terminal capabilities—enabling them to explore and solve non-code tasks through external resource access (e.g., autonomously installing domain-specific libraries like OPSIN for chemical structure conversion), file management (e.g., using grep and sed to process 100K-token documents), and code execution (e.g., writing Python scripts to iteratively refine sentences for instruction-following constraints). Across mathematics, physics, chemistry, biomedicine, long-context understanding, and instruction following, strong agentic models like Claude-Sonnet-4.5-Think and GPT-5 achieve substantial gains without additional training—up to +24.2% accuracy on AIME25 for Qwen3-Coder—while a lightweight reinforcement learning method, LLM-in-Sandbox-RL, trained on general non-agentic context-based data, teaches weaker models to exploit the sandbox effectively, closing their performance gap and even improving their vanilla LLM-mode outputs, establishing that sandbox interaction skills transfer across domains and inference modes only when models learn purposeful exploration rather than aimless wandering.

2. Context and Motivation

The Core Problem: LLMs Are Trapped in a Text-in-Text-out Paradigm

The fundamental gap this paper addresses is architectural rather than methodological: current LLM inference operates within a text-in-text-out straitjacket that constrains what models can actually accomplish, regardless of their underlying reasoning capabilities. When you ask an LLM to solve a math problem, it generates tokens that represent a solution—but it never actually computes anything. When you ask it to follow complex formatting constraints, it generates tokens that approximate compliance—but it never validates them against executable rules. When you give it a 100,000-token document, it processes that entire document in its limited context window—but it never searches through it efficiently the way a computer would.

The paper's central observation is that humans don't work this way. When a human faces a challenging analytical task, they don't sit still and think sequentially until an answer emerges. They reach for tools: they open a spreadsheet to compute, they search through documents with Ctrl+F, they install specialized software when they need domain-specific capabilities, they write scripts to automate repetitive validation. The computer—as a platform—is the most versatile problem-solving environment ever created, yet LLMs are typically denied access to it during inference.

This gap manifests concretely in three categories of tasks where text-only generation is fundamentally mismatched to the problem structure:

Computation-intensive tasks (mathematics, physics, chemistry): LLMs must perform multi-step calculations through autoregressive token generation, which is both inefficient and error-prone. The model has no mechanism to verify intermediate results through actual execution. A human solving AIME problems doesn't do algebra entirely in their head; they use scratch paper, calculators, and verification.
Knowledge-intensive tasks requiring external resources (biomedicine, chemistry): LLMs can only draw on knowledge embedded in their training parameters. If a chemistry task requires converting an IUPAC name to a molecular structure, the model must either have memorized that specific mapping or generate plausible-sounding but potentially incorrect output. A human would simply look it up or use a cheminformatics library.
Format-constrained tasks (instruction following, long-context extraction): LLMs must satisfy precise constraints—exact character counts, non-overlapping word sets, specific structured outputs—through pure text generation, with no ability to programmatically check their work before submitting. This leads to subtle failures that a simple script could catch.

The paper's insight is that these aren't separate problems requiring separate solutions. They're all symptoms of the same underlying limitation: LLMs lack access to a general-purpose computational environment during inference.

Why This Problem Matters: Beyond Benchmark Scores

The significance of closing this gap extends well beyond improving accuracy on a few benchmarks. The paper identifies several dimensions where text-in-text-out fundamentally limits what LLMs can achieve:

Verifiability as a prerequisite for reliability. When an LLM generates a math solution, you cannot distinguish a correct answer from a confident hallucination without external validation. But when the model executes Python code and shows the output, you get grounded computation—not linguistic plausibility. This is the difference between "the model says the answer is 42" and "the model computed the answer using an algorithm you can inspect." The paper's design choice to extract final answers from /testbed/answer.txt rather than from model-generated text is deliberate: it forces the model to produce verifiable, execution-backed outputs rather than persuasive prose.

Autonomy as a path to general intelligence. The paper frames sandbox access as enabling autonomous tool acquisition—the model doesn't call predefined APIs in a constrained tool-use setup; it discovers, installs, and learns to use arbitrary software libraries on demand. This is a qualitatively different capability. A model with predefined calculator access can perform arithmetic. A model with sandbox access can install SymPy, read its documentation, and solve symbolic integration problems it has never encountered before. The paper's case studies (Section 2.4.1) demonstrating that models spontaneously install Java runtimes and cheminformatics libraries to solve chemistry problems they weren't explicitly trained for exemplifies this.

Efficiency as an enabler of deployment. The practical deployment implications are substantial. As demonstrated in the paper's computational analysis (Section 4), long-context tasks that would consume 100K input tokens can be handled with 13K tokens when documents are stored as files and searched programmatically—an ~8× cost reduction. For organizations running LLMs at scale, this isn't a marginal improvement; it's the difference between feasible and infeasible deployment for document-intensive applications.

Opening new application categories. Section 5 makes explicit what was implicit throughout: many tasks are fundamentally impossible for text-only LLMs. You cannot generate an interactive map visualization, a professional poster in PNG format, an animated birthday video, or an original musical composition through text generation alone—you can only describe them. LLM-in-Sandbox enables models to actually produce these artifacts by orchestrating specialized software. This isn't an incremental improvement; it's a categorical expansion of what LLM-based systems can accomplish.

Where Existing Approaches Fall Short

The paper carefully positions itself against several existing paradigms, each of which addresses part of the problem but none of which provides a complete solution:

Chain-of-thought and reasoning elicitation methods (Wei et al., 2022; DeepSeek-R1; etc.) improve the quality of generated reasoning traces but remain within the text-in-text-out paradigm. A model can reason step-by-step about a computation, but it cannot execute that computation. This creates a ceiling: reasoning about code behavior from first principles will always be less reliable than running the code and observing the output. The paper's experiments demonstrate this ceiling concretely—strong LLMs already reason well, yet they still gain 5–15 percentage points on math and physics benchmarks when given sandbox access (Table 2), because execution provides something reasoning alone cannot.

LLM-as-agent frameworks with predefined tools provide models with curated sets of capabilities (calculators, search engines, specific APIs). This approach has two critical limitations that sandbox access eliminates. First, the toolset is bounded by what developers anticipated—the model cannot acquire new capabilities at runtime. Second, the tools are typically thin wrappers with limited composability. The paper's approach of providing a complete computer environment means the model can chain arbitrary operations: install a library, read its documentation, write a script that uses it, debug errors from execution feedback, and iteratively refine—all without any tool being explicitly predefined. This distinction between "tool use" and "computer use" is central to the paper's argument.

Code agents in software engineering contexts (SWE-Agent, OpenHands, R2E-Gym, Claude Code) do provide sandbox environments, but they are designed for and evaluated on software engineering tasks. These systems use task-specific sandbox configurations—pre-installed dependencies, pre-configured repositories, per-task Docker images that can require terabyte-scale storage for benchmark-scale evaluation (Table 1 documents SWE-Gym requiring 6 TB for task-specific images). The paper argues this design is fundamentally unscalable to general-domain tasks because it requires knowing in advance what tools each task will need. Their lightweight 1.1 GB shared image that delegates all domain-specific configuration to the model at runtime is a deliberate architectural departure.

Reinforcement learning for LLMs without sandboxes (LLM-RL, GRPO-based training) can improve model capabilities using general-domain data, but the model never learns to interact with an environment. The paper's experiments in Table 6 demonstrate the consequence: LLM-RL improves vanilla LLM mode performance but provides minimal gains—and sometimes regressions—in sandbox mode, because the model hasn't learned to exploit environmental feedback. The trained model is better at text generation but no better at computer interaction.

Software engineering RL with sandboxes (SWE-RL, DeepSWE) does train models in sandbox environments, but using domain-specific software engineering data. The paper's key methodological insight is that sandbox interaction skills—exploring files, reading execution feedback, making purposeful tool calls—are transferable across domains, and can be learned from general context-based tasks that are much easier to curate at scale than software engineering data. Table 5 formalizes this comparison: only LLM-in-Sandbox-RL combines sandbox utilization, general-domain data, and scalability across both data and environment dimensions.

Agentic benchmarks evaluate specific tool-use capabilities but don't measure the general ability to leverage computational environments. The paper proposes that the metric $\Delta = \text{LLM-in-Sandbox} - \text{LLM}$ itself serves as an agentic capability indicator—quantifying how effectively a model can augment its reasoning with environmental interaction. This is a meta-contribution: providing a unified evaluation framework rather than yet another task-specific benchmark.

How This Paper Positions Itself

The paper does not propose a fundamentally new model architecture, a new training objective, or even a new class of algorithms. Instead, it proposes a paradigm shift in how LLMs are deployed during inference—moving from isolated text generation to environmentally-situated problem-solving—and provides systematic evidence that this shift unlocks capabilities that were latent in existing models.

The intellectual contribution operates at three levels:

At the deployment level, the paper makes the case that sandbox access should become the default inference infrastructure for LLMs, analogous to how operating systems provide standard capabilities (file systems, networking, process management) that all applications benefit from. The computational analysis in Section 4 makes this case practical: the overhead is manageable, the cost savings on long-context tasks are substantial, and the infrastructure is lightweight enough for large-scale deployment.

At the training level, LLM-in-Sandbox-RL demonstrates that sandbox interaction can be trained as a general skill using non-agentic data, and that this skill transfers across domains and inference modes. This is the paper's most surprising finding—that training a model to explore files in a sandbox for context-based QA transfers to improved math reasoning, chemistry problem-solving, and even text-only generation. The mechanism (analyzed in Section 3.3) is that sandbox interaction teaches structured reasoning patterns and verification behaviors that persist even when the sandbox is removed.

At the conceptual level, the paper reframes the relationship between language models and computation. Rather than viewing computation as something models simulate through reasoning traces, it proposes that models should leverage computation as an external resource—just as humans do. This reframing has implications for how we think about scaling: test-time compute has typically been studied as increased autoregressive sampling (best-of-N, beam search), but the paper argues that test-time compute should include environmental interaction—program execution, file operations, tool use—which can be far more efficient per FLOP than generating more text.

The paper's positioning relative to the broader AI trajectory is explicit in its opening paragraph: in-context learning showed models could generalize without finetuning; chain-of-thought elicited reasoning through decomposition; agentic frameworks enabled tool use across turns. LLM-in-Sandbox is proposed as the next step along this trajectory—not just giving models tools, but giving them the platform from which all tools emerge. This is an ambitious claim, and the paper supports it not through a single killer result but through convergent evidence across six domains, seven models, and two training paradigms.

3. Technical Approach

This is primarily a systems and training paradigm paper whose core idea is to treat a code sandbox as a universal inference-time resource for LLMs, enabling models to solve non-code tasks through programmatic exploration rather than pure text generation, and to train sandbox interaction as a transferable skill using general-domain reinforcement learning.

3.1 Reader Orientation

The paper builds two integrated systems: (1) a deployment framework that wraps any LLM with a lightweight virtual computer, enabling it to autonomously write and execute code, manage files, and access external resources during inference, and (2) a reinforcement learning pipeline (LLM-in-Sandbox-RL) that trains models to use this environment effectively using only general context-based tasks. The central problem is that text-only LLMs cannot compute, validate, or search—they can only simulate these operations through token generation—and the solution is to provide models with the same computational environment humans use, then train them to exploit it purposefully rather than wander aimlessly.

3.2 Big-Picture Architecture (Diagram in Words)

The system has five major components:

Code Sandbox — a Docker container running Ubuntu with a standard Python interpreter, essential scientific libraries (NumPy, SciPy), three core tools (execute_bash, str_replace_editor, submit), and internet access. It is a general-purpose, shared environment (~1.1 GB image) with no task-specific pre-configuration.
LLM-in-Sandbox Runtime — a ReAct-based multi-turn loop that iteratively receives observations from the sandbox (tool execution results), feeds them back to the LLM as context, and dispatches the LLM's next action to the sandbox, continuing until submit is called or a turn limit is reached.
Task Input/Output Handler — uses the sandbox file system to flexibly handle inputs (documents placed in /testbed/documents/, requirements in /testbed/) and extract final outputs from a designated location (/testbed/answer.txt), separating exploration from answer delivery.
LLM-in-Sandbox-RL Training Pipeline — a GRPO++ training loop that places training data contexts as files within sandboxes, runs multi-turn trajectories, computes outcome-based rewards from the final extracted answer, and updates the model policy based on trajectory-level returns.
Deployment Infrastructure — integrates with vLLM and SGLang serving engines, manages concurrent sandbox containers (~50 MB idle, ~200 MB peak per container), and handles sandbox lifecycle (creation, execution, cleanup).

Information flows as follows during inference: a task prompt enters → the sandbox is initialized with any task files placed at designated paths → the LLM generates a tool call (bash command, file edit, or submit) → the sandbox executes the call and returns the output → the observation is appended to the interaction history → the LLM generates the next tool call → this repeats until submit → the final answer is extracted from /testbed/answer.txt and returned.

During training, the flow is identical except that task contexts are pre-placed as files, distractors are added for single-file contexts, and after the trajectory completes, a reward is computed by comparing the extracted answer to the ground truth, with the full trajectory rewarded via GRPO++.

3.3 Roadmap for the Deep Dive

First, the sandbox environment, because it is the physical substrate everything else builds on — its design principles (lightweight, general-purpose, minimal toolset), its three core tools and their specifications, and how it differs from software engineering sandboxes in configurability and scalability.
Second, the LLM-in-Sandbox workflow, because it defines the interaction protocol that both inference and training share — the ReAct loop, the system prompt that encourages exploration, and the file-based input/output mechanism that cleanly separates process from product.
Third, the training-free evaluation framework, because the paper first establishes what models can do without additional training — the domain coverage, model selection, benchmark handling, and the $\Delta = \text{LLM-in-Sandbox} - \text{LLM}$ metric that quantifies sandbox benefit.
Fourth, the sandbox capability classification system, because the paper's behavioral analysis depends on operationalizing "resource access," "file management," and "computation" as pattern-detectable categories.
Fifth, LLM-in-Sandbox-RL, because this is the paper's novel training contribution — the data sourcing and sandbox configuration strategies, the GRPO++ training loop, the reward design across task types, and the key insight that sandbox interaction is a transferable meta-skill.
Sixth, the deployment infrastructure and computational analysis, because practical adoption requires understanding the cost/speed/storage tradeoffs — token consumption patterns, environment token prefill advantages, and sandbox container overhead.

3.4 Detailed, Sentence-Based Technical Breakdown

Code Sandbox: A Lightweight General-Purpose Virtual Computer

The sandbox is a Docker container running Ubuntu with terminal access and full system capabilities. Unlike sandboxes in software engineering systems (SWE-Agent, OpenHands, R2E-Gym) that require task-specific pre-configuration — pre-installed package dependencies, pre-cloned repositories, and per-task Docker images — LLM-in-Sandbox provides a single shared image of approximately 1.1 GB equipped only with a standard Python interpreter and essential scientific computing libraries such as NumPy and SciPy. All domain-specific tool acquisition is delegated to the model at runtime: the model decides what packages to install, what tools to download, and how to configure the environment for the current task.

This design is motivated by two requirements. First, generalizability: a single environment must support tasks spanning mathematics, physics, chemistry, biomedicine, long-context understanding, and instruction following without manual reconfiguration, which is impossible if each domain requires pre-installed tools. Second, scalability at inference and training time: software engineering benchmarks like SWE-Gym require up to 6 TB of storage for task-specific images (Pan et al., 2024), whereas the shared-image approach maintains constant 1.1 GB regardless of the number of tasks, enabling efficient large-scale evaluation and RL training where thousands of sandbox instances may run concurrently.

The contrast is formalized in Table 1:

Aspect	SWE Agents	LLM-in-Sandbox
Environment Setup	Task-specific	General-purpose
Dependencies	Pre-configured	Runtime installation
Storage Scaling	Per-task images	Single shared image

Within the sandbox, the model is equipped with exactly three tools that together realize the core meta-capabilities of a general-purpose computer:

execute_bash — executes arbitrary terminal commands. This is the most fundamental tool because virtually any computer operation can be expressed as a shell command: installing packages (pip install, apt-get install), managing files (cp, mv, rm), running programs (python script.py), downloading resources (curl, wget), searching text (grep, sed, awk), and chaining operations through pipes and redirects. The paper deliberately provides bash access rather than higher-level abstractions because bash is the universal interface that composes all other capabilities.
str_replace_editor — provides file creation, viewing, and editing capabilities. This is necessary because while bash can manipulate files through commands like cat and sed, a structured editor enables the model to view file contents with line numbers, make targeted replacements, and create new files with precise content control. The tool specification (detailed in Appendix A) includes operations for viewing files with line number prefixes, creating new files with specified content, and performing string replacements identified by old string and new string.
submit — signals task completion. When the model calls submit, the multi-turn loop terminates and the system extracts the final output from the designated location (/testbed/answer.txt). This tool serves as an explicit termination signal, distinguishing task completion from mid-exploration tool calls.

The toolset is deliberately minimal. The paper explicitly avoids providing domain-specific tools (calculators, chemistry libraries, equation solvers) as predefined capabilities because doing so would constrain the model to whatever tools developers anticipated. Instead, the model is expected to use execute_bash to install whatever it needs — a design choice that the paper's case studies (Section 2.4.1) validate by showing models spontaneously installing Java runtimes, cheminformatics libraries, and JavaScript visualization frameworks.

LLM-in-Sandbox Workflow: Multi-Turn Exploration with File-Based I/O

The inference workflow builds on the ReAct framework (Yao et al., 2022), where the model iteratively reasons about its observations and acts by calling tools, receiving environmental feedback that informs its next reasoning step. The paper's Algorithm 1 formalizes this as a loop with maximum turns $T$ :

Step-by-step execution:

The sandbox $\mathcal{S}$ is configured with task requirements $r$ (if any), which typically means placing input files at designated paths — for long-context tasks, documents go in /testbed/documents/; for other tasks, any necessary configuration files are placed in /testbed/.
The turn counter $t$ is initialized to 0, and the available toolset is {execute_bash, str_replace_editor, submit}.
At each turn while $t < T$ :
- The model generates a tool call $a_t$ based on the original task prompt $p$ and the accumulated interaction history (all previous tool calls and their observations).
- If $a_t$ is a submit call, the loop terminates immediately.
- Otherwise, $a_t$ is executed in the sandbox $\mathcal{S}$ , producing an observation $obs_t$ — the standard output, standard error, and return code of the executed command.
- The pair $(a_t, obs_t)$ is appended to the interaction history.
- $t$ is incremented.
After the loop terminates (either via submit or reaching $T$ ), the final output $o$ is extracted from a designated location in the sandbox — typically /testbed/answer.txt, as specified in the system prompt.

The maximum turn limit $T$ is set to 100 turns across experiments. The maximum generation length per turn is 65,536 tokens for all models except Claude-Sonnet-4.5-Think (limited to 64,000 tokens due to API constraints). For long-context understanding tasks specifically, the total trajectory length cap (including prompt, model output, and environment output) is raised to 131,072 tokens to accommodate the extended context.

System prompt design (Appendix F, Figures 6–7). The system prompt is crafted to encourage exploratory behavior while establishing clear interaction protocols. It contains three key principles:

Leverage computational tools: the prompt explicitly tells the model to use programming rather than performing calculations in natural language — "You should leverage computational tools like Python rather than performing calculations or analysis through natural language."
Derive answers through execution: the model is instructed to compute results programmatically rather than hardcoding answers — "You should derive answers through program execution; do NOT directly hardcode the final answer as text."
Safe exploration environment: the prompt informs the model that the sandbox is isolated and safe, encouraging diverse solution strategies — "The sandbox is a safe, isolated environment where you can freely explore different approaches."

The instance prompt template (Figure 6) provides the task-specific problem statement and specifies the output location: "Put your final answer in /testbed/answer.txt. The file MUST contain ONLY the final answer."

Flexible input/output handling. The paper leverages the sandbox file system to handle diverse input and output formats across domains. For inputs, content can be provided either in the model prompt (standard for math, physics, chemistry problems) or as files within the sandbox (for long-context documents). This bifurcation is essential for long-context tasks because including 100K-token documents in the prompt would consume massive context window capacity and incur high per-query token costs. By placing documents as files, the model can use programmatic search (grep, sed, Python scripts) to locate relevant information without loading the entire corpus into its context window.

For outputs, the model is instructed to place the final result at /testbed/answer.txt, containing only the final answer without intermediate reasoning or exploration content. After task completion, the system extracts the content of this file as the final output. This approach cleanly separates the exploration process (which may involve many multi-turn interactions, file reads/writes, and debug outputs) from the final deliverable (which is what gets evaluated). It also naturally accommodates various output formats: a numeric answer for math, a multiple-choice letter for biomedicine, a structured response for instruction following.

Training-Free Evaluation: Measuring Sandbox Benefit

Models evaluated. The paper tests seven models spanning frontier proprietary, open-weight, code-specialized, and smaller general-purpose categories: Claude-Sonnet-4.5-Thinking, GPT-5, DeepSeek-V3.2-Thinking, MiniMax-M2, Kimi-K2-Thinking, Qwen3-Coder-30B-A3B-Instruct, and Qwen3-4B-Instruct-2507. Inference configurations are summarized in Appendix B, Table 13, with sampling parameters set to each model supplier's recommendations (e.g., temperature 1.0 for most frontier models, 0.7 for Qwen models with top_p 0.8, min_p 0.0, top_k 20, and repetition penalty 1.05 for Qwen3-Coder).

Six evaluation domains (Appendix C, Table 14):

Mathematics: All 30 problems from AIME 2025 (olympiad-level), repeated 16 times each to reduce variance (480 total evaluations), graded with Math-Verify using \boxed{} extraction.
Physics: 650 problems from UGPhysics spanning 13 undergraduate subjects (50 per subject), evaluated via LLM judge using Qwen3-30B-A3B-Instruct-2507.
Chemistry: 450 single-choice questions from ChemBench across nine core tasks (50 per sub-domain), exact match evaluation.
Biomedicine: 500 text-based multiple-choice questions from MedXpertQA, exact match evaluation.
Long-Context Understanding: 100 challenging multi-document reasoning questions from AA-LCR (documents averaging ~100K tokens), repeated 4 times each, evaluated with LLM equality checker using Qwen3-235B-A22B-Instruct-2507. In LLM-in-Sandbox mode, all related documents are stored as text files in /testbed/documents/, each named after its original title.
Instruction Following: 300 single-turn questions from IFBench with 58 diverse verifiable constraints, evaluated with official loose-mode code.
Software Engineering (RL experiments only): 500 problems from SWE-bench Verified, evaluated with official rule-based script, using R2E-Gym sandbox setup.

The $\Delta$ metric. The paper's central evaluation construct is the performance difference:

$\Delta = \text{LLM-in-Sandbox} - \text{LLM}$

where LLM is vanilla text generation (direct output without sandbox) and LLM-in-Sandbox is the same model with sandbox access. This $\Delta$ captures the net benefit of environmental interaction — a positive $\Delta$ means the model effectively leverages the sandbox, while a negative $\Delta$ means sandbox access degrades performance (typically because the model wastes turns on ineffective exploration or gets distracted by tool execution failures).

Benchmark integrity measures. Because models have internet access in the sandbox, the paper takes precautions to prevent benchmark hacking: "we reframe test problems to prevent benchmark hacking and manually verify sampled trajectories to ensure valid reasoning." The specific reframing method is not detailed in the main text, but the intent is clear — prevent models from simply looking up answers online rather than solving problems through computation and reasoning.

Sandbox Capability Classification: Operationalizing Behavioral Analysis

To quantitatively analyze how models use the sandbox, the paper defines a pattern-matching system (Appendix D, Table 15) that classifies each model action into three capability categories:

External Resources — operations that acquire knowledge or tools from outside the sandbox:

Package installation: pip install, apt-get install
HTTP requests: requests.get, curl, wget
Web scraping: BeautifulSoup, selenium
Domain-specific library loading: rdkit, biopython, pubchempy

File Management — operations that read, write, search, or organize persistent data:

Python file I/O: open(), json.load, pd.read_csv
Shell file commands: cat, grep, find, head/tail
Path operations: os.path, pathlib, glob
Data serialization: pickle.load, np.load/np.save

Computation — operations that perform numerical or algorithmic work:

Numerical solvers: scipy.optimize, fsolve, minimize
Integration: odeint, solve_ivp, quad
Iterative algorithms: large loops where range(N) with N > 100, while loops
Combinatorics: itertools.permutations/combinations

For each trajectory, the system extracts all code blocks from model actions (both Python scripts and bash commands) and applies these patterns. The capability usage rate is computed as:

$\text{Usage Rate} = \frac{\text{Turns containing at least one matched pattern}}{\text{Total interaction turns}}$

This gives a per-category frequency measure (e.g., "43.4% of turns involved computation in mathematics tasks") that enables cross-domain and cross-model comparison of sandbox utilization strategies.

LLM-in-Sandbox-RL: Training Sandbox Interaction as a Transferable Skill

This is the paper's core training contribution. The method trains models to effectively explore sandbox environments using only general context-based task data, without any specialized agentic or domain-specific training examples. The key insight is that sandbox interaction is a meta-skill — the ability to explore files, read execution feedback, make purposeful tool calls, and recover from errors transfers across domains — and can be learned from tasks far simpler than software engineering.

Formal positioning against alternatives (Table 5):

	LLM-RL	SWE-RL	LLM-in-Sandbox-RL
Sandbox Utilization	✗	✓	✓
General Domain Data	✓	✗	✓
Data Scalability	✓	✗	✓
Environment Scalability	N/A	✗	✓

LLM-RL (standard text-only RL training, e.g., Tülu 3) can use general-domain data but the model never interacts with an environment. SWE-RL (e.g., DeepSWE) trains models in sandboxes but requires domain-specific software engineering data that is expensive to curate. LLM-in-Sandbox-RL combines the benefits: sandbox-based training with easily-scalable general-purpose data in lightweight, reusable environments.

Data source. Training data comes from context-based task datasets — specifically, the seed data used for fine-tuning the synthesizer in Instruction Pre-Training (Cheng et al., 2024). This data covers diverse domains: encyclopedia articles, fiction passages, expert materials, academic tests, news articles, social media content, and trivia. Each data instance consists of background material (context) paired with a series of related tasks. Task types include free-form generation, multiple choice, and reasoning. The crucial property is that the content is general-domain (not software engineering), but the format requires the model to reference external material to answer questions.

Sandbox configuration for training (Figure 3). The training setup is designed to force models to actively explore the sandbox file system rather than relying on context provided in the prompt. The paper employs two strategies:

Multi-document or long contexts: If a task's background material consists of multiple documents or a single very long document, the content is split into separate files. For example, a research paper is divided into sections stored as introduction.txt, methods.txt, results.txt, etc. This forces the model to navigate multiple files, decide which are relevant, and synthesize information across file boundaries.
Single-file contexts with distractors: If the context would result in only one file, additional files are sampled from the same dataset and added as distractors. This prevents the model from simply reading the only available file; it must filter relevant from irrelevant information, mimicking real-world scenarios where relevant documents are embedded in larger file systems.

All context files (both relevant and distractor) are placed in /testbed/documents/. The model is informed via the prompt that relevant files are in this directory, but it must discover which files are relevant through exploration.

Task setup and in-context examples (Figure 4). Each data instance may have multiple related tasks that depend on each other in a fixed order — for example, a set of questions about a passage where earlier questions establish context for later ones. The paper samples one task as the testing task and uses prior tasks in the sequence as in-context examples in the prompt. This provides the model with demonstrations of the expected interaction pattern without requiring separate agentic training data. The prompt also explicitly instructs the model to write the final answer to /testbed/answer.txt.

RL training procedure. The training follows the GRPO++ algorithm, a variant of Group Relative Policy Optimization (GRPO; Shao et al., 2024) adapted for agentic training in the DeepSWE framework (Luo et al., 2025), implemented within the rLLM training infrastructure (Tan et al., 2025). The key hyperparameters (Table 16) are:

RL Algorithm: GRPO++
Learning Rate: $1 \times 10^{-6}$
Train (Prompt) Batch Size: 8
Update mini batch size: 8 (one update per batch — fully on-policy)
Rollouts per Prompt: 8
Train Steps: 150 for Qwen3-4B-Instruct-2507, 50 for Qwen3-Coder-30B-A3B
Max Turns: 100
KL Reward/Loss: None (no KL penalty to the reference policy)
Rollout Temperature: 1.0
Rollout Top_p: 0.8
Rollout Top_k: 20
Max Response Length: 65,536 Tokens

The training loop operates as follows: for each prompt in a batch, the model generates 8 complete trajectories in LLM-in-Sandbox mode (multi-turn sandbox interaction). Each trajectory is scored with an outcome-based reward computed from the final answer extracted from /testbed/answer.txt. The GRPO++ algorithm then updates the policy based on the relative advantage of trajectories within each prompt's rollout group. Notably, no KL penalty is applied — the paper does not constrain the policy to remain close to the reference model, relying instead on the limited number of training steps to prevent catastrophic drift.

Reward design. Rewards are tailored to task type using rule-based functions:

Multiple-choice tasks: positive reward for selecting the correct option, 0 otherwise. When multiple correct options exist, F1 score is used as the reward (rewarding partial correctness).
Free-form generation tasks: ROUGE-L score (Lin, 2004) between the generated answer and the ground truth, measuring n-gram overlap as a continuous reward signal.
Binary correctness tasks (e.g., math): +1 for correct, 0 for incorrect.

A length penalty is also applied: "if the model exceeds maximum turns/tokens without submitting an answer, the episode is terminated with zero reward." This discourages unproductive wandering and incentivizes efficient exploration.

Training on two capability levels. The paper trains two base models that exhibited different behaviors in the training-free evaluation (Section 2.3): Qwen3-4B-Instruct-2507 (a small general-purpose model that performed worse in sandbox mode than LLM mode — negative $\Delta$ values) and Qwen3-Coder-30B-A3B (a code-specialized model that already showed positive $\Delta$ values). The training data is identical for both, testing whether LLM-in-Sandbox-RL can (1) teach weak models to exploit the sandbox effectively, and (2) further enhance models that already possess sandbox skills.

Data ablation (Table 7). To isolate the effect of data choice, the paper compares four training variants on Qwen3-4B-Instruct-2507:

Math: mathematical reasoning data from DAPO (Yu et al., 2025) — domain-specific data, testing whether math training transfers to non-math sandbox tasks.
SWE: software engineering data from R2E-Gym (Jain et al., 2025) — domain-specific agentic data, testing whether SWE training transfers to general domains.
Gen. in Prompt: the general context-based data with context placed in the prompt rather than in sandbox files — an ablation testing whether the sandbox interaction during training is necessary, or whether simply training on context-based data is sufficient.
Gen. in Sandbox: the full LLM-in-Sandbox-RL setup with general context-based data and context placed as files in the sandbox.

The comparison between Gen. in Prompt and Gen. in Sandbox is the critical ablation: both use identical data, but only Gen. in Sandbox forces the model to interact with the environment to access that data. Any performance difference between these two conditions isolates the effect of learning sandbox exploration skills during training.

Deployment Infrastructure and Computational Analysis

The paper analyzes practical deployment considerations using local model serving across different LLMs and serving engines: DeepSeek-V3.2-Thinking and Kimi-K2-Thinking served via SGLang, MiniMax-M2 and Qwen3-Coder-30B-A3B served via vLLM. All experiments run on a single NVIDIA DGX node with query concurrency set to 64.

Token consumption (Table 10). Total tokens per query are measured as the sum of prompt tokens, model-generated tokens, and environment-generated tokens (tool outputs). The key finding is task-dependent:

For most tasks (math, physics, chemistry, biomedicine, instruction following), LLM-in-Sandbox consumes more tokens due to multi-turn exploration — increases range from +1.9K to +18.4K tokens depending on model and task.
For long-context tasks, LLM-in-Sandbox dramatically reduces tokens by storing content in files rather than in the prompt. The reductions are striking: DeepSeek goes from 90.3K to 25.4K ( $-64.9$ K, a $-72\%$ reduction), Qwen goes from 102.9K to 12.9K ( $-90.0$ K, an $-87\%$ reduction, approximately 8× savings).
Aggregated across all tasks, LLM-in-Sandbox consumes only 0.49× to 0.84× the total tokens of LLM mode (depending on model). The long-context savings partially offset the multi-turn overhead on other tasks.

Inference speed (Table 11). A critical efficiency insight: environment-generated tokens are processed via the fast Prefill path (FlashAttention), not the slow autoregressive decode path. Across models, 37%–51% of all trajectory tokens come from the environment, and environment execution accounts for less than 4% of total time (2.2%–3.5%). End-to-end throughput is measured in QPM (Queries Per Minute):

$\text{QPM Ratio} = \frac{\text{QPM}_{\text{LLM-in-Sandbox}}}{\text{QPM}_{\text{LLM}}}$

Results range from 0.6× (DeepSeek, slightly slower due to multi-turn overhead) to 2.2× (MiniMax, faster because environment prefill is faster than generating equivalent reasoning tokens). Overall, LLM-in-Sandbox achieves competitive or better throughput.

Storage overhead (Table 12, left). The single shared Docker image is ~1.1 GB, compared to task-specific image storage requirements of 6 TB (SWE-Gym), 295 GB (SWE-Smith), or 257 GB (SWE-bench Verified). This is a 3–4 order of magnitude reduction in storage requirements.

Memory overhead (Table 12, right). Each sandbox container consumes approximately 50 MB at idle and 200 MB at peak. On a DGX node with 2 TB system RAM:

64 concurrent sandboxes: 13 GB, approximately 0.7% of system RAM
512 concurrent sandboxes: 100 GB, approximately 5% of system RAM

Even at high concurrency (512 sandboxes), the memory overhead is modest relative to the memory consumed by the LLM itself, making large-scale deployment feasible on standard hardware.

Summary of Design Choices and Their Justifications

Single shared sandbox image over task-specific images: enables generalizability across arbitrary tasks without per-task configuration, and reduces storage from terabyte-scale to 1.1 GB, making large-scale deployment and RL training feasible.
Three-tool toolset (execute_bash, str_replace_editor, submit) over predefined domain tools: bash is the universal computer interface that composes all other capabilities; predefined tools would constrain models to whatever developers anticipated rather than enabling autonomous tool acquisition.
File-based output extraction (/testbed/answer.txt) over parsing model text: cleanly separates exploration process from final deliverable, enables evaluation of arbitrary output formats, and forces models to produce verifiable execution-backed outputs.
Context-based training data with distractor files: forces models to learn active file exploration and relevance filtering — the meta-skill that transfers across domains — rather than passive reading of provided context.
GRPO++ with no KL penalty: the limited number of training steps (50–150) naturally prevents catastrophic drift without needing to constrain the policy's divergence from the reference model, simplifying the training procedure.
Outcome-based rewards only: avoids the need for process supervision (step-level rewards), which would require annotating correct intermediate actions and would be domain-specific, undermining the goal of general-domain training.
Comparison of Gen. in Prompt vs. Gen. in Sandbox for training: isolates the effect of learning sandbox interaction from the effect of learning from context-based data, establishing that how the model accesses information during training determines whether it learns transferable exploration skills.

4. Key Insights and Innovations

Innovation 1: The Sandbox as Inference Substrate, Not a Tool Wrapper

The paper's deepest conceptual move is not adding a new capability to LLMs but changing the category of what an LLM is during inference. Before this work, the dominant paradigm treated LLMs as text processing units that optionally call tools through predefined APIs. Tool use is an extension of the text generator: the model generates text, occasionally delegates to a calculator, and continues generating text. The environment is a servant that the model summons.

LLM-in-Sandbox flips this relationship. The model is no longer a text generator that occasionally calls tools; it is an agent situated in a computational environment, and text generation is merely the mechanism by which it issues commands to that environment. The sandbox is not a set of tools; it is a computer — a general-purpose platform where tools can be created, composed, and discarded at will, including tools that no developer anticipated. This is the difference between giving someone a calculator and giving them a terminal with pip install privileges.

This shift matters because it changes what "capability" means. In the tool-use paradigm, a model's capabilities are bounded by its predefined toolset. If you didn't give it a chemistry library, it cannot solve cheminformatics problems regardless of how smart it is. In the sandbox paradigm, the model's capabilities are bounded by what software exists on the internet. The model can autonomously identify its capability gaps and fill them — as the paper's chemistry case study demonstrates, where the model installs a Java runtime and downloads OPSIN to convert IUPAC names to molecular structures (Section 2.4.1), neither of which was pre-configured. The capability ceiling is not the toolset but the model's resourcefulness in discovering and deploying available software.

This reframing connects to a broader philosophical question about what general intelligence requires. The field has debated whether scaling model parameters and training data eventually yields general intelligence, or whether interaction with environments is essential. The paper doesn't resolve this debate, but it provides an architectural argument: if general intelligence involves solving arbitrary problems, then a system that can only draw on its training-time knowledge will be bounded by what it has seen, while a system that can acquire new tools at runtime approaches an open-ended capability profile. The 1.1 GB shared image is the architectural manifestation of this argument — deliberately not pre-loaded with domain tools precisely because pre-loading would constrain the model to whatever developers thought it might need.

The practical significance of this reframing emerges in the deployment analysis. If a sandbox is just a "tool wrapper," then long-context tasks would still require loading documents into the prompt — the tool wrapper doesn't change what the model processes. But if the sandbox is genuinely an inference substrate, then the model can fundamentally restructure how it solves problems. The ~8× token reduction on long-context tasks (100K → 13K for Qwen, Table 10) is not an optimization; it's a different problem-solving strategy enabled by a different relationship between model and environment.

Innovation 2: Sandbox Interaction as a Transferable Meta-Skill

The paper's second major contribution is the empirical discovery — and training-validation — that effective computer interaction is a generalizable skill that can be learned from non-agentic data and transfers across domains and inference modes. This finding contradicts a natural assumption: that learning to use a computer for one type of task (e.g., reading files to answer questions) should only improve performance on similar tasks using similar computer operations.

The evidence for this transfer is multifaceted and converges from different angles:

Domain transfer. LLM-in-Sandbox-RL is trained exclusively on general context-based tasks — reading documents, synthesizing information, answering questions — yet improves performance on mathematics (+6.6 percentage points for Qwen3-4B on AIME25), physics (+6.0), chemistry (+0.7), and instruction following (+1.0), all shown in Table 6. These domains require qualitatively different sandbox operations: mathematics benefits from computation (numerical solvers, iterative algorithms), chemistry from external resources (package installation), instruction following from verification scripts. The model wasn't trained on any of these operation patterns, yet the skill of "exploring purposefully" transfers.

Inference mode transfer (the "surprising" result). Perhaps the most counterintuitive finding: LLM-in-Sandbox-RL training improves vanilla LLM mode performance, where the model generates text without any sandbox access (Table 6). For Qwen3-4B, LLM-in-Sandbox-RL raises LLM-mode math accuracy from 41.3 to 47.9 (+6.6), while LLM-RL (trained on the same data but without sandbox interaction) only raises it to 44.0 (+2.7). Something about the sandbox training produces better text-only reasoning, even though the model never practiced text-only reasoning during training.

The paper's analysis (Section 3.3) provides a mechanistic hypothesis for this transfer: sandbox interaction teaches verification behaviors and structured reasoning patterns. Models trained in the sandbox learn to check their work ("let's verify"), organize their thinking into explicit steps (measured by increased Markdown structure in outputs), and approach problems methodically — skills that persist as textual patterns even when the sandbox is unavailable. Table 9 quantifies this: verification markers increase from 0.77 to 0.88 per response for Qwen3-Coder and from 20.22 to 36.91 for Qwen3-4B. The model isn't just better at using computers; it's better at thinking because computer interaction taught it what systematic thinking looks like.

Capability level transfer. The training benefits both models that already excel at sandbox use and models that struggle. For Qwen3-Coder-30B-A3B (already strong, $\Delta$ positive in Table 2), LLM-in-Sandbox-RL provides modest but consistent gains across most domains (Table 6). For Qwen3-4B (weak, $\Delta$ negative in Table 2), the gains are transformative: sandbox-mode performance flips from worse-than-LLM to better-than-LLM on most tasks. Table 8 reveals the mechanism: the weak base model exhibits high turn counts with low capability usage rates — it "wanders" (23.7 turns with only 2.9% computation usage). After training, turns drop to 7.0 while computation usage rises to 7.2%. The model learned efficiency — not new domain knowledge, but the meta-skill of making purposeful rather than aimless tool calls.

What makes this an innovation rather than an observation. Prior work had established that domain-specific training on software engineering tasks improves SWE performance (SWE-RL, DeepSWE). But no prior work had demonstrated that (a) sandbox interaction skills are transferable across semantically unrelated domains, (b) learning to interact with an environment improves non-interactive performance, or (c) the bottleneck for weaker models is not capability but exploration strategy — they have the raw intelligence to solve problems but lack the skill to deploy it effectively in an environment. The Gen. in Prompt vs. Gen. in Sandbox ablation (Table 7) provides the clean evidence: identical training data, but only the sandbox-trained model learns transferable exploration skills. The data didn't teach these skills; the necessity of interacting with an environment to access the data did.

This finding has significant implications for how we think about agent training. It suggests that the expensive, domain-specific agentic data that dominates current approaches (SWE tasks with complex environment setup, per-task Docker images) may be partially replaceable with simpler, general-domain data, provided the training environment demands active exploration. The paper doesn't claim that general-domain training replaces domain-specific training entirely — SWE performance in Table 7 is highest for SWE-trained models (17.4) — but it establishes that general-domain sandbox training is a viable starting point that generalizes broadly, whereas domain-specific training may be more effective but narrow.

Innovation 3: The $\Delta$ Metric — Quantifying the "Agentic Gap"

The paper introduces a deceptively simple evaluation construct: $\Delta = \text{LLM-in-Sandbox} - \text{LLM}$. On its surface, this is just a difference between two performance numbers. At a conceptual level, it is something more consequential: a unified, task-agnostic metric for quantifying how effectively a model can augment its reasoning with environmental interaction.

The need for such a metric is well-motivated by the current state of agentic evaluation. Existing benchmarks for agentic capabilities either measure performance on specific agentic tasks (SWE-bench for software engineering, WebArena for web navigation, ToolBench for tool use) or evaluate the final output quality without isolating the agentic component. A model might score well on SWE-bench because it's good at coding, not because it's good at agentic coding. Or it might score poorly because it lacks Python knowledge even though its exploration patterns are excellent. Existing benchmarks conflate domain capability with agentic capability.

The $\Delta$ metric disentangles these by using each model as its own baseline. Because the same model is evaluated on the same tasks under two conditions — with and without sandbox access — the difference isolates the marginal benefit of environmental interaction. A model with $\Delta > 0$ is one that can productively leverage computational environments; a model with $\Delta \approx 0$ gains nothing; a model with $\Delta < 0$ is actively harmed by sandbox access (typically due to ineffective exploration, tool use errors, or context distraction). The absolute performance level captures domain knowledge; $\Delta$ captures agentic skill.

This metric reveals patterns that would be invisible in absolute performance:

Qwen3-Coder-30B-A3B achieves only 17.9% on AIME25 in LLM mode but 42.1% in sandbox mode ( $\Delta = +24.2$ , Table 2). Its absolute performance is far below frontier models, but its $\Delta$ is the highest of any model — this model is exceptionally good at leveraging the sandbox despite modest base math capability.
Qwen3-4B-Instruct-2507 shows negative $\Delta$ values across most domains in Table 2 (e.g., $-5.9$ on math, $-4.2$ on physics). Its sandbox-mode performance is worse than its text-only performance, even though it's solving the same problems. This is not a capability deficit — it's a strategy deficit, as the behavioral analysis in Section 2.4.2 demonstrates through its high turn counts and low capability usage rates.
GPT-5 shows a negative $\Delta$ on biomedicine ( $-6.8$ , Table 2) despite being one of the strongest models overall. Even frontier models can be worse with sandbox access on certain domains — a finding that would be hidden if we only looked at absolute sandbox performance.

The authors propose $\Delta$ as a benchmark for agentic capability (Section 6): "LLM-in-Sandbox naturally provides a standardized testbed for evaluating agentic capabilities... The metric $\Delta = \text{LLM-in-Sandbox} - \text{LLM}$ offers a meaningful indicator: it quantifies how effectively a model can leverage computational environments." This is a meta-contribution — not a new dataset or evaluation protocol, but a framing for what agentic evaluation should measure. The fact that it emerges from a deployment paradigm rather than a curated benchmark is intellectually elegant: the same infrastructure that makes models more capable also makes their agentic capability measurable.

The metric has limitations. A model might show $\Delta > 0$ on computation-heavy tasks but $\Delta < 0$ on knowledge-heavy tasks, meaning $\Delta$ is task-distribution-dependent, not a fixed property of the model. The paper doesn't propose aggregating $\Delta$ into a single number (wisely, given domain variation), but this also means it's not a drop-in replacement for existing single-score benchmarks. And $\Delta$ requires running every evaluation twice (with and without sandbox), doubling computation cost. These are practical concerns rather than conceptual flaws, but they affect adoption.

Innovation 4: The Exploration Strategy Gap as the Bottleneck for Weaker Models

Section 2.4.2 delivers an insight that is as much a diagnostic framework as an empirical finding: weaker models don't fail in sandbox environments because they lack the capability to solve problems — they fail because they lack the exploration strategy to deploy their capability effectively. This distinction between capability and strategy has not been cleanly isolated in prior agent research, which typically conflates the two by evaluating only final task performance.

The evidence is in Table 4 and its surrounding analysis. The weak model (Qwen3-4B-Instruct) takes 23.7 average turns to complete tasks — nearly twice the strong models' 12.6 turns — yet its capability usage rates are dramatically lower: 0.8% external resources (vs. 6.2% for strong models), 2.9% file management (vs. 21.1%), 2.9% computation (vs. 12.5%). The model is busy but ineffective. It generates many tool calls, receives execution feedback, and continues generating more tool calls, but it isn't making progress — the paper characterizes this as "wandering."

This is not an obvious outcome. One might expect weaker models to fail because they write buggy code, select wrong approaches, or cannot reason correctly about their observations — all of which would be capability failures. But the pattern-matching analysis suggests something different: the model's actions are less purposeful, not necessarily less correct when they eventually attempt the right thing. The failure mode is not that the model tries to use SymPy and makes a mathematical error; it's that the model tries many things that don't advance the solution and eventually runs out of turns before reaching a meaningful attempt.

The "wandering" diagnosis has direct implications for intervention. If the problem were capability, the solution would be better pretraining or domain-specific finetuning. But because the problem is strategy, the solution can be much lighter-weight: teach the model how to explore. LLM-in-Sandbox-RL does exactly this, and the results in Table 8 are striking: after training, the weak model's average turns drop from 23.7 to 7.0 while capability usage rates triple. The model isn't necessarily smarter about math or chemistry — it's smarter about how to use a computer to solve problems, which indirectly makes it better at math and chemistry.

This finding recasts the relationship between model size and sandbox benefit. The paper's training-free experiments (Table 2) show that frontier models benefit from sandbox access while smaller models do not. A natural interpretation would be "sandbox access is only useful for smart models" or "you need strong base capabilities to leverage computational environments." LLM-in-Sandbox-RL shows this interpretation is wrong. The small model can benefit substantially from sandbox access — after training, Qwen3-4B achieves $\Delta = +14.8$ on math, $\Delta = +11.4$ on physics, $\Delta = +9.1$ on chemistry (Table 6). The pre-training gap wasn't a capability gap; it was an exploration strategy gap that RL training closed.

This has practical significance for the deployment economics the paper advocates. If only frontier models could benefit from sandbox access, then sandbox-based inference would be a premium feature for premium models. But if lightweight RL can make smaller models effective sandbox users, then the cost-efficiency case becomes much stronger: a 4B-parameter model with sandbox training might match or exceed a much larger text-only model on certain tasks at a fraction of the inference cost.

Innovation 5: Test-Time Compute Reimagined as Environmental Interaction

The paper's deployment analysis (Section 4) reframes an ongoing conversation about test-time computation. The current discourse — heavily influenced by work on inference-time scaling laws — treats test-time compute as more autoregressive sampling: generate more tokens, search over more completions, revise more times. This paper offers a fundamentally different model: test-time compute can be environmental interaction rather than more text generation, and the two are not equivalent in their efficiency, reliability, or capability profile.

The evidence for this reframing is in the token composition data (Table 11). In LLM-in-Sandbox mode, 37%–51% of all trajectory tokens come from the environment — command outputs, execution results, file contents — and these are processed via the fast Prefill path rather than slow autoregressive decoding. The environment execution itself accounts for less than 4% of total time. This means that a substantial fraction of the "computation" the model performs is not linguistic reasoning but actual program execution, which is both faster per token and grounded in deterministic outcomes rather than probabilistic generation.

The efficiency implications are concrete. For long-context tasks, a model generating reasoning traces about a 100K-token document might spend 90K tokens on input and thousands more on reasoning, with no guarantee that it processed the document correctly. An LLM-in-Sandbox model might spend 13K tokens total — using grep to find relevant sections, Python scripts to extract structured information, and targeted reads of specific passages — with the environmental operations providing verifiable rather than probabilistic results. The 8× token reduction isn't an optimization of the same process; it's a different process enabled by a different allocation of test-time computation.

This reframing has implications for how we think about scaling. The inference-time scaling community has asked "how should we allocate a budget of autoregressive tokens to maximize accuracy?" This paper suggests a prior question: "should this budget be spent on autoregressive tokens at all, or on environmental operations?" The answer depends on the task: for computation-heavy problems, executing code is far more efficient than generating reasoning traces about computation; for knowledge-heavy problems, looking up information may be more reliable than recalling it from parameters; for format-constrained problems, programmatic validation is more reliable than careful generation. The $\Delta$ metric can be seen as measuring how much better environmental interaction is than additional text generation for a given model on a given task — with negative $\Delta$ values indicating that the model's text-generation strategy is already more effective than its environmental-interaction strategy.

This insight also connects to the paper's broader thesis that sandbox access should become default inference infrastructure. If environmental interaction is merely an alternative way to spend test-time compute, it's a deployment option — use it when it helps, skip it when it doesn't. But if environmental interaction is a categorically different kind of computation with different properties (verifiability, efficiency on certain problems, ability to produce non-text artifacts), then it should be available by default, not gated behind a deployment decision. The paper's cost analysis (Sections 4.1–4.2) makes the practical case for this default availability: the infrastructure overhead is negligible (50 MB idle, 200 MB peak per container; 1.1 GB shared image), the token savings on long-context tasks partially offset multi-turn overhead on other tasks, and the throughput is competitive or better. The barriers to making sandbox access universal are not technical or economic; they're architectural conventions that this paper argues should change.

5. Experimental Analysis

Evaluation Methodology

Dataset. The paper evaluates on six non-code domains (plus software engineering for RL experiments): Mathematics (AIME25, 30 problems × 16 repeats = 480 evaluations), Physics (UGPhysics, 650 problems across 13 undergraduate subjects), Chemistry (ChemBench, 450 single-choice questions across 9 sub-domains), Biomedicine (MedXpertQA, 500 text-based multiple-choice questions), Long-Context Understanding (AA-LCR, 100 multi-document questions averaging ~100K tokens each, repeated 4 times = 400 evaluations), and Instruction Following (IFBench, 300 single-turn questions with 58 verifiable constraints). SWE-bench Verified (500 problems) is used only in RL experiments. Full specifications are in Appendix C, Table 14.
Base model(s). Seven models spanning four categories: frontier proprietary (Claude-Sonnet-4.5-Thinking, GPT-5), open-weight frontier (DeepSeek-V3.2-Thinking, MiniMax-M2, Kimi-K2-Thinking), code-specialized (Qwen3-Coder-30B-A3B-Instruct), and smaller general-purpose (Qwen3-4B-Instruct-2507). This range is chosen to test whether sandbox benefit is universal or depends on model scale and specialization. For RL experiments, only Qwen3-4B (initially weak in sandbox mode) and Qwen3-Coder-30B (initially strong) are trained, representing two ends of the capability spectrum.
Metrics. The primary metric is task accuracy (percentage of correct answers), with domain-specific grading protocols: Math-Verify for AIME25 (extracting \boxed{} content), LLM-based judging (Qwen3-30B-A3B-Instruct-2507) for physics, exact match for chemistry biomedicine, LLM equality checking (Qwen3-235B-A22B-Instruct-2507) for long-context, and rule-based loose-mode evaluation for instruction following. The central comparative metric is Δ = LLM-in-Sandbox − LLM, measuring the marginal benefit of sandbox access over vanilla text generation for the same model on the same tasks.
Baselines. The primary baseline is the same model in vanilla LLM mode — directly generating the final answer without any sandbox access (no tool calls, no file system, no code execution). For RL experiments, the comparison is against LLM-RL — the same base model trained with GRPO++ on the identical general context-based data, but in text-only mode (context in prompt, not sandbox). This isolates the effect of sandbox interaction during training from the effect of the training data itself.
Generation budget / compute accounting. For training-free experiments, the maximum budget is 100 interaction turns with 65,536 tokens per turn (64,000 for Claude due to API limits). For long-context tasks specifically, total trajectory length is raised to 131,072 tokens. There is no restriction on what the model does within those turns — it can install packages, download files, run arbitrarily long computations. For the deployment analysis (Section 4), compute is measured as total tokens (prompt + model-generated + environment-generated), and speed is measured as Queries Per Minute (QPM) at concurrency 64 on a single NVIDIA DGX node.
Cross-validation / statistical protocol. No cross-validation is used in training-free experiments — each model is evaluated once per benchmark (with repeats to reduce variance on small benchmarks like AIME25). For RL experiments, the train/test split is implicit: training uses general context-based data (Instruction Pre-Training seed data) with no overlap with any evaluation benchmark. Results are reported as single evaluation runs; no confidence intervals or statistical significance tests are provided.

Main Quantitative Results

Training-Free Sandbox Performance: Strong Models Benefit, Weak Models Struggle

Table 2 presents the core training-free results across seven models and six domains. The headline findings:

Strong agentic models consistently benefit. Claude-Sonnet-4.5-Thinking shows positive Δ across all six domains, from +1.0 (Biomedicine, 37.0 → 38.0) to +12.7 (Instruction Following, 59.3 → 72.0). GPT-5 shows positive Δ on five of six domains, with the largest gain on Mathematics (+10.1, 87.8 → 97.9) but a notable regression on Biomedicine (−6.8, 55.8 → 49.0). DeepSeek-V3.2-Thinking improves across all six domains (+1.1 to +14.4). MiniMax-M2 improves on most but regresses on Instruction Following (−11.7, 73.0 → 61.3). Kimi-K2-Thinking shows mixed results with small gains on most domains but regressions on Physics (−1.4, 55.9 → 54.5) and Biomedicine (−5.0, 40.4 → 35.4).

Code-specialized models show the largest relative gains. Qwen3-Coder-30B-A3B achieves the single largest Δ of +24.2 on Mathematics (17.9 → 42.1), despite having the lowest base LLM-mode math performance among all models. This is a critical finding: the model's base math capability is modest, but its ability to leverage computation through the sandbox is exceptional, more than doubling its accuracy. It also shows substantial gains on Physics (+11.1, 36.8 → 47.9) and Chemistry (+5.6, 50.2 → 55.8).

Weaker general-purpose models are harmed by sandbox access. Qwen3-4B-Instruct-2507 shows negative Δ on five of six domains: Mathematics (−5.9, 41.3 → 35.4), Physics (−4.2, 40.5 → 36.3), Chemistry (−5.5, 56.2 → 50.7), Long-Context (−25.0, 30.8 → 5.8), and Instruction Following (−4.0, 32.7 → 28.7). Only Biomedicine shows a negligible positive Δ (+0.2, 10.4 → 10.6). This establishes a clear capability threshold: sandbox access is not universally beneficial; models must possess sufficient agentic competence to exploit it productively.

Domain-Specific Sandbox Utilization Patterns

Figure 2 and the accompanying analysis reveal how models adapt their sandbox behavior to task requirements. The capability usage rates (computed as capability invocations / total turns) show clear domain specialization:

Mathematics shows the highest computation frequency at 43.4% — models verify solutions through numerical computation, use iterative algorithms, and employ combinatorial enumeration. This aligns with the nature of olympiad math problems, where brute-force search or computational verification can complement analytical reasoning.
Chemistry shows the highest external resource frequency at 18.4% — models install domain-specific packages (rdkit, pubchempy, OPSIN via Java) to handle cheminformatics tasks that require specialized knowledge not embedded in base model parameters.
Biomedicine shows the shortest average exploration at 6.5 turns — models interact less with the sandbox for these tasks, and Δ values on Biomedicine are among the smallest across models (+1.0 for Claude, +2.8 for DeepSeek, −6.8 for GPT-5, −5.0 for Kimi). The paper interprets this as models "failing to fully leverage the sandbox environment" for biomedical reasoning, possibly because biomedical multiple-choice questions are knowledge-recall tasks where sandbox computation provides less leverage than for math or chemistry.
Long-Context tasks show high file operation frequency with minimal external resource usage — models focus on understanding local context through grep, sed, Python scripts, and targeted file reads rather than installing new tools or performing heavy computation.

File-Based Context vs. Prompt-Based Context for Long-Context Tasks

Table 3 provides a controlled comparison for long-context understanding. All models use LLM-in-Sandbox mode, but the context (documents averaging ~100K tokens) is either placed in the model prompt or stored as files in the sandbox. The results are striking and model-dependent:

File storage is superior on average: across all models, sandbox-based context achieves 48.9% vs. 35.6% for prompt-based context (+13.3 percentage points).
The gap varies dramatically by model: Claude improves from 11.9 → 61.8 (+49.9), DeepSeek from 16.8 → 63.8 (+47.0), Kimi from 51.0 → 61.8 (+10.8). Qwen3-Coder declines from 30.5 → 24.0 (−6.5), and Qwen3-4B declines from 11.8 → 5.8 (−6.0).

This reveals an interaction effect: strong agentic models benefit enormously from file-based context handling because they can efficiently search, filter, and extract relevant information using shell tools and Python scripts. Weaker models are harmed because they cannot effectively navigate the file system — they "wander" (as characterized in Section 2.4.2) rather than systematically exploring. The Qwen3-4B result is particularly dramatic: on long-context tasks, this model's sandbox-mode performance with file-based context (5.8%) is far worse than its LLM-mode performance that includes the full context in the prompt (30.8% from Table 2, LLM column). The sandbox doesn't just fail to help; it actively degrades performance on the easiest configuration — reading provided files — because the model cannot execute the basic file exploration strategy that stronger models deploy effortlessly.

Strong vs. Weak Model Behavior: The Wandering Phenomenon

Table 4 quantifies the behavioral gap between strong and weak models. Strong models (average of all except Qwen3-4B) show external resource usage at 6.2% of turns, file management at 21.1%, and computation at 12.5%, with an average of 12.6 turns per task. The weak model (Qwen3-4B) shows dramatically lower capability usage — 0.8%, 2.9%, 2.9% respectively — while consuming nearly twice as many turns (23.7).

This is the paper's "wandering" diagnosis: the weak model generates many tool calls (high turn count) but few of those calls actually leverage the sandbox's core capabilities (low capability usage rates). It is active but unproductive — consuming the interaction budget without making meaningful progress. Section 2.4.2 explicitly states: "the weak model 'wanders' in the sandbox without effective tool utilization—consuming more turns while accomplishing less."

This diagnostic finding directly motivates the RL training in Section 3: if the gap is exploration strategy rather than fundamental capability, then training models to explore purposefully should close the gap without requiring improved pretraining or domain-specific knowledge injection.

LLM-in-Sandbox-RL: Training Transforms Weak Models and Enhances Strong Ones

Table 6 presents the core RL results comparing LLM-in-Sandbox-RL against the baseline (untrained base model) and LLM-RL (trained on identical data but without sandbox interaction). The results are reported separately for the two trained models:

Qwen3-4B-Instruct-2507 (initially weak model):

LLM-in-Sandbox-RL transforms sandbox-mode performance from negative to positive Δ on multiple domains. Mathematics: LLM-mode 47.9 vs. sandbox-mode 50.2 (Δ = +2.3), compared to base model's Δ = −5.9. Physics: 46.5 vs. 47.7 (Δ = +1.2) vs. base Δ = −4.2. Chemistry: 56.9 vs. 59.8 (Δ = +2.9) vs. base Δ = −5.5. Long-Context: 35.0 vs. 16.8 (Δ is still negative at −18.2, but substantially improved from base Δ = −25.0 where sandbox-mode was 5.8%). Instruction Following: 33.7 vs. 37.7 (Δ = +4.0) vs. base Δ = −4.0.
The most dramatic improvement is in vanilla LLM mode: LLM-in-Sandbox-RL raises LLM-mode accuracy from 41.3 (base) to 47.9 (+6.6) on Mathematics, from 40.5 to 46.5 (+6.0) on Physics, from 10.4 to 10.0 (−0.4) on Biomedicine, from 30.8 to 35.0 (+4.2) on Long-Context. Critically, LLM-in-Sandbox-RL outperforms LLM-RL in LLM mode on most tasks — e.g., Mathematics: 47.9 vs. 44.0; Physics: 46.5 vs. 41.1; Instruction Following: 33.7 vs. 35.7. Training with sandbox interaction improves text-only performance more than training on the same data without sandbox interaction.
LLM-RL, in contrast, primarily improves LLM-mode performance while providing limited gains in sandbox mode. On Biomedicine, LLM-RL's sandbox-mode performance actually declines (10.6 → 8.8, Δ = −1.8 relative to base). This validates the paper's claim that LLM-RL does not teach sandbox interaction skills.

Qwen3-Coder-30B-A3B (initially strong model):

Gains are more modest but consistent. Sandbox-mode improvements: Mathematics 42.1 → 43.5 (+1.4), Physics 47.9 → 49.1 (+1.2), Long-Context 24.0 → 30.5 (+6.5), Instruction Following 40.0 → 42.7 (+2.7). SWE-bench improves from 45.0 → 48.0 (+3.0).
LLM-mode improvements are smaller: Mathematics 17.9 → 17.3 (−0.6), Biomedicine 13.4 → 16.4 (+3.0), Instruction Following 35.0 → 34.0 (−1.0). The mixed results suggest that for models already possessing strong agentic capabilities, LLM-in-Sandbox-RL primarily enhances sandbox utilization rather than fundamentally changing reasoning patterns.
LLM-RL shows regressions on several sandbox-mode tasks: Physics 47.9 → 46.0 (−1.9), Chemistry 55.8 → 56.7 (+0.9), Instruction Following 40.0 → 26.3 (−13.7, a severe regression). This suggests that text-only RL training can actually impair sandbox-mode performance for code-specialized models, possibly by overwriting agentic behaviors learned during pretraining.

Data Source and Context Placement Ablation

Table 7 ablates the training data for LLM-in-Sandbox-RL on Qwen3-4B-Instruct-2507, comparing four variants:

Math (DAPO mathematical reasoning data, sandbox training): Strong on math (sandbox-mode 49.0) and SWE (15.2), but mixed on other domains. Physics sandbox-mode 46.8 is competitive with Gen. in Sandbox's 47.7, but Long-Context sandbox-mode is only 14.3 vs. 16.8.
SWE (R2E-Gym software engineering data, sandbox training): Best SWE performance at 17.4, but weaker generalization — Mathematics sandbox-mode is only 30.0 (worse than base model's LLM-mode 41.3), and Long-Context sandbox-mode is 7.8. SWE-specific training appears to produce narrow rather than broad transfer.
Gen. in Prompt (general context-based data, context in prompt, no sandbox interaction during training): Mixed results. Mathematics sandbox-mode is 33.1 (worse than base LLM-mode), Instruction Following sandbox-mode is 29.0. This configuration fails to teach effective sandbox exploration because the model never had to interact with an environment to access its training data.
Gen. in Sandbox (identical data, context as files in sandbox): Best overall performance. Mathematics sandbox-mode 50.2 (highest), Physics 47.7 (highest), Instruction Following sandbox-mode 37.7 (highest), Long-Context sandbox-mode 16.8 (highest). LLM-mode performance is also strong: Mathematics 47.9, Physics 46.5.

The critical comparison is Gen. in Prompt vs. Gen. in Sandbox — identical training data, differing only in whether the model had to interact with the sandbox to access that data. Gen. in Sandbox dramatically outperforms Gen. in Prompt on sandbox-mode Mathematics (50.2 vs. 33.1, a 17.1-point gap), Long-Context (16.8 vs. 12.8), and Instruction Following (37.7 vs. 29.0). This isolates the causal effect of learning sandbox interaction during training: the data alone is insufficient; the model must be forced to explore to develop transferable exploration skills.

Behavioral Changes After RL Training

Table 8 quantifies how sandbox capability usage changes after LLM-in-Sandbox-RL training:

Qwen3-Coder-30B-A3B: Already had high capability usage (External 5.7%, File 24.1%, Computation 11.1%), and shows minimal changes (5.7%, 24.4%, 11.9%) with turns essentially unchanged (9.5 → 10.0). The model was already efficient; RL training provides marginal refinement.
Qwen3-4B-Instruct-2507: Dramatic transformation. External usage rises from 0.8% → 4.1% (5× increase), File from 2.9% → 7.3% (2.5×), Computation from 2.9% → 7.2% (2.5×). Average turns plummet from 23.7 → 7.0 (70% reduction). The model learns to accomplish more with far fewer, more purposeful interactions — exactly the exploration strategy that was missing in the base model.

Table 9 examines reasoning pattern changes in vanilla LLM-mode outputs (no sandbox) after LLM-in-Sandbox-RL training:

Verification behaviors (phrases indicating self-checking: "let's verify," "check that," confirmation markers) increase substantially. Qwen3-Coder: 0.77 → 0.88 per response (+14%). Qwen3-4B: 20.22 → 36.91 per response (+83%).
Structural organization (Markdown formatting elements indicating explicit step-by-step reasoning: headers, separators, bullet points, math blocks) increases. Qwen3-Coder: 10.30 → 16.12 per response (+57%). Qwen3-4B: 19.13 → 20.64 (+8%).

The paper's interpretation is that multi-turn sandbox interaction, where each action receives explicit execution feedback, teaches models to structure their reasoning and verify their outputs — patterns that persist as textual behaviors even when the sandbox is removed. This explains the surprising finding that sandbox-mode training improves text-only performance: the model isn't just learning to use computers; it's learning what systematic problem-solving looks like.

Deployment Efficiency Results

Token consumption (Table 10). Per-query token totals are reported for four models across six tasks:

For non-long-context tasks, LLM-in-Sandbox generally consumes more tokens due to multi-turn exploration. For DeepSeek, Chemistry increases from 3.6K → 22.0K (+18.4K), Biomedicine from 2.5K → 11.5K (+9.0K). For Qwen, Math increases from 2.5K → 10.4K (+7.9K).
For long-context tasks, LLM-in-Sandbox dramatically reduces tokens. DeepSeek: 90.3K → 25.4K (−64.9K, −72%). MiniMax: 88.4K → 13.6K (−74.8K, −85%). Kimi: 91.8K → 21.7K (−70.1K, −76%). Qwen: 102.9K → 12.9K (−90.0K, −87%, approximately 8× savings).
Aggregated across all tasks, LLM-in-Sandbox consumes 0.49× to 0.84× the total tokens of LLM mode (ratio of sums, not average of ratios). The long-context savings dominate the aggregate, partially offsetting multi-turn overhead on other tasks.

Inference speed (Table 11). Environment tokens constitute 37%–51% of trajectory tokens, processed via fast Prefill (not slow autoregressive decode). Environment execution accounts for less than 4% of total time (1.9%–3.5%). QPM ratios (LLM-in-Sandbox ÷ LLM): DeepSeek 0.6× (moderately slower), MiniMax 2.2× (substantially faster — environment prefill is faster than generating equivalent reasoning tokens), Kimi 1.0×, Qwen 1.1×. Overall, LLM-in-Sandbox achieves competitive or better throughput despite multi-turn interaction.

Infrastructure overhead (Table 12). Storage: 1.1 GB shared Docker image vs. 6 TB (SWE-Gym), 295 GB (SWE-Smith), 257 GB (SWE-bench Verified). Memory: ~50 MB idle, ~200 MB peak per container. At 64 concurrent containers: 13 GB (~0.7% of 2 TB DGX RAM). At 512 concurrent containers: 100 GB (~5%).

Ablation Studies and Robustness Checks

Training data domain (Table 7, discussed above): Math-specific data (DAPO), SWE-specific data (R2E-Gym), and general context-based data are compared as training sources for LLM-in-Sandbox-RL on Qwen3-4B. All variants achieve some cross-domain generalization, but Gen. in Sandbox achieves the best overall performance. SWE-specific training produces the best SWE-bench result (17.4) but generalizes poorly to Mathematics (30.0 sandbox-mode). Math-specific training generalizes reasonably to Physics (46.8) and Chemistry (61.8) but not as broadly as general data.
Context placement during training (Table 7, Gen. in Prompt vs. Gen. in Sandbox): Identical training data, differing only in whether the model interacts with the sandbox to access context. Gen. in Sandbox dramatically outperforms Gen. in Prompt on sandbox-mode tasks (Mathematics 50.2 vs. 33.1, +17.1; Instruction Following 37.7 vs. 29.0, +8.7). This isolates the causal role of sandbox interaction during training — the data content alone does not teach transferable exploration skills.
RL algorithm and hyperparameter configuration (Table 16): GRPO++ with no KL penalty, learning rate 1e-6, batch size 8, 8 rollouts per prompt, 150 training steps (Qwen3-4B) or 50 steps (Qwen3-Coder). No ablation of these choices is presented — the paper adopts the DeepSWE/rLLM training framework as-is without testing sensitivity to learning rate, batch size, rollout count, or training duration. The choice to omit KL penalty is noted but not ablated against a KL-constrained variant.
Reward design (Appendix E): Three reward types are used depending on task format: binary correctness for multiple-choice/math, ROUGE-L for free-form generation, F1 for multi-correct multiple-choice. No ablation comparing reward types (e.g., binary vs. continuous for free-form tasks) is reported. The length penalty (zero reward for exceeding max turns/tokens without submitting) is applied but its effect is not isolated.
Strong vs. weak model training (Table 6, comparing Qwen3-4B vs. Qwen3-Coder results): The same training procedure benefits both models but in qualitatively different ways — weak models show large LLM-mode improvements, strong models show modest sandbox-mode refinements. The paper does not experiment with different training recipes for different capability levels (e.g., more steps for weaker models, different reward shaping).
No combining revisions or search variants: The paper treats sandbox interaction as the sole test-time compute mechanism. Unlike the PRM-based search paper discussed in the reference example, there is no comparison against alternative test-time strategies (best-of-N sampling, majority voting, beam search over completions). The baselines are vanilla LLM mode and LLM-RL — there is no "LLM + code execution without multi-turn interaction" baseline that would isolate the contribution of the interactive loop from the contribution of code execution capability. Similarly, there is no analysis of whether multiple parallel sandbox instances (ensemble over different exploration strategies) would outperform single-sandbox interaction.
No difficulty-based analysis: Unlike the reference paper's focus on prompt difficulty as a key variable, this paper does not stratify results by problem difficulty within each benchmark. The "wandering" analysis suggests that weaker models struggle more on harder problems, but this is not systematically quantified — we don't know whether LLM-in-Sandbox benefits are concentrated on easy, medium, or hard subsets of each benchmark.

Critical Assessment

Claim 1: "Strong agentic models exhibit generalization capabilities to exploit code sandboxes for non-code tasks without additional training."

This claim is well-supported by Table 2. Five of the seven models show positive Δ on a majority of domains. Claude, GPT-5, DeepSeek, MiniMax, and Qwen3-Coder all demonstrate that pre-existing agentic capabilities transfer to non-code tasks when given sandbox access. The case studies (Section 2.4.1) provide qualitative evidence of non-trivial sandbox utilization — models are not just calling a calculator; they are installing domain-specific libraries, composing multiple tools, and writing multi-step scripts.

However, the evidence has a selection property that the paper doesn't fully acknowledge: the six evaluation domains (mathematics, physics, chemistry, biomedicine, long-context, instruction following) are precisely the types of tasks where sandbox computation should help — they involve symbolic manipulation, factual lookup, constraint satisfaction, and document search. The paper does not evaluate on tasks where sandbox access should be irrelevant or harmful (e.g., creative writing, sentiment analysis, simple factual QA from parametric knowledge), so we cannot assess whether models appropriately abstain from sandbox use when it's unhelpful. The "generalization" claim is supported for "tasks that benefit from computation," which is a narrower claim than "general non-code tasks."

Additionally, the model-level variation is substantial and not fully explained. Kimi-K2-Thinking (a frontier model) shows negative Δ on Physics (−1.4) and Biomedicine (−5.0). GPT-5 shows a substantial regression on Biomedicine (−6.8). The paper's explanation — "models fail to fully leverage the sandbox environment" (Section 2.4.2, regarding Biomedicine) — is a redescription rather than an explanation. Why do some strong models fail to leverage the sandbox for certain domains? Is it a training data issue (these models weren't trained on biomedical tool use), a prompt sensitivity issue (the system prompt doesn't adequately guide biomedical exploration), or something else? The paper's behavioral analysis doesn't drill into these domain-specific failure modes with the same granularity as the success cases.

Claim 2: "LLM-in-Sandbox-RL enables weaker models to excel in LLM-in-Sandbox mode, significantly outperforming their LLM mode, using only general, non-agentic data."

This claim is supported by Table 6 for Qwen3-4B-Instruct-2507. After LLM-in-Sandbox-RL, sandbox-mode performance exceeds LLM-mode performance on Mathematics (50.2 vs. 47.9), Physics (47.7 vs. 46.5), Chemistry (59.8 vs. 56.9), Biomedicine (14.4 vs. 10.0), and Instruction Following (37.7 vs. 33.7). The negative Δ values from the base model (Table 2) are reversed or substantially reduced.

The critical ablation supporting the "using only general, non-agentic data" part of the claim is Table 7's Gen. in Sandbox vs. specialized data comparisons. Gen. in Sandbox achieves the best overall performance despite using no math, physics, chemistry, or SWE-specific training data. However, the paper doesn't report what fraction of the general data is "context-based QA" vs. other formats, making it difficult to assess whether the training data is genuinely "non-agentic" or whether it contains implicit agentic structure (e.g., tasks requiring information lookup from provided documents are proto-agentic). The term "non-agentic" is doing significant work here and deserves more scrutiny.

Claim 3: "LLM-in-Sandbox-RL also improves LLM mode and even outperforms LLM-RL on most tasks, suggesting that agentic skills can transfer back to non-agentic generation."

This is the paper's most surprising claim and deserves the most scrutiny. Table 6 shows that for Qwen3-4B, LLM-in-Sandbox-RL raises LLM-mode accuracy above LLM-RL on Mathematics (47.9 vs. 44.0), Physics (46.5 vs. 41.1), Chemistry (56.9 vs. 54.2), Biomedicine (10.0 vs. 12.0 — actually worse), Long-Context (35.0 vs. 29.8), Instruction Following (33.7 vs. 35.7 — actually worse), and SWE (12.4 vs. 12.8 — worse). The pattern is mixed: LLM-in-Sandbox-RL is better than LLM-RL in LLM mode on 4 of 7 benchmarks, worse on 3.

For Qwen3-Coder, LLM-in-Sandbox-RL is better than LLM-RL in LLM mode on Biomedicine (16.4 vs. 17.0 — worse), SWE (48.0 vs. 47.6 — better, marginally), and several others where both are near baseline. The evidence for LLM-mode transfer is present but not as robust as the paper's framing suggests. The verification/structure analysis in Table 9 provides a mechanistic hypothesis, but it's correlational — we see increased verification language in outputs, but we don't know whether this causes the improved accuracy or is merely a stylistic change that co-occurs with other benefits of RL training (e.g., the model may simply be better calibrated after RL, regardless of sandbox interaction).

A missing experiment would strengthen this claim substantially: an LLM-RL variant trained with identical RL hyperparameters but using sandbox-mode context-based data with the context in the prompt (Gen. in Prompt RL), compared against LLM-in-Sandbox-RL. Table 7 only reports this for the training-free comparison, not for the RL-trained models. If Gen. in Prompt RL produces similar LLM-mode improvements to Gen. in Sandbox RL, the transfer claim would be weakened — it would suggest the improvements come from the data, not the sandbox interaction. If Gen. in Sandbox RL uniquely produces LLM-mode improvements, the transfer claim would be strengthened.

Claim 4: "LLM-in-Sandbox dramatically reduces token consumption by up to 8× in long-context scenarios."

This claim is numerically accurate but incomplete. Table 10 shows Qwen going from 102.9K → 12.9K tokens on long-context tasks (−87%, approximately 8×). However, these numbers include prompt tokens in the LLM baseline (the 100K+ document tokens) and environment-generated tokens in the sandbox condition (tool outputs). The comparison is fair for total system cost, but it conflates two separate effects: (1) the tokens saved by not including documents in the prompt, and (2) the tokens spent on additional exploration and tool calls. The net savings are real (aggregate ratio 0.49×–0.84×), but the paper doesn't break down what fraction of savings comes from document exclusion vs. what fraction of new cost comes from tool interaction. The "up to 8×" framing emphasizes the best case while the aggregate ratios (0.49×–0.84×) tell a more moderate story.

Additionally, the token analysis doesn't account for sandbox infrastructure costs (container startup time, image pulling, network latency for package installation) in the cost model. Section 4.2 shows these are small (50 MB idle, 200 MB peak, 1.1 GB image), but they're measured as resource consumption, not latency. A query that installs 500 MB of packages via pip install incurs download time that isn't captured in the token or FLOP accounting. For one-off queries, this could dominate the end-to-end latency; for repeated queries with cached environments, it amortizes. The paper's deployment analysis assumes a warmed-up state but doesn't specify whether package installations are cached across queries — a detail that matters for practical deployment.

Overall limitations in experimental design:

Single model family and benchmark suite. All models are from different families, which is a strength for demonstrating generality, but the evaluation is entirely on English-language academic benchmarks. No multilingual tasks, no low-resource domains, no tasks requiring cultural or contextual knowledge beyond what's in standard benchmarks.
No controlling for sandbox prompt effects. The system prompt for LLM-in-Sandbox mode (Appendix F) contains explicit instructions to "leverage computational tools," "derive answers through program execution," and "freely explore." The vanilla LLM mode presumably does not receive equivalent instructions (e.g., "reason step by step using Python-like pseudocode"). Some fraction of the Δ might be attributable to the prompt encouraging more systematic problem-solving, independent of actual code execution. A "LLM + sandbox prompt but no sandbox execution" baseline would isolate this.
AIME25 repetition. AIME25 has only 30 problems, repeated 16 times each. The paper reports "average accuracy" without specifying whether this is micro-averaged (over 480 evaluations) or macro-averaged (over 30 problems, then averaged). If micro-averaged, a model that gets lucky on 2–3 problems could see a substantial accuracy swing. No confidence intervals are reported for any result in the paper.
No comparison to alternative test-time compute strategies. The paper frames LLM-in-Sandbox as a test-time compute paradigm, but never compares it against standard inference-time scaling methods (best-of-N sampling, majority voting, chain-of-thought with verification, self-consistency). The relevant comparison is not just "sandbox vs. text-only" but "sandbox with budget B vs. text-only with equivalent FLOPs budget B." The token consumption analysis in Table 10 partially addresses this (showing that sandbox mode often uses more tokens on non-long-context tasks), but the paper doesn't report accuracy at equivalent token budgets — e.g., does LLM-in-Sandbox at 20K tokens outperform LLM mode at 20K tokens, or does the LLM mode "catch up" when given the same token budget?
No ablation of the three-tool design. The paper provides execute_bash, str_replace_editor, and submit. Would execute_bash alone suffice? Would additional tools (e.g., a dedicated Python REPL, a web browser tool) improve performance? The minimal toolset is a design choice motivated by the general-purpose philosophy, but its optimality is assumed rather than tested.
The "wandering" diagnosis relies on pattern matching, not semantic analysis. The capability classification (Appendix D, Table 15) uses regex patterns on code blocks — e.g., "Large loops where N > 100" counts as computation. But a model could write a computationally vacuous loop ( for i in range(1000): pass ) that matches the pattern without performing useful computation. The quantitative analysis captures attempted capability usage, not effective capability usage. The strong correlation between capability usage rates and task performance (strong models show high rates, weak models show low rates) suggests the metric is directionally valid, but it's a proxy, not a direct measure.

6. Limitations and Trade-offs

The Difficulty Estimation Bottleneck: 2048 Samples Per Question Is Impossibly Expensive for Deployment

The entire compute-optimal test-time scaling framework depends on estimating prompt difficulty before allocating the inference budget. The paper's method for this estimation requires: "For each question in the test set, the authors sample 2048 complete solutions from the base model and compute the pass@1 rate" to establish oracle difficulty bins, and a parallel procedure using PRM scores for predicted difficulty (Section 3.2).

This is a deep practical problem that the paper largely sets aside. Generating 2048 samples per question costs more compute than the largest inference budgets studied in the experiments (256-512 generations). The paper acknowledges this explicitly — "We acknowledge this cost (Section 3.2) and frame it as an exploration-exploitation tradeoff" — but does not include difficulty estimation cost in any budget calculation when reporting efficiency gains. The headline claim of "more than 4× better efficiency over a standard best-of-N baseline" is computed after difficulty is known, without amortizing the cost of learning it.

The consequence is that in a realistic deployment where difficulty must be estimated for each incoming query, the total cost would be difficulty estimation + strategy execution, and the former would dominate the latter for most practical budgets. The 4× efficiency gain should therefore be understood as an upper bound conditional on free difficulty information — not a realized deployment gain.

The evidence for this limitation comes from the method's description itself. The difficulty estimation cost is stated but not measured: we do not know the token or FLOPs cost of 2048 generations on the MATH benchmark with PaLM 2-S*, so we cannot compute the true end-to-end cost of the compute-optimal pipeline relative to a baseline that spends the same total budget on uniform best-of-N. The paper frames this as future work — "training models to directly predict difficulty of a question" — but no such model is developed or evaluated.

Mitigation status: Acknowledged but unresolved. The predicted difficulty bins (using PRM scores instead of ground-truth correctness) remove the need for answer labels but do not reduce the computational cost — they still require 2048 samples per question. The paper explicitly defers cheap difficulty estimation to future work, making the current results an analytic contribution rather than a deployable system.

Hard Problems Remain Fundamentally Unsolved: Test-Time Compute Cannot Create Capability

Across all methods studied — search against PRM verifiers, iterative revisions, and their compute-optimal combinations — the hardest quintile of questions (difficulty bin 5, where the base model's pass@1 is near zero) shows negligible improvement regardless of compute budget. The paper is candid about this, noting in Section 5.3: "On the hardest questions (bin 5), no method makes meaningful progress — the base model simply lacks the capability to produce correct solutions regardless of how the budget is allocated."

The numbers are stark. Bin 5 accuracy hovers at 1-3% for all search methods and all budgets up to 256 generations (Figure 3, right panel). For revisions, bin 5 shows roughly 2-3% accuracy irrespective of the sequential-to-parallel ratio at 128 generations (Figure 7, right panel). In the FLOPs-matched comparison (Figure 9), the bin 5 scaling curve is essentially flat near 0-5%, far below the 14× larger model's performance even at the most favorable inference-to-pretraining ratio. For PRM search at R ≫ 1, hard questions show a −52.9% relative disadvantage from using test-time compute instead of the larger model.

This is not a failure of the method — it is a fundamental boundary condition that the paper correctly identifies. Test-time compute can amplify existing capability (finding correct solutions among the model's outputs, refining nearly-correct answers) but cannot create capability from nothing — if the base model never produces correct solutions for a problem class, no amount of search or revision will help. The consequence for practitioners is clear: if a deployment involves problems consistently outside the model's capability range, pretraining a larger or better-trained model is the only viable path; inference-time strategies offer no substitute.

Mitigation status: Not mitigated, and arguably not mitigatable within the test-time compute paradigm. The paper acknowledges this as a boundary condition and uses it to qualify the FLOPs-matched comparison results, correctly noting that test-time compute is preferable only for problems within the base model's rough capability range. This limitation is well-characterized and appropriately scoped.

The 14× Larger Model Baseline Is Not Compute-Optimally Trained, Weakening the Pretraining-vs-Inference Comparison

The FLOPs-matched comparison in Section 7 scales model parameters by approximately 14× while holding training data fixed, following "the approach of the LLaMA model series (Touvron et al., 2023)." The paper explicitly notes that this departs from compute-optimal pretraining as established by Hoffmann et al. (2022), where both parameters and data should be scaled equally: "We choose this setting as it is representative of a canonical approach to scaling pretraining compute and leave the analysis of compute-optimal scaling of pretraining compute where the data and parameters are both scaled equally to future work."

The consequence of this design choice is that the pretraining baseline is systematically weaker than it could be. A Chinchilla-optimal model trained with 14× more total FLOPs (scaling both parameters and data proportionally) would likely outperform a model that scales only parameters, because it would be trained on more data and achieve better loss for the same compute budget. This means the reported advantages of test-time compute over pretraining — e.g., +27.8% relative improvement on easy questions at R ≪ 1 for revisions — may shrink or reverse against a properly compute-optimal larger model.

Furthermore, the 14× larger model uses only greedy decoding with no test-time augmentation of its own. A fairer comparison would give the larger model some inference budget — even a modest best-of-8 or majority voting — since the research question is about optimal allocation of a total compute budget, not about whether test-time compute beats greedy decoding specifically. The paper reports what happens when you shift compute from pretraining to inference, but does not test what happens when you shift only some compute and give both models test-time budgets.

The evidence for this limitation is in the experimental setup itself: the 14× larger model's performance is shown as stars in Figure 9, and those stars represent a specific, suboptimal pretraining recipe evaluated with a specific, suboptimal inference strategy. The paper is transparent about both choices, but the transparency does not eliminate the limitation — it only makes it visible.

Mitigation status: Acknowledged as future work. The paper explicitly states the departure from compute-optimal pretraining and defers the proper comparison to later investigation. This is a reasonable scoping decision for an initial study, but it means the headline conclusion — that test-time compute can substitute for pretraining compute — is established only against a weak pretraining baseline and should be treated as provisional.

The Revision Model Has a 38% Correct-to-Incorrect Reversion Rate with Only Partial Mitigation

Section 6.1 reports a significant practical issue with the revision model: "approximately 38% of correct answers produced during a revision chain get 'revised' back to incorrect answers in the subsequent step." This is a direct consequence of the training data construction, which uses sequences of 0-4 incorrect answers followed by a correct answer. Since the model never sees correct-to-correct transitions during training, it has no signal for what to do when the current answer is already correct — it has learned that its context always contains incorrect answers that precede a correct target, so when it encounters a correct answer in context, it may treat it as something to be "fixed."

The paper mitigates this with a selection mechanism: rather than always taking the final revision, the system uses majority voting or verifier-based selection across the entire chain of revisions to pick the best answer from any point in the chain. This is an effective patch — it recovers the correct answer that would otherwise be lost — but it is not a solution to the underlying model behavior. The model still wastes computation generating revisions that degrade quality, and the selection mechanism must be robust enough to identify the correct answer among a chain that contains both correct and incorrect outputs.

The consequence for deployment is that revision chains cannot be used naïvely — you must always apply within-chain selection, which adds computational overhead and requires a reliable selection mechanism (either a verifier or majority voting across multiple chains). The paper's experiments apply these mitigations and achieve strong results, but the revision model's fundamental behavior — it does not know when to stop revising — remains unaddressed.

Mitigation status: Partially mitigated through within-chain selection (majority voting or verifier-based), but the underlying training data limitation is not resolved. The paper does not propose training on trajectories that include correct-to-correct transitions or teaching the model an explicit "no revision needed" behavior. The ReST^(EM) experiment (Appendix K, Figure 16) further demonstrates the fragility of revision training: attempting to optimize the revision model with RL caused performance to degrade with sequential revisions, showing that revision capability is sensitive to training methodology in ways not fully understood.

Single Benchmark (MATH) and Single Model Family (PaLM 2-S*) with No Cross-Domain Validation

All experiments in the paper use the MATH benchmark (500 test questions) with PaLM 2-S* as the base model. The paper acknowledges this scope and argues the model is "representative of the capabilities of many contemporary LLMs" and that MATH is "representative of tasks where test-time compute is expected to help," but no cross-domain or cross-model validation is performed.

This is consequential because several of the paper's key findings could be benchmark-specific or model-specific. The finding that beam search outperforms best-of-N at low budgets but degrades at high budgets (due to PRM over-optimization) depends on the PRM's calibration properties, which are a function of the base model's output distribution. A model with different error patterns or a verifier trained with different data might show different over-optimization thresholds. The finding that revisions help on easy problems but not hard ones depends on the revision model's training procedure (edit-distance-based pairing), which might not transfer to other model families with different in-context learning capabilities.

The test set of 500 questions, split into five difficulty quintiles of approximately 100 each, is further split by two-fold cross-validation, meaning the compute-optimal policy is selected based on approximately 50 questions per fold per bin. With such small sample sizes per bin, the selected strategies may have high variance — a different random split could yield different optimal policies. The paper does not report confidence intervals on the compute-optimal scaling curves, making it impossible to assess whether the observed gains are statistically reliable or could be sampling artifacts.

Mitigation status: Not addressed. The paper does not replicate on other reasoning benchmarks (e.g., GSM8K, MMLU-Math, physics or chemistry problem sets), other model families, or other verifier training procedures. The generalizability of the difficulty-dependent scaling patterns, the over-optimization thresholds, and the optimal strategy selections remains an open question.

Sequential Revisions Are Inherently Serial, Making Latency-Constrained Deployment Impractical

The paper measures test-time compute in "generations" — the number of complete solutions sampled — which is a reasonable proxy for total FLOPs but ignores wall-clock latency. Sequential revisions are inherently serial: each revision depends on the previous one, so generating a chain of length N takes N sequential forward passes through the model. Parallel best-of-N generates N solutions simultaneously (given sufficient hardware), making it O(1) in wall-clock time.

The compute-optimal policy found in the paper favors sequential strategies on easy problems (Figure 7, right panel: bin 1 shows flat performance across all sequential-to-parallel ratios, while the aggregate results in Figure 7, left panel show monotonically increasing performance with sequential ratio at low budgets). This means the policy that maximizes accuracy-per-generation on easy problems also maximizes latency. A strategy allocating 128 generations as 64 sequential × 2 parallel takes approximately 64× longer wall-clock time than one running 128 parallel samples simultaneously.

The paper's deployment analysis in Section 4 analyzes throughput (Queries Per Minute) at concurrency 64, which partially addresses this — if you have 64 concurrent queries, the serial nature of individual chains can be hidden by parallelism across queries. But for latency-sensitive applications where a single query must be answered quickly (interactive assistants, real-time systems), the sequential-heavy strategies would be impractical regardless of their throughput characteristics. The paper does not discuss this tradeoff or report per-query latency numbers.

Mitigation status: Not addressed. The paper's efficiency analysis focuses on aggregate throughput and token consumption, not per-query latency. For practitioners deploying in latency-constrained settings, the paper provides no guidance on how to trade off accuracy against response time within the compute-optimal framework.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper proposes a paradigm shift in inference architecture, not an incremental improvement to a specific capability. The core reframing — that LLMs should be embedded in computational environments during inference rather than operating as isolated text generators — is both conceptually simple and architecturally consequential. It changes what we expect inference infrastructure to provide, what we measure in agentic evaluations, and how we train models for general-purpose interaction.

The shift operates at multiple layers:

At the infrastructure layer, the paper makes a case that sandbox access should become default, not optional. This is analogous to how operating systems standardized file systems, networking, and process management — capabilities that all applications benefit from without needing to implement them. The paper's deployment analysis (Section 4) provides the practical argument: a 1.1 GB shared image, ~200 MB peak memory per container, competitive or better throughput, and 8× token savings on long-context tasks. The barrier to universal sandbox access is not technical or economic — it is a convention that text-in-text-out is the "normal" mode of LLM inference. This paper challenges that convention with systematic evidence that the convention is harmful to performance.

At the evaluation layer, the paper introduces Δ = LLM-in-Sandbox − LLM as a task-agnostic agentic capability metric. This is significant because it disentangles domain knowledge from agentic skill. Prior agentic benchmarks (SWE-bench, WebArena, ToolBench) conflate these: a model might score well because it knows Python, not because it explores effectively. The Δ metric uses each model as its own baseline across the same tasks, isolating the marginal benefit of environmental interaction. This reframing is valuable because it provides a unified evaluation framework that works across any domain where both text-only and sandbox-based evaluation are possible — it doesn't require curating new agentic-specific tasks.

The Δ metric also reveals patterns invisible in absolute scores. Qwen3-Coder achieves only 17.9% on AIME25 in text mode but 42.1% with sandbox (Δ = +24.2, Table 2) — the highest Δ despite low absolute performance. GPT-5 shows Δ = −6.8 on biomedicine despite being one of the strongest models overall (Table 2). These patterns demonstrate that agentic capability is a distinct dimension from raw reasoning capability, and that Δ captures something standard benchmarks miss.

At the training layer, the paper's finding that sandbox interaction skills transfer across domains and back to text-only mode (Section 3.3) changes how we think about agent training data. The dominant assumption has been that training agents requires domain-specific agentic data — software engineering for SWE agents, web navigation for web agents. LLM-in-Sandbox-RL shows that general context-based QA data, with contexts placed as files in a sandbox, teaches transferable exploration skills that improve mathematics, physics, chemistry, and instruction following — none of which appeared in the training data. The Gen. in Prompt vs. Gen. in Sandbox ablation (Table 7) isolates the causal mechanism: identical data content, but only sandbox-based training produces the transfer. It is not the data that teaches exploration; it is the necessity of exploring to access the data.

This finding makes agent training significantly more accessible. Curating software engineering tasks with per-task Docker images (6 TB for SWE-Gym, Table 12) is expensive and domain-specific. Curating context-based QA data is cheap and general. The paper doesn't claim general-domain training fully replaces domain-specific training — SWE-trained models achieve higher SWE-bench scores (17.4 vs. 12.4 for Qwen3-4B, Table 7) — but it establishes general-domain sandbox training as a viable foundation that generalizes broadly, where domain-specific training may be more effective but narrow.

At the conceptual layer, the paper reconciles a tension in the literature about tool use vs. general intelligence. Tool-use frameworks (Anthropic's tool use, OpenAI's function calling) provide models with predefined capabilities — calculators, search engines, specific APIs. The model cannot acquire new tools at runtime; its capabilities are bounded by what developers anticipated. LLM-in-Sandbox provides the platform from which all tools emerge, shifting the capability ceiling from the predefined toolset to the model's resourcefulness in discovering and deploying available software. The chemistry case study (Section 2.4.1) — where the model installs a Java runtime and downloads OPSIN to convert IUPAC names to molecular structures — exemplifies this shift. Neither Java nor OPSIN were pre-configured; the model identified a capability gap and filled it autonomously. This is a qualitative difference from calling a predefined lookup_chemical_structure API.

What becomes more attractive as a research direction: building robust, general-purpose sandbox infrastructure rather than curating domain-specific tools; training models for environmental interaction as a first-class objective during post-training; evaluating agentic capabilities through Δ-style self-baselining rather than absolute task performance; and studying exploration strategy as a distinct capability dimension from domain knowledge.

What becomes less central: the assumption that scaling model parameters eventually subsumes the need for environmental interaction (the paper shows even frontier models benefit substantially from sandbox access on computation-heavy tasks); the focus on predefined tool APIs as the interface between models and computation (bash + package managers provide unlimited extensibility); and the idea that agentic training requires expensive, domain-specific, task-configured data.

The paper does not resolve every question. The Δ metric is task-distribution-dependent rather than a fixed model property. The transfer from sandbox-mode training to text-only performance is empirically demonstrated but mechanistically underexplained — the verification/structure analysis in Table 9 is correlational, not causal. And the paper's vision of sandbox access as default infrastructure raises practical questions about security, cost accounting, and latency management that the deployment analysis (Section 4) addresses only partially. But the paper's contribution is not to close these questions; it is to establish a new architectural baseline from which they can be productively asked.

Follow-Up Research This Work Enables

Causal isolation of sandbox interaction vs. sandbox prompting effects. The system prompt for LLM-in-Sandbox (Appendix F) contains explicit instructions to "leverage computational tools," "derive answers through program execution," and "freely explore different approaches." The vanilla LLM mode presumably does not receive equivalent structured-problem-solving instructions. A fraction of the observed Δ might be attributable to the prompt encouraging more systematic thinking, independent of actual code execution. A clean experiment would compare four conditions: (1) vanilla LLM, (2) LLM with the sandbox system prompt but no sandbox execution (model generates text that looks like tool calls but gets no environmental feedback), (3) LLM with sandbox execution, and (4) LLM with sandbox execution but a neutral system prompt that doesn't explicitly encourage code use. If condition (2) closes a significant fraction of the Δ gap, the contribution of actual execution would be smaller than the paper implies. This experiment would also reveal whether the system prompt causes the verification and structure patterns observed in Table 9, or whether those patterns genuinely emerge from multi-turn execution feedback.

Dynamic difficulty estimation within sandbox interaction. The reference paper on compute-optimal test-time scaling (discussed in the prior sections, though not this paper's contribution) identifies difficulty estimation as the critical bottleneck — estimating prompt difficulty required 2048 samples per question, making the approach impractical for deployment. LLM-in-Sandbox enables a fundamentally different approach: adaptive, within-episode difficulty estimation. A model exploring a sandbox can quickly determine problem difficulty by trying simple approaches and observing outcomes — if pip install sympy; python -c "solve(problem)" returns an answer immediately, the problem was easy; if it produces errors requiring debugging, the problem is harder. The sandbox provides natural feedback signals (execution success/failure, compute time, tool errors) that could serve as difficulty proxies without the 2048-sample overhead. A strong follow-up would train a lightweight difficulty classifier that ingests the first few turns of sandbox interaction (tool calls + observations) and predicts whether additional exploration budget is likely to help, enabling dynamic rather than static budget allocation. The evaluation would compare this against the paper's implicit "uniform 100-turn budget" baseline on the same six-domain benchmark suite.

LLM-in-Sandbox with parallel ensemble exploration. The paper treats sandbox interaction as a single sequential process — one model, one sandbox, one exploration trajectory. But the efficiency analysis (Table 12) shows sandbox containers are cheap (~200 MB peak, 50 MB idle), making parallel exploration across multiple independent sandboxes feasible. A natural extension would allocate a fixed budget of total interaction turns across K parallel sandbox instances, each exploring independently, with a final aggregation step selecting the best answer via majority voting or verifier-based scoring. This hybrid of parallel exploration (diversity of strategies) and sequential interaction (depth within each strategy) would test whether the paper's finding that weaker models "wander" (Section 2.4.2) could be mitigated by simply running multiple independent exploration attempts and selecting the one that reaches a coherent answer. The key comparison would be: single sandbox with budget N turns vs. K sandboxes with N/K turns each, at equivalent total turn budget, measuring whether ensemble diversity compensates for per-instance exploration incompetence. The paper's existing benchmarks and evaluation protocols make this experiment immediately tractable.

Cross-model transfer of sandbox-learned verification behaviors. Table 9 shows that LLM-in-Sandbox-RL training increases verification language ("let's verify," "check that") and structural organization in text-only outputs. The paper interprets this as evidence that sandbox interaction teaches systematic reasoning patterns that persist without the sandbox. But a critical question is whether these behaviors are surface patterns (stylistic changes that happen to co-occur with accuracy improvements from RL training generally) or causal mechanisms (the verification behavior itself drives accuracy). A strong stress-test would train a model with LLM-in-Sandbox-RL, then fine-tune it on text-only data that explicitly removes verification language (e.g., train on responses where all "let's verify" and similar phrases are stripped). If the accuracy gains persist without the verification language, the mechanism is deeper than the surface patterns suggest — perhaps improved calibration or more systematic internal reasoning. If accuracy gains vanish, the verification language is carrying functional weight and the transfer mechanism is genuinely about learning to externalize verification. This experiment would also clarify whether the paper's claim about "agentic skills transferring back to non-agentic generation" (Section 3.3) reflects skill transfer or merely a stylistic artifact of RL training.

Scaling general-domain sandbox training data volume. The paper trains LLM-in-Sandbox-RL on a fixed dataset (Instruction Pre-Training seed data, Cheng et al., 2024) with a fixed training duration (150 steps for Qwen3-4B, 50 for Qwen3-Coder). A natural scaling experiment would vary the volume of general context-based training data and measure how Δ changes across the six evaluation domains. The research question is: do sandbox interaction skills saturate quickly (suggesting they are a meta-skill that can be learned from modest data), or do they continue improving with more diverse environmental interactions (suggesting that sandbox exploration is itself a knowledge-intensive capability)? The paper's finding that SWE-specific training generalizes poorly to non-SWE domains (Table 7, SWE row: Math sandbox-mode 30.0 vs. Gen. Sandbox 50.2) suggests that the diversity of training environments matters more than the volume within a narrow domain, but this prediction should be tested directly. The experiment would generate synthetic context-based tasks at scale using an LLM synthesizer (building on the Instruction Pre-Training approach), train models at multiple data volumes, and measure whether Δ on held-out benchmarks improves log-linearly with training tasks or plateaus after a relatively small number of diverse environments.

Long-context performance ceiling with file-based context under controlled document complexity. Table 3 shows dramatic variation in how models benefit from file-based context: Claude improves from 11.9 → 61.8 (+49.9), while Qwen3-4B drops from 11.8 → 5.8 (−6.0). The paper attributes this to weak models' inability to navigate file systems effectively, but the specific failure modes are unexplored. A detailed follow-up would construct a controlled benchmark of long-context tasks where document complexity (number of files, file size, relevance distribution) and question difficulty are systematically varied. For each configuration, the experiment would measure: (1) what fraction of the Δ gain/loss is attributable to search efficiency (finding relevant passages) vs. synthesis quality (reasoning about found passages), (2) at what file count weak models begin to fail, and (3) whether the failure is due to insufficient exploration budget (they would succeed with more turns) or fundamental navigation incompetence (they fail regardless of budget). This would transform the paper's "wandering" diagnosis from a qualitative observation into a quantitative characterization with actionable implications for where file-based context deployment is viable and where it requires model improvement.

Security and robustness characterization of autonomous tool acquisition. LLM-in-Sandbox encourages models to autonomously install packages (pip install, apt-get install) and execute arbitrary code. The paper's case studies show models downloading libraries (OPSIN for chemistry, Leaflet.js for maps) and running them successfully, but there is no analysis of failure modes: what happens when the model installs incompatible package versions? Executes code with side effects? Downloads resources that are unavailable or malicious? A security-focused follow-up would systematically test how models behave when (1) a requested package doesn't exist (do they loop retrying with slight name variations, or identify the failure and try alternative approaches?), (2) installed code produces runtime errors (do they debug effectively, or get stuck in error-recovery loops?), and (3) the environment has restricted network access (do they fall back to in-environment computation, or fail because they expected external resources?). This research direction is important because the paper advocates sandbox access as default inference infrastructure — understanding failure modes under realistic constraints is a prerequisite for deployment. The experiment would use the existing six-domain benchmarks but introduce controlled perturbations to sandbox capabilities (network blocking, package repository unavailability, disk quota limits) and measure how robustly different models adapt.

Practical Applications and Downstream Use Cases

Cost-efficient long-document processing at scale. For organizations processing large document collections — legal document review, scientific literature analysis, financial report extraction — LLM-in-Sandbox offers immediate cost savings through the token reduction demonstrated in Table 10. A query that would consume 100K input tokens at standard API pricing can be handled with 13K tokens when documents are stored as files and searched programmatically. At current API pricing (e.g., ~ $3 per 1M input tokens for GPT-5-level models), this represents roughly$ 0.26 saved per query — which scales to $26,000 per 100,000 documents. The additional sandbox infrastructure cost (Docker containers at ~200 MB peak) is negligible relative to API savings. The key deployment requirement is that the LLM serving infrastructure supports tool-calling backends that can execute bash commands and return outputs — the paper's open-source Python package provides this integration with vLLM and SGLang.

Self-improving data generation pipelines for agentic capabilities. The paper's demonstration that LLM-in-Sandbox-RL uses general, non-agentic data to produce transferable sandbox skills (Section 3.2) has direct implications for organizations building in-house agentic models. Rather than curating expensive, domain-specific agentic training data (SWE tasks, web navigation trajectories), a team could: (1) generate diverse context-based QA data using an LLM synthesizer (following the Instruction Pre-Training recipe the paper uses), (2) train with LLM-in-Sandbox-RL to establish baseline exploration competence, and (3) fine-tune on a small amount of domain-specific agentic data for the target application. The paper's data ablation (Table 7) shows that SWE-specific training achieves the best SWE-bench performance (17.4 for Qwen3-4B), but general sandbox training achieves broad competence across six domains while SWE training generalizes poorly to non-SWE tasks. For organizations that need agents across multiple domains, the general-first approach is more data-efficient than training separate domain-specific agents from scratch.

Verification-grounded scientific and analytical reasoning. For domains where correctness matters more than fluency — mathematical problem-solving, chemical computation, physics simulation — LLM-in-Sandbox provides a path to grounded rather than plausible outputs. The paper's AIME25 results are illustrative: a model like Qwen3-Coder achieves 17.9% accuracy in text-only mode but 42.1% with sandbox access (Table 2) because the sandbox enables actual computation rather than simulated reasoning. For deployment in scientific workflows, this difference is categorical: a system that computes is auditably correct or incorrect; a system that generates is only probabilistically plausible. The paper's design choice to extract answers from /testbed/answer.txt rather than from model-generated text (Section 2.2) cleanly separates the exploration process from the verified output, making the system suitable for pipelines where answers feed into downstream automated processes that require structured, validated inputs.

Interactive document and data exploration interfaces. The long-context results (Table 3) suggest a deployment pattern where users interact with document collections through natural language queries, with the LLM orchestrating programmatic search rather than loading entire documents into context. This is fundamentally more scalable than RAG (retrieval-augmented generation) approaches that chunk documents and embed them for similarity search: LLM-in-Sandbox can execute precise grep patterns, run Python scripts for structured extraction, and iteratively refine searches based on intermediate results — capabilities that fixed retrieval pipelines cannot match. The infrastructure overhead is modest (Table 12: 50 MB idle, 200 MB peak per container), meaning a single server could support hundreds of concurrent document exploration sessions. The open-source Python package makes this deployable today with existing LLM serving infrastructure.