SandMLE: A Multi-Agent Framework that Generates Synthetic Sandbox Environments for Training Machine Learning Engineering Agents

ArXiv: 2604.04872

🎯 Pitch

Training ML engineering agents with on-policy RL was infeasible because verifying a single solution requires training a full model on millions of samples. SandMLE makes it possible by generating diverse, micro-scale synthetic tasks of just 50–200 samples, slashing execution time by 13× while preserving the mathematical complexity needed for agents to learn, leading to massive gains over supervised fine-tuning.

---

1. Executive Summary

This paper introduces SandMLE, a multi-agent framework that generates diverse, verifiable synthetic machine learning engineering (MLE) environments with micro-scale datasets (50–200 training samples) to make trajectory-wise on-policy reinforcement learning (RL) practical for MLE agents. By procedurally extracting structural DNA from seed tasks and constraining execution to lightweight sandboxes, SandMLE reduces per-step execution time by over 13× (from ~196 seconds to ~14 seconds on average) and enables large-scale GRPO training with a dense, milestone-based reward formulation (rewarding progressive performance tiers from format compliance through bronze, silver, and gold medal thresholds rather than only the final score). On the MLE-bench-lite benchmark using Qwen3-8B, 14B, and 30B-A3B models, SandMLE yields relative Any Medal rate improvements of 20.3% to 66.9% over the SFT baseline, with the trained policy generalizing across unseen agentic scaffolds—achieving up to 32.4% better HumanRank score on MLE-Dojo—establishing that synthetic micro-scale environments can serve as effective proxies for real-world MLE tasks only when the reduced data volume preserves the structural complexity and mathematical hidden rules needed for meaningful policy optimization.

2. Context and Motivation

The Core Problem: MLE Verification Is Orders of Magnitude Slower Than SWE Verification

This paper addresses a fundamental bottleneck that has prevented on-policy reinforcement learning from being applied to machine learning engineering (MLE) agents: the prohibitive cost of environment interaction during RL rollouts. To understand why this is a bottleneck, you need to understand the contrast with software engineering (SWE) tasks, where trajectory-wise RL has been successfully deployed.

When an LLM agent works on a SWE task—say, fixing a bug in a code repository—the verification step is cheap. The agent writes code, the environment runs unit tests against it, and those tests execute in seconds. Computational cost is dominated by compilation and test logic, not by the volume of data being processed. This fast feedback loop makes it feasible to run thousands of on-policy rollouts: the agent proposes a fix, tests run, the reward signal arrives quickly, and the policy updates. Frameworks like DeepSWE (Luo et al., 2025) and RLLM (Tan et al., 2025) have successfully applied GRPO in this setting.

Now consider an MLE task. The agent is given a Kaggle-style competition: a dataset, a target metric, and a goal to produce a winning submission. To verify whether the agent's proposed ML pipeline is any good, the environment must:

1. Preprocess the training data
2. Train a model (potentially a deep neural network for many epochs)
3. Run inference on a held-out test set
4. Compute the evaluation metric

Each of these steps operates over the full competition dataset, which in MLE-bench contains an average of roughly 4.09 million samples per task (Section 5.2). The paper reports that a single code execution on standard MLE-bench problems averages nearly 200 seconds (specifically 196.17 seconds, as shown in Figure 6). This is not a one-time cost: in trajectory-wise GRPO, each rollout step invokes the environment, and each trajectory can span up to $T_{\text{max}}$ steps (set to 20 during training). For a single training example, with group size $N = 4$ (the paper's setting), the wall-clock cost scales as $O(N \cdot T_{\text{max}} \cdot c_{\text{exec}})$ — approximately $4 \times 20 \times 196 = 15,680$ seconds (over 4 hours) just for the environment interactions of one training batch. Running GRPO for 100 steps across thousands of tasks becomes completely infeasible.

This is the execution-latency bottleneck that the paper identifies as the root cause preventing on-policy RL for MLE agents (Section 3.1, "The Execution-Latency Bottleneck").

---

Why This Problem Matters

The significance of closing this gap extends beyond MLE-bench performance. There are three reasons this problem is important:

1. MLE tasks represent a frontier for agentic AI that cannot be solved with single-turn reasoning. Unlike math problems or coding exercises where a single model generation can produce the correct answer, MLE tasks are inherently multi-turn and require trial-and-error reasoning: the agent must propose a hypothesis (e.g., "XGBoost with these hyperparameters will work"), write code, observe results (metrics, errors, data characteristics), and iteratively refine its approach. This is the kind of open-ended scientific reasoning that makes MLE a compelling testbed for advancing agent capabilities, but it also means that the environment must be queried repeatedly during both training and evaluation. Any approach that cannot afford to interact with the environment at scale is fundamentally limited in what it can teach the agent.

2. Agent scaffolding alone does not produce transferable reasoning skills. The paper is careful to distinguish between two ways of improving MLE performance (Section 2.1): (a) building more sophisticated agent scaffolds (AIDE, AIRA, ML-Master) that orchestrate the agent's interaction with the environment at inference time, and (b) training the underlying model to be a better MLE reasoner. Prior work has focused heavily on (a), but the paper's own cross-framework evaluation (Tables 2 and 3) reveals that "the same base model can exhibit dramatically different performance depending on the scaffold, underscoring that scaffolding alone does not yield robust, transferable MLE capabilities." A model trained only to follow a specific scaffold's workflow may collapse when deployed in a different framework. In contrast, training the model's intrinsic reasoning through RL with environment feedback should—if successful—produce capabilities that transfer across scaffolds. This is the paper's central bet, and it can only be tested if on-policy RL is made practical.

3. The training-inference paradigm shift. The paper sits within a broader movement in the LLM agent community: the realization that for complex, long-horizon tasks, post-training matters as much as pretraining. SWE agents and web agents have shown that trajectory-wise RL can substantially improve performance (DeepSWE, WebDancer), but MLE has been left behind because of the execution-latency wall. Solving this problem opens MLE as a domain where the same RL-driven capability improvements can be realized, and the paper's FLOPs-matched comparisons (Section 5.3) suggest that RL-trained small models can rival much larger closed-source models, making this a practically impactful efficiency gain.

---

Prior Approaches and Where They Fall Short

The paper identifies three strategies that prior work has used to work around the execution-latency bottleneck, and explains why each is unsatisfactory:

1. Supervised Fine-Tuning (SFT) on expert trajectories. The most common approach is to avoid reinforcement learning entirely: collect multi-turn trajectories from a strong model (e.g., Claude-4.5-Sonnet) on seed tasks, and fine-tune the target model to imitate those trajectories. The paper implements this as the Seed-SFT baseline (Section 5.1): Claude-4.5-Sonnet generates reasoning trajectories for the 60 seed tasks, and the base Qwen3 models are fine-tuned on these interactions.

Why this falls short: Behavioral cloning on expert trajectories does not teach the model why certain decisions are made or how to recover from failures. The paper's results bear this out starkly. In Table 1, Seed-SFT produces essentially zero improvement over the Base model for Qwen3-8B (both at 13.6% Any Medal) and only marginal improvement at 14B (18.2% for both). On MLE-Dojo (Table 3), the SFT models actually regress dramatically when deployed outside the specific scaffold used during data generation: Qwen3-30B-Seed-SFT collapses to a 17.7% Valid Submission rate with the MLE-Agent framework, compared to 71.0% for the Base model. The paper explains this as the SFT model having memorized scaffold-specific behavioral patterns that break when the interaction protocol changes. This is the core limitation of SFT: it teaches what to output, not how to think through the problem, and the learned behaviors are brittle to distribution shift.

Additionally, the paper cites Chu et al. (2025) in the introduction: "SFT memorizes, RL generalizes," positioning the entire work within this comparative framework.

2. Off-policy or asynchronous RL with proxy rewards. A handful of prior works have attempted to apply RL to MLE agents, but resort to approximations that degrade the learning signal:

• Asynchronous GRPO architectures (Cai et al., 2026) hide execution latency by using trajectories generated from "lagging policy states"—the actor generates rollouts using an older version of the policy while the learner updates asynchronously. The problem, as the paper notes in Section 2.2, is that this introduces distribution shift: the trajectories used to compute the policy gradient were not generated by the current policy, violating the on-policy assumption that GRPO and PPO rely on for stable optimization. The paper is explicit: "both strategies are fundamentally off-policy; for instance, asynchronous GRPO relies on trajectories generated from lagging policy states, creating a distribution shift."

• Step-wise RL with offline proxy rewards (Liu et al., 2025b) breaks the trajectory into individual steps and assigns rewards using a learned proxy rather than actual environment execution. This avoids the latency problem but the proxy reward may not accurately reflect the true task outcome, and the step-wise decomposition loses the credit assignment signal across the full trajectory.

Why these fall short: The paper argues that for long-horizon tasks like MLE, the quality of the learning signal is paramount. Off-policy methods introduce noise through stale trajectories; proxy rewards introduce noise through approximation error. Both degrade the policy gradient, potentially leading to unstable training or convergence to suboptimal policies. The paper's position is that strict on-policy training with true environment rewards is the gold standard, and the engineering challenge is to make this computationally feasible rather than to accept degraded alternatives.

3. Inference-time scaffolding improvements. As discussed in Section 2.1, a large body of work—AIDE (Jiang et al., 2025), AIRA (Toledo et al., 2025), ML-Master (Liu et al., 2025a), R&D Agent (Yang et al., 2025d), FM Agent (Li et al., 2025), MLE-Star (Nam et al., 2025)—has focused on designing more sophisticated agent scaffolds that better orchestrate the model's interaction with the environment. These frameworks provide mechanisms for the agent to maintain state across turns, search over code solutions, or leverage test-time compute scaling.

Why this falls short: The gains from scaffolding are contingent on the specific scaffold design and do not improve the underlying model's reasoning. The paper demonstrates this empirically: in Table 2, the Base Qwen3-14B model achieves 27.3% Any Medal with AIDE but only 9.1% with AIRA—a 3× difference driven entirely by scaffold choice. An agent trained only via scaffolding improvements is fragile; change the interaction protocol and it may fail. The paper's goal is to produce a model that is robust across scaffolds, which requires training the model's intrinsic MLE reasoning capabilities rather than optimizing its fit to a particular scaffold.

---

The Root Cause Insight: Dataset Size Is the Dominant Bottleneck

The paper makes a critical observation that none of the prior approaches directly address (Section 1, "A closer analysis reveals a surprisingly simple root cause"):

"Unlike SWE, where execution time is dominated by compilation and test logic, MLE latency is overwhelmingly driven by dataset size."

This is not merely an engineering detail—it is the insight that unlocks the entire approach. In SWE, unit tests run in seconds regardless of how large the codebase is, because tests exercise specific functionality on small, targeted inputs. In MLE, every code execution must process the full training dataset to train a model, and that dataset contains millions of samples. The paper measures this empirically: the original MLE-bench seed tasks average ~4.09 million samples, and code execution averages 196.17 seconds (Figure 6).

The natural response—"just downsample the dataset"—is insufficient for two reasons. First, downsampling existing tasks corrupts the evaluation: the fixed test set no longer matches the downsampled training distribution, and the established leaderboard baselines (medal thresholds) become invalid because they were computed on the full dataset. Second, and more important, downsampling alone does not address the scarcity of diverse training tasks. For robust policy optimization, the RL agent needs exposure to many different MLE scenarios. Simply shrinking 60 seed tasks would at best produce 60 small tasks, which is insufficient for generalization (the paper's synthetic pipeline produces 848 diverse training tasks from those same 60 seeds).

The paper's key conceptual move is to recognize that dataset size and task diversity are independent levers. You can generate new tasks from scratch, with small datasets by construction, while preserving the structural complexity that makes MLE tasks challenging—and this simultaneously solves both the latency and diversity problems.

---

How This Paper Positions Itself Relative to Existing Work

The paper positions itself at the intersection of three research threads and claims a novel synthesis:

Relative to MLE benchmarking (MLE-bench, MLE-Dojo, MLE-Smith): SandMLE is not competing with these benchmarks but rather providing a training substrate that complements them. The benchmarks remain the evaluation targets; SandMLE generates the synthetic training environments. This is a "training curriculum" approach, analogous to how procedural environment generation is used in game-playing RL (e.g., Procgen, OpenAI's domain randomization), but applied to the domain of ML engineering tasks.

Relative to SWE/web agent RL (DeepSWE, WebDancer, Experience Synthesis): The paper borrows the trajectory-wise GRPO training paradigm that has proven effective in these domains but claims to be the first to make it computationally feasible for MLE by addressing the execution-latency bottleneck head-on. The key difference is that SWE/web tasks do not require this kind of environment redesign because their native execution is already fast. The paper is therefore not advancing the RL algorithm itself (GRPO is standard) but rather the infrastructure that enables GRPO to be applied to a previously inaccessible domain.

Relative to prior MLE training approaches (SFT-only, asynchronous RL): The paper's strongest contrast is with off-policy or proxy-reward methods. It argues that these approaches accept a degraded learning signal in exchange for computational feasibility, and that SandMLE's contribution is to remove the need for this tradeoff: by making on-policy training feasible through synthetic micro-scale environments, SandMLE provides "a high-fidelity gradient signal for effective and stable learning" (Section 2.2) without the distribution shift or approximation error of prior methods.

The conceptual framework: The paper reframes the MLE-RL bottleneck as fundamentally an environment design problem, not an algorithm design problem. This is a shift in perspective: prior work asked "how can we make RL work despite slow environments?" (leading to asynchronous training, proxy rewards, or abandoning RL entirely), while SandMLE asks "how can we make the environments fast enough for RL?" This reframing—from algorithm adaptation to environment generation—is the paper's primary intellectual contribution, with the multi-agent pipeline being the concrete instantiation.

What distinguishes SandMLE from simple dataset downsampling: The paper explicitly anticipates the objection that reducing dataset size is the obvious solution. The multi-agent generation pipeline is necessary because: (1) downsampling existing tasks breaks evaluation integrity (test set mismatch, invalid baselines), (2) the number of seed tasks is too small for diverse RL training, and (3) the generated tasks must preserve mathematically rigorous hidden rules that make MLE problems non-trivial—merely shrinking a dataset does not guarantee the problem remains challenging or well-calibrated. The paper validates this calibration empirically in Figure 4, showing that synthetic tasks cleanly separate models of known capability (Claude-4.5-Sonnet dominates, GPT-4o-mini is weakest), confirming that the generated environments retain meaningful difficulty despite their micro-scale.

3. Technical Approach

3.1 Reader Orientation (Approachable Technical Breakdown)

At its core, SandMLE is a training data factory: a multi-agent pipeline that takes a small number of real-world ML competition tasks as seeds and procedurally generates hundreds of diverse, self-contained synthetic ML tasks, each with its own tiny dataset (50–200 samples) and built-in evaluation environment, so that an LLM can practice solving ML engineering problems thousands of times with fast feedback. The system solves the problem that real MLE tasks are too slow to run during reinforcement learning by constructing fast-executing synthetic proxies—not by changing the RL algorithm, but by redesigning the environments themselves so each execution step drops from ~196 seconds to ~14 seconds.

3.2 Big-Picture Architecture (Diagram in Words)

The SandMLE framework has two major stages:

Stage 1 — Synthetic Environment Generation (the "factory"):

• Data Strategist: Extracts the abstract mathematical structure ("Structural DNA") from a seed task, mutates it into a new domain, injects difficulty via noise, and specifies baseline methods and milestone thresholds.
• ML Developer: Writes executable Python code that procedurally generates the micro-scale dataset, implements the hidden mathematical rules, trains baselines to compute milestone thresholds, and produces a sample submission file.
• MLOps Engineer: Constructs a deterministic evaluation script that loads an agent's submission, computes the evaluation metric against hidden ground truth, and outputs the final score plus the milestone thresholds.
• Technical Writer: Compiles all metadata into a comprehensive task description markdown document that serves as the initial prompt for the RL agent.

Stage 2 — Trajectory-Wise GRPO Training:

• The base LLM (Qwen3 family) interacts with the synthetic environments using the ReAct framework over multi-turn rollouts.
• A dense, milestone-based reward function provides granular feedback: format compliance, execution success, and progressive performance tiers (median, bronze, silver, gold) rather than only the final leaderboard score.
• Standard GRPO is applied with loss masking on observation tokens, optimizing the model's multi-turn reasoning for MLE problem-solving.

Information flows: seed tasks → four-agent factory → 848 synthetic training tasks → GRPO rollout loop (agent writes code, environment executes and scores it, reward computed, policy updated) → trained model evaluated on real-world MLE-bench-lite and MLE-Dojo.

3.3 Roadmap for the Deep Dive

• First, the multi-agent synthetic environment generation pipeline (§3.4.1)—the factory that produces training tasks—since without fast environments, RL is infeasible.
• Second, the environment sanity verification mechanism (§3.4.2)—the filter that ensures generated tasks are logically consistent before RL training begins.
• Third, the trajectory-level GRPO training setup (§3.4.3)—how the agent interacts with synthetic environments, why on-policy training is now possible, and the boundary conditions enforced.
• Fourth, the dense milestone-based reward formulation (§3.4.4)—the critical design choice that makes credit assignment work in long-horizon MLE tasks, including the exact weight allocations and the rationale behind them.
• Fifth, the selective masking strategy for backpropagation (§3.4.5)—a technical detail that prevents the model from learning to predict environment observations rather than actions.
• Sixth, a summary table of all design choices and their justifications—consolidating the "why" behind each architecture decision.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems and training methodology paper whose core idea is that the prohibitive execution latency of real-world MLE tasks can be circumvented by procedurally generating synthetic micro-scale environments that preserve structural complexity while drastically reducing dataset sizes, enabling on-policy trajectory-wise RL for the first time in this domain.

---

3.4.1 The Multi-Agent Synthetic Environment Generation Pipeline

The synthetic environment generation is a four-stage pipeline where each stage is performed by a specialized LLM agent with a specific role and output responsibility. The paper describes this as the "Agentic MLE Environment Factory" (Figure 2) and provides the full prompt templates for each agent in Appendix D.2 (Tables 6–9).

Data Strategist: Seed Task Amplification and Specification

The Data Strategist agent performs four sequential operations to transform a real-world seed task (from MLE-bench) into an abstract specification for a new synthetic task.

Step 1 — Structural DNA Extraction. The agent is instructed to "IGNORE the domain context (the 'Story')" of the seed task and instead "convert these into Abstract Mathematical Concepts." For example, "Age" becomes "Continuous variable, positive, right-skewed"; "Cabin Number" becomes "High-cardinality categorical, high missing rate"; "Survived" becomes "Binary Target, Class Imbalance 60/40." The output is a JSON object called the Task DNA that captures:

• modality: one of Tabular, Image, Text, Audio, or Graph
• task_type: one of Classification, Regression, Segmentation, or Object Detection
• dataset_stats: sample_count (integer) and is_imbalanced (boolean)
• target_info: type (Label, BoundingBox, Mask, or Text), cardinality (integer), and distribution (Balanced or Long-tail)
• Additional modality-specific fields (e.g., image resolution, text length characteristics, feature cardinalities for tabular data)

The purpose of stripping semantic context is to enable domain transfer: the structural pattern of an animal species classification task can be re-applied to road damage detection, document classification, or any other domain that shares the same mathematical skeleton. This is the mechanism that amplifies 60 seed tasks into 848 diverse training tasks.

Step 2 — Domain Attribution. Using the extracted DNA, the agent is prompted to "brainstorm 5 distinct 'Industry Scenarios' that could naturally generate data with this exact structure." For each scenario, the agent must provide a domain (e.g., "Legal Tech"), a scenario (e.g., "Contract Comparison"), and a justification that explicitly connects the abstract features to the new domain. The constraint is that "the scenarios must strictly justify the features"—if the DNA specifies "Long Text," the domain must involve documents, logs, or dialogue; if it specifies "Images," the domain must involve visual sensors. This ensures the generated tasks maintain ecological validity: the feature types make sense in the assigned domain.

Step 3 — Adversarial Mutation. To ensure the generated tasks are non-trivial and require genuine ML reasoning rather than simple pattern matching, the agent injects difficulty via a noise configuration $\epsilon$. The agent is prompted to "generate a 'Mutation Config' that increases the difficulty of the task for an AI Agent" by selecting three mutations appropriate to the modality. Example mutations for image tasks include salt-and-pepper noise (amount: 0.05), rotation (degrees: 45), and class imbalance (ratio: 1:10). These mutations are designed to be systematic and recoverable—they degrade the signal in principled ways that a competent ML pipeline can address through appropriate preprocessing, but that naive pipelines will fail on.

Step 4 — Concrete Specification Compilation. The agent merges the structural DNA, chosen domain, and noise configuration into a New DNA specification (referred to in the prompts as the Concrete Task Specification). This includes:

• task_name: a creative title for the new task
• domain_context: the industry setting
• final_dimensions: n_samples (constrained to a random integer between 50 and 200—this is the micro-scale constraint) and n_features (from the original DNA)
• feature_mapping: a mapping from each abstract feature name (e.g., feat_0) to a domain-specific name (e.g., systolic_blood_pressure) with an explicit generation_logic specifying the statistical distribution from which that feature is sampled
• hidden_rule_logic: the mathematical formula $l = f(\mathbf{z}) + \epsilon$ that deterministically maps features $\mathbf{z}$ to labels $l$, incorporating the noise configuration. The prompt explicitly instructs: "please do not be so simple, make it complex and realistic"—this is the mechanism that preserves structural complexity despite the reduced data scale
• evaluation_specs: the evaluation metric (same as the original DNA), plus a thresholds_logic that defines how Kaggle-style medal thresholds are derived. The example in the prompt: "Gold = 0.90 (due to 10% noise). Silver = Random Forest baseline. Bronze = Linear Regression baseline. Median = Majority Class baseline."

The final output is a complete, domain-specific, mathematically rigorous task specification that constrains the total dataset to 50–200 samples—a 20,000× to 80,000× reduction from the average 4.09 million samples in the original MLE-bench seed tasks.

ML Developer: Synthetic Data Generation

The ML Developer agent receives the New DNA specification and writes a single self-contained Python script called generate_task_env.py that programmatically creates all the assets for the environment.

Operation 1 — Dataset Synthesis. The script procedurally generates the micro-scale dataset $\mathcal{Z}$ following the feature generation logic specified in the New DNA. For tabular data, it builds pandas DataFrames and saves them as train.csv and test.csv (test set is approximately 20% the size of the training set). For image data, it creates an images/ directory, draws synthetic images using Pillow or OpenCV following the blueprint (e.g., generating road textures with Perlin noise and overlaying damage types), saves them as .png files, and creates train.csv and test.csv mapping filenames to labels (test labels are blank but the column is present to maintain schema consistency). For audio data, it creates an audio/ directory, synthesizes waveforms using numpy and scipy.io.wavfile, and creates corresponding CSV files. For text data, it either embeds short texts directly in the CSV or saves long documents to a docs/ directory referenced by the CSV.

Operation 2 — Hidden Rule Implementation. The script explicitly implements the mathematical hidden rule $\mathcal{H}: l = f(\mathbf{z}) + \epsilon$ from the New DNA. The paper gives a concrete qualitative example in Appendix C: in an urban road damage detection task, the rule might dictate that a specific edge pixel ratio $f(\mathbf{z})$ combined with injected blur noise $\epsilon$ (applied directionally to simulate vehicle motion) strictly determines the final damage severity label $l$. The temporal blur accumulation is described as: "Images are generated in temporal sequences (1–5 frames). Motion blur is applied via a directional line-kernel, with the blur severity linearly increasing with the frame number to simulate a moving vehicle." This is not random noise—it is structured degradation that an ML pipeline must learn to address (e.g., through deblurring or sequence aggregation).

Operation 3 — Baseline Training and Threshold Computation. The script trains and evaluates the baseline methods specified in the Data Strategist's thresholds logic to empirically compute and lock in the set of progressive milestone thresholds $\mathcal{S}$. For example, it might train a Majority Class predictor, a Linear Regression model, and a Random Forest model on the generated training data, evaluate each on the held-out test set, and record their scores. These scores become the median, bronze, silver, and gold thresholds respectively. The thresholds are saved to threshold.json with exactly the keys: "gold_threshold", "silver_threshold", "bronze_threshold", "median_threshold".

This is a critical design choice: the thresholds are computed once during environment generation and frozen, not recomputed dynamically during RL training. This ensures that the reward signal is deterministic and stable across rollouts—the same submission always receives the same score relative to the same thresholds.

Operation 4 — Schema Artifacts. The script produces a sample_submission.csv containing test-set IDs and random/dummy predictions in the exact format expected from the agent, serving as a strict schema reference. It also produces answer.csv containing the true labels for the test set (kept hidden from the agent during training and used only by the evaluator).

Reliability via Execution-Based Verification. The paper enforces a practical reliability mechanism: "if the script fails, the current trace is automatically returned to the agent for iterative debugging" with up to five generation attempts per task. The paper reports that from an initial pool of 1,200 tasks, after this execution-based filtering, 1,119 tasks successfully passed the data generation stage (Section C.1). This is "execution-based verification" in the context of environment generation: the system does not trust the agent's code on the first attempt but requires it to actually run without errors.

MLOps Engineer: Evaluation Environment Setup

The MLOps Engineer agent writes a second Python script, evaluator.py, that serves as the automated scoring sandbox for RL rollouts. This script is what actually runs when the agent submits a solution during GRPO training.

Requirement 1 — Metric Standardization. The evaluator script hardcodes the evaluation metric $\mathcal{M}$ (e.g., Macro-F1, accuracy, RMSE), the optimization direction (is_lower_better as a boolean), and the set of progressive milestone thresholds $\mathcal{S}$ (loaded from threshold.json). This ensures deterministic scoring: every execution of the evaluator on the same submission produces identical output.

Requirement 2 — Data Alignment and Computation. The script accepts a --submission_path argument (defaulting to sample_submission.csv). It loads both the agent's submission file and the hidden ground truth answer.csv, aligns them by an ID column if present (or by row order with a warning if no ID column exists), computes the specified metric $\mathcal{M}(\hat{\mathbf{p}}, \mathbf{l})$ where $\hat{\mathbf{p}}$ is the agent's predictions and $\mathbf{l}$ is the ground truth, and outputs a JSON object to stdout with keys: "score", "gold_threshold", "silver_threshold", "bronze_threshold", "median_threshold", and "is_lower_better".

Requirement 3 — Graceful Error Handling. On any error, the script prints a JSON error object to stdout (not stderr) and exits gracefully. This is important for the RL training loop: the agent's reward computation can always parse a valid JSON response, even if the submission contained errors, preventing the training process from crashing on malformed agent outputs.

Requirement 4 — Execution-Based Verification. The system automatically tests evaluator.py against the dummy sample_submission.csv to verify it runs without errors. As with the ML Developer, failed scripts are fed back for iterative debugging, with up to three generation attempts. After this stage, the paper reports that 1,106 tasks remained viable from the original 1,119 (Section C.1).

The design intent of separating the MLOps Engineer from the ML Developer is separation of concerns: the data generation script is complex and may embed domain-specific logic, while the evaluator must be a clean, robust, fast-executing scoring function. Having a separate agent write the evaluator reduces the risk that bugs in the generation logic corrupt the evaluation signal.

Technical Writer: Task Description Synthesis

The Technical Writer agent produces the final description.md file that serves as the initial task specification $\mathcal{I}$ (as defined in §3.1 of the paper) for the RL agent.

Step 1 — Contextual Integration. The agent adapts the narrative structure of the original seed task to match the new synthetic domain, updating names and problem contexts. This ensures the task description reads like a coherent, realistic ML competition prompt rather than a disjointed assembly of generated components.

Step 2 — Content Generation. The agent drafts a problem overview with specific data format descriptions and the required evaluation metric. The prompt specifies that this should be a "concise, clear description" maintaining a consistent structure with the seed task's example description.

Step 3 — Submission Formatting. The agent explicitly defines the expected output format using the generated sample_submission.csv schema. This is operationally critical: during RL rollouts, the agent reads this description to understand what columns and format its submission file must have.

Step 4 — File Transparency. The agent catalogs all public data files (e.g., train.csv, test.csv, sample_submission.csv, images/*, audio/*, docs/*) while "strictly omitting hidden answers to prevent leakage"—meaning answer.csv and threshold.json are never mentioned in the task description.

The full prompt templates for all four agents are provided in Appendix D.2 (Tables 6–9), totaling approximately 15 pages of detailed instructions covering edge cases, output formats, and modality-specific requirements.

---

3.4.2 Environment Sanity Verification

Before any synthetic task is admitted to the training curriculum, it must pass a logical consistency check on its evaluation thresholds. The purpose is to catch corrupted or inconsistent environments where the milestone thresholds do not form a valid progression, which would produce a broken reward signal during RL training.

The verification operates on the following inputs: the dummy sample submission's score $s_{\text{sample}}$ (computed by running the evaluator on the random/dummy predictions), and the predefined milestone thresholds $\mathcal{S} = \{s_1, s_2, \ldots, s_k\}$ where $s_1$ represents the most rigorous performance standard (gold) and $s_k$ represents the most basic (median).

Let the boolean indicator $\mathcal{I} \in \{0, 1\}$ denote whether a lower score is better for metric $\mathcal{M}$ (so $\mathcal{I} = 1$ for metrics like RMSE, $\mathcal{I} = 0$ for metrics like accuracy or F1). A synthetic task is retained only if its thresholds satisfy the strict monotonic constraint:

$$\begin{cases} s_1 < s_2 < \cdots < s_k \land s_1 < s_{\text{sample}}, & \text{if } \mathcal{I} = 1 \\ s_1 > s_2 > \cdots > s_k \land s_1 > s_{\text{sample}}, & \text{if } \mathcal{I} = 0 \end{cases}$$

where $s_1 < s_2 < \cdots < s_k$ means each threshold is strictly less than the next (for lower-is-better metrics, the best performance is the smallest score), and $s_1 < s_{\text{sample}}$ means the gold threshold is strictly better than a random baseline—confirming there is actually room for improvement above chance.

What this constraint enforces: The thresholds must form a strictly monotonic ordering in the correct direction for the metric. If $\mathcal{I} = 0$ (higher is better, e.g., accuracy), the gold threshold $s_1$ must be the highest value, silver $s_2$ the next highest, and so on. If this ordering is violated—for example, if the bronze threshold accidentally exceeds the silver threshold—the reward signal becomes inconsistent: an agent improving from bronze to silver would see its reward decrease, which would confuse the policy gradient.

Additionally, the gold threshold must be strictly better than the random sample submission score, ensuring that (a) the environment is not trivial (the random baseline doesn't already achieve gold), and (b) there is a genuine gradient for the agent to climb from random performance to competitive performance.

Outcome: The paper reports that after this sanity check, the final corpus contains 912 valid tasks (from the 1,106 that passed evaluation environment construction). The 848/64 train/validation split is drawn from these verified tasks. The verification is described as ensuring "the RL agent only optimizes against valid, monotonic reward signals"—tasks with corrupted or inconsistent thresholds are "automatically discarded from the training curriculum."

---

3.4.3 Trajectory-Level GRPO: Enabling On-Policy RL for MLE

With the synthetic environments constructed and verified, the paper now applies trajectory-wise GRPO—the same algorithm used in DeepSWE and WebDancer—to train MLE agents.

Agent-Environment Interaction Protocol

During the rollout phase, the LLM agent interacts with a SandMLE environment using the ReAct framework (Yao et al., 2022). Starting from the initial task specification $\mathcal{I}$ (the description.md produced by the Technical Writer), the agent iteratively:

1. Observes the current trajectory history $\mathbf{h}_t = \{\mathcal{I}, \mathcal{T}, a_1, o_1, \ldots, a_{t-1}, o_{t-1}\}$ comprising the task specification, available tool set $\mathcal{T}$, and all prior action-observation pairs.
2. Generates an action $a_t \sim \pi_\theta(\cdot \mid \mathbf{h}_t)$, which typically consists of code to construct or update the ML pipeline (or reasoning about what to try next).
3. The environment $\mathcal{E}$ executes $a_t$ and returns an observation $o_t$, which may include standard output, runtime errors, or intermediate evaluation metrics.
4. The history extends to $\mathbf{h}_{t+1} = \mathbf{h}_t \cup \{a_t, o_t\}$.

This loop continues until a termination condition: the agent emits a final submission action, reaches the maximum trajectory length $T_{\max}$, or encounters a terminal error.

Why On-Policy Training Is Now Possible

The paper's central enabling claim is numerical: the synthetic micro-scale environments reduce per-step execution time from an average of 196.17 seconds on original MLE-bench tasks to 14.31 seconds on SandMLE tasks (Figure 6), a reduction of over 13×. The paper measures this using "code implementations generated by Gemini-2.5-Flash" to control for model-specific coding efficiency.

The practical impact: in trajectory-wise GRPO, for each training example, the agent generates $N = 4$ candidate trajectories (the group size), each spanning up to $T_{\max} = 20$ steps. The total environment interaction cost per training batch scales as $O(N \cdot T_{\max} \cdot c_{\text{exec}})$. With $c_{\text{exec}} = 196$ seconds, this is infeasible; with $c_{\text{exec}} = 14$ seconds, it becomes practical to run for 100 GRPO steps across 848 training tasks. The paper reports the full training dynamics over 80 steps in Appendix B.1 (Figure 8), validating that convergence is achieved within this budget.

Boundary Conditions

To ensure computational efficiency and prevent infinite generation loops, the paper enforces two strict boundary conditions:

• Per-step execution time limit $\tau_{\max} = 90$ seconds during GRPO rollouts. If the agent's code execution exceeds this limit, the step is terminated. This is described in Section 5.1 under Implementation Details. The 90-second limit is chosen to be generous enough for legitimate ML operations on the micro-scale datasets while preventing runaway processes.
• Maximum trajectory length $T_{\max} = 20$ steps. The agent is limited to 20 interaction turns per task during training. This is described in Section 5.1: "To accommodate the context window capacity of Qwen3, we restrict the maximum interaction limit to $T_{\max} = 20$."

During evaluation on the real MLE-bench-lite, these constraints are relaxed: $\tau_{\max}$ is extended to 4 hours (consistent with standard MLE-bench evaluation protocols where agents can take up to 24 hours), and the generation temperature is set to 0.0 for deterministic outputs. The training constraints are intentionally tight to make RL feasible; the evaluation constraints match the benchmark's standard protocol to ensure fair comparison.

---

3.4.4 Dense Milestone-Based Reward Formulation

This is the most important design choice in the training pipeline. In complex, long-horizon agentic tasks, relying solely on the final performance metric produces a sparse reward signal—the agent receives feedback only at the very end of a potentially 20-step trajectory, and that feedback is a single scalar that collapses all the intermediate decisions into one number. Credit assignment becomes extremely difficult.

The paper addresses this by designing a dense reward function composed of two primary components: a format reward and a milestone-based reward. The final verifiable reward $r$ is:

$r = w_{\text{format}} \cdot r_{\text{format}} + w_{\text{execute}} \cdot \mathbb{I}_{\text{execute}} + \sum_{i=1}^{k} w_{s_i} \cdot \mathbb{I}_{s_i}$

where $r_{\text{format}} \in [0, 1]$ is the format reward (ratio of steps properly using required reasoning tags), $\mathbb{I}_{\text{execute}} \in \{0, 1\}$ is an indicator for successfully generating and formatting a valid output file, and each $\mathbb{I}_{s_i} \in \{0, 1\}$ is an indicator for whether the final quantitative score surpasses the $i$-th predefined milestone threshold $s_i \in \mathcal{S}$, and $k = 4$ milestones are used (median, bronze, silver, gold).

What it computes: The function produces a scalar reward between 0 and 1 (since the weights sum to 1.0) by adding weighted contributions from each achievement. The format reward gives partial credit for using the correct reasoning structure; the execution indicator rewards even producing a valid submission file (regardless of quality); and each milestone indicator fires if the agent's submission score meets or exceeds that performance tier. An agent that produces a valid submission achieving above-median but below-bronze performance receives $w_{\text{format}} \cdot r_{\text{format}} + w_{\text{execute}} + w_{s_{\text{median}}}$.

Why this form: The progressive structure addresses the credit assignment problem by scaffolding the learning signal. A sparse reward (only the gold tier) would mean the agent receives zero reward on almost all early trajectories, providing no gradient for improvement. The milestone-based reward decomposes the learning problem into subgoals: first learn to produce valid submissions (rewarded by $\mathbb{I}_{\text{execute}}$), then learn to beat the median (rewarded by $w_{s_{\text{median}}}$), then progress through bronze, silver, and gold. The agent can improve incrementally because it receives partial credit for partial progress.

Specific Weight Assignments

The paper assigns the following weights (Section 5.1, Implementation Details):

• $w_{\text{format}} = 0.1$
• $w_{\text{execute}} = 0.3$
• $w_{s_{\text{median}}} = 0.1$ (median threshold, the most basic performance standard)
• $w_{s_{\text{bronze}}} = 0.2$ (bronze threshold)
• $w_{s_{\text{silver}}} = 0.2$ (silver threshold)
• $w_{s_{\text{gold}}} = 0.1$ (gold threshold, the most rigorous standard)

Sum: $0.1 + 0.3 + 0.1 + 0.2 + 0.2 + 0.1 = 1.0$.

Why these weights: The execution reward (0.3) is the single largest component, reflecting that producing a valid, executable submission is the primary barrier for early-stage training. The silver and bronze milestones are weighted higher (0.2 each) than gold (0.1) because they represent the "meat" of the performance distribution—achieving bronze or silver is a meaningful accomplishment that should be incentivized, while gold is rare and placing too much weight on it would make the reward sparse. The format reward (0.1) provides a small but consistent signal for structural compliance without dominating the objective.

The weights are described as "weighting the higher-tier milestones more heavily to incentivize competitive performance," but this refers specifically to the relative weights of performance milestones: median (0.1) < bronze (0.2) = silver (0.2) > gold (0.1). Gold is lower than silver/bronze because it is achieved too rarely to serve as a reliable training signal, and placing high weight on it would effectively make the reward sparse again.

The Sparse Reward Baseline (Ablation)

To validate the necessity of dense rewards, the paper designs a Sparse Reward baseline for ablation (Section 6.2):

$r = 0.1 \cdot r_{\text{format}} + 0.9 \cdot \mathbb{I}_{s_{\text{gold}}}$

Here, the only performance feedback is whether the agent achieves the gold standard. No credit is given for valid execution, beating the median, or achieving bronze/silver.

The empirical comparison (Table 4): Under the sparse reward, the Qwen3-30B model's Any Medal rate collapses from 27.3% (dense reward) to 13.6% (sparse reward)—the same as the untrained base model. The 30B valid submission rate drops from 100.0% to 86.4%. At the 8B and 14B scales, the sparse reward fails to lift performance beyond the SFT baseline. The paper concludes: "when deprived of intermediate stepping stones, the Qwen3-30B model's Any Medal rate collapses," confirming that "a hierarchical reward landscape—which independently validates format, execution, and progressive performance tiers—is essential for stabilizing policy optimization."

Alignment with SandMLE Architecture

The milestone thresholds $\mathcal{S}$ are generated by the Data Strategist and empirically computed by the ML Developer during environment construction (§4.1, §4.2). Each synthetic task has its own set of thresholds, meaning the reward function is environment-specific: what counts as "bronze" in one task may require a very different absolute score than in another task. This per-task calibration ensures the reward signal is meaningful relative to the difficulty of each environment, rather than applying a one-size-fits-all standard.

---

3.4.5 Selective Masking for Backpropagation

A technical detail critical to correctly applying trajectory-wise GRPO in agentic settings is loss masking. The GRPO objective computes the policy gradient only over the tokens that the model generated, not over tokens that came from the environment or from the prompt.

The paper specifies (Section 4.3): "we apply strict loss masking during backpropagation to calculate the policy gradient exclusively on the agent's generated reasoning and action tokens, and we entirely mask trajectories where code execution exceeds the defined time limit, thereby preventing the incorrect optimization of static environmental observations and prompts."

Why this matters: In a multi-turn trajectory, the sequence alternates between model-generated actions and environment-generated observations. If loss masking were not applied, the model would receive gradients on the observation tokens as well, which would encourage it to "predict" the environment output—an impossible and counterproductive objective. By masking observation tokens, the policy gradient flows only through the model's own generations, optimizing the decision-making process (what code to write, what reasoning to produce) rather than attempting to model the environment.

Trajectory-level masking: Trajectories where code execution times out are entirely excluded from the gradient computation. This prevents the model from learning from failed trajectories where the timeout interrupted the intended action sequence, which would introduce noise into the policy gradient. Including partial trajectories of failed rollouts could teach the model to produce code that times out, since those trajectories might still achieve some partial reward.

This masking strategy is described as "following previous work for trajectory-level GRPO" (citing Luo et al., 2025; Wu et al., 2025a), and is a standard practice in agentic RL, not a novel contribution of this paper.

---

3.4.6 Summary of Design Choices and Their Justifications

4. Key Insights and Innovations

Innovation 1: Reframing the MLE-RL Bottleneck as an Environment Design Problem, Not an Algorithm Design Problem

The paper's most fundamental intellectual contribution is a diagnostic reframing of why on-policy RL has failed to reach MLE agents. Before SandMLE, the dominant response to slow MLE execution was to adapt the training algorithm: make it asynchronous (Cai et al., 2026), replace environment rewards with proxy rewards (Liu et al., 2025b), or abandon RL entirely in favor of SFT on expert trajectories. Each of these strategies accepts a degraded learning signal—distribution shift, approximation error, or lack of exploration—in exchange for computational feasibility. The implicit assumption across all prior work was that the environment is fixed and the algorithm must accommodate it.

SandMLE rejects this assumption entirely. The paper's diagnostic move is to ask: "What if the environments are the thing we should change, not the algorithm?" This is the shift from "how do we make RL work despite slow environments?" to "how do we make environments fast enough for RL?" It seems obvious in retrospect—the paper identifies dataset size as the root cause of latency (Section 1), and the natural solution is to reduce dataset size—but prior work treated MLE benchmarks as immutable. The insight that environments can be procedurally generated at micro-scale while preserving structural complexity transforms MLE-RL from an algorithm-adaptation challenge into an environment-generation challenge.

Why this is more than an engineering convenience. This reframing has consequences beyond enabling GRPO. It decouples two concerns that were previously conflated: the quality of the RL learning signal (on-policy, true environment reward, no distribution shift) and the computational cost of environment interaction. Prior approaches accepted a tradeoff between them—you could have a good signal (on-policy, true reward) or feasible computation, but not both. SandMLE dissolves this tradeoff by making on-policy training with true environment rewards computationally practical, which means the field no longer needs to compromise on learning signal quality for MLE tasks. This is a conceptual contribution that extends beyond the specific pipeline the paper builds.

Evidence for the shift. The paper doesn't propose a new RL algorithm; GRPO is standard. Every component of the training pipeline—trajectory-wise rollouts, ReAct interaction, loss masking—follows established practice from SWE and web agent RL (DeepSWE, WebDancer). The novelty is entirely in the environment generation, and the paper's results validate that this reframing works: the 13× speedup (Figure 6) enables stable GRPO convergence across three model scales (Figure 8), and the trained policies transfer to real MLE benchmarks (Table 1). The ablation in Table 4 further confirms that algorithm design alone (sparse vs. dense rewards) is insufficient without the right environments—even with the same GRPO training, the sparse-reward baseline failed because the environments were unchanged, and gold-level performance was too rare to provide a learning signal.

Comparison to prior MLE training approaches. Cai et al. (2026) and Liu et al. (2025b) both accepted the latency problem as given and designed around it. The paper is explicit that these approaches "sacrifice the exploration and generalization benefits typically achieved through on-policy algorithms" (Section 1). SandMLE's contribution is not that it outperforms these methods (the paper doesn't directly compare against asynchronous GRPO or proxy-reward RL), but that it removes the need for the compromises they make, providing a path to pure on-policy training that was previously inaccessible.

---

Innovation 2: Structural DNA Extraction as a Task-Amplification Mechanism for Curriculum Generation

The second distinctive contribution is the Structural DNA concept—the idea that MLE tasks can be abstracted into a modality- and domain-agnostic mathematical schema, mutated, and re-instantiated in new domains to generate diverse training curricula. This is not simply data augmentation; it is a task generation primitive that amplifies a small number of seed tasks (60) into a large, diverse curriculum (848 synthetic training tasks) while preserving the properties that make MLE problems challenging.

What makes this novel. Prior work on synthetic data generation for ML training (e.g., experience synthesis in Chen et al., 2025) typically generates additional trajectories or examples within the same task distribution. SandMLE generates entirely new tasks—new domains, new feature schemas, new hidden rules, new evaluation metrics. The DNA extraction achieves this by separating structure (the mathematical relationships between features and labels, the statistical properties of the data, the evaluation protocol) from semantics (whether the data represents animals, road damage, or financial transactions). This separation enables systematic domain transfer: the structure of an animal classification task becomes road damage classification, document categorization, or any other domain compatible with the same abstract schema.

The amplification is dramatic: 60 seeds → 848 diverse training tasks, spanning five modalities (image, tabular, text, audio, graph), six application domains (healthcare, retail, manufacturing, IT, transportation, finance), and multiple task formulations (classification, regression, ranking, forecasting). Figure 3 visualizes this diversity, showing that tasks are not clustered in a few domains or modalities but distributed across the full spectrum—and this is all procedurally derived, not manually curated.

Why this matters beyond the immediate paper. The DNA abstraction suggests a more general principle: task distributions for RL curriculum learning can be procedurally generated from seed tasks by extracting and mutating structural schemas. This is conceptually similar to domain randomization in robotic RL (where physics parameters are randomized to train robust policies) but applied to a cognitive domain (ML engineering) rather than a physical one. The paper doesn't claim this generality, but the mechanism—extract structure, mutate domain, inject noise, regenerate data—is applicable to any domain where tasks can be decomposed into structural and semantic components.

Connection to the paper's validation. The sanity check in Section 4.2 and Figure 4 provide indirect validation that the DNA extraction preserves meaningful difficulty. If the generated tasks were trivial or random, they would not cleanly separate models of known capability. The fact that Claude-4.5-Sonnet achieves a 92.9% pairwise win rate over GPT-4o-mini on these synthetic tasks (Figure 4) confirms they capture genuine MLE reasoning difficulty despite the micro-scale datasets.

---

Innovation 3: Milestone-Based Reward Shaping as a Credit Assignment Solution for Long-Horizon Agentic Tasks

The third contribution is the dense, milestone-based reward function and the empirical demonstration that it is essential for stable RL optimization in long-horizon MLE tasks. While reward shaping is a well-established technique in RL, its application here is distinctive for two reasons: the milestones are task-specific, empirically calibrated thresholds generated during environment construction, and the ablation study (Table 4) provides unusually clean evidence for why sparse rewards fail in this domain.

What's distinctive about the milestone design. The milestones are not generic (e.g., "write some code," "run without errors") but are tied to the specific difficulty of each synthetic task. The baseline methods trained during environment generation (e.g., Majority Class, Linear Regression, Random Forest) produce concrete performance thresholds that become the median, bronze, silver, and gold milestones for that specific task. This means an agent must genuinely outperform reasonable baselines to earn milestone rewards—it cannot game the system by producing valid-but-mediocre submissions. The per-task calibration ensures that the reward signal is meaningful: the same absolute metric score might earn gold on an easy task and nothing on a hard one, preventing the agent from learning to target a fixed score independent of task difficulty.

The ablation tells a clean story. The sparse-reward ablation (Table 4) is not merely a "dense rewards are better" result—it reveals a qualitative failure mode of sparse rewards in this setting. With only format + gold reward, the Qwen3-30B model's Any Medal rate collapses from 27.3% to 13.6%, matching the untrained base model. The model never discovers gold-level performance through random exploration, so it receives essentially zero performance reward on almost all trajectories. The policy gradient provides no signal for improvement, and the model regresses to producing valid-but-random submissions (86.4% valid submission vs. 100% with dense rewards). This is not just worse performance—it is complete failure of the optimization process.

The paper's interpretation is that the milestones function as stepping stones that decompose the exploration problem: the agent first learns format compliance (rewarded at 0.1), then learns to produce executable code (rewarded at 0.3), then progressively learns to beat the median, bronze, and silver thresholds (rewarded at 0.1–0.2 each). The gold threshold at 0.1 is intentionally de-weighted because it is so rarely achieved that placing high weight on it would effectively recreate the sparse reward problem. This is a careful calibration of reward weights to the discovery probability of each milestone during exploration, not an arbitrary allocation.

Comparison to prior reward designs for agentic RL. DeepSWE (Luo et al., 2025) and WebDancer (Wu et al., 2025a) used task-completion rewards (did the bug get fixed? was the information found?), which are naturally sparse but work because SWE and web search tasks have shorter horizons and higher success rates from early exploration. MLE tasks have longer horizons and lower initial success rates, making sparse rewards non-viable. The paper's contribution is recognizing this domain-specific challenge and designing a reward structure that provides intermediate feedback calibrated to MLE task structure—leveraging the fact that MLE tasks have natural performance tiers (Kaggle medal thresholds) that can serve as milestones.

---

Innovation 4: Framework-Agnostic Generalization as Evidence That RL Teaches Reasoning, Not Scaffold-Specific Behavior

The paper's fourth contribution is an empirical finding with significant implications for the agent training literature: RL-trained models generalize across unseen agentic scaffolds, while SFT-trained models do not. This is not a performance claim but a diagnostic result that distinguishes what RL teaches from what SFT teaches.

The evidence. Tables 2 and 3 present cross-scaffold evaluation: models trained with ReAct during RL are tested with AIDE, AIRA, and MLE-Agent scaffolds that they never saw during training. The SandMLE models consistently outperform base models across all scaffold-benchmark combinations. On MLE-Dojo with the MLE-Agent scaffold, Qwen3-30B-SandMLE achieves 83.9% valid submission and a 38.56 HumanRank score, compared to 71.0% and 29.12 for the base model. On MLE-Bench-Lite with AIRA, Qwen3-30B-SandMLE achieves 27.3% Any Medal with 100% valid submission, compared to 18.2% and 86.4% for the base model.

The SFT contrast. Seed-SFT models—trained on Claude-4.5-Sonnet trajectories generated with a specific scaffold—are brittle under scaffold shift. On MLE-Dojo with MLE-Agent, Qwen3-30B-Seed-SFT collapses to a 17.7% valid submission rate (vs. 71.0% for the base model, and 83.9% for SandMLE). On MLE-Dojo with AIDE, Qwen3-14B-Seed-SFT drops from 74.2% (base) to 62.9% valid submission. The SFT models don't just fail to improve—they regress below the base model when deployed outside the scaffold used during data generation.

Why this is a conceptual finding, not just a metric gain. The paper's interpretation is that SFT teaches the model behavioral imitation of a specific expert working within a specific scaffold's interaction protocol. When the scaffold changes—different tool names, different turn structures, different observation formats—the imitated behaviors break because the model never learned the underlying reasoning. RL, by contrast, teaches the model to reason about the problem through direct environment interaction: the model learns that if it writes code that processes data in a certain way, it gets a certain metric score, regardless of the scaffold wrapping that code. This is a specific instantiation of the "SFT memorizes, RL generalizes" thesis (Chu et al., 2025) applied to agentic scaffolds, and it provides empirical grounding for the paper's core motivation: that RL produces transferable reasoning capabilities while SFT produces scaffold-locked behaviors.

The finding also validates the choice to train with the simple ReAct scaffold rather than a more sophisticated one (AIDE, AIRA). A more complex scaffold would produce higher training performance but risk coupling the model's learned behaviors to scaffold-specific features. ReAct's simplicity forces the model to learn generalizable problem-solving strategies, and the cross-scaffold results confirm this bet paid off.

Significance for the field. This finding challenges the dominant paradigm in MLE agent research, which has focused heavily on building better scaffolds (AIDE, AIRA, ML-Master, R&D Agent, FM Agent). The paper's results suggest that scaffold improvements provide brittle gains—they boost performance within the scaffold but don't transfer—while training improvements provide robust gains that persist across scaffolds. This reframes the research priority: rather than optimizing scaffold design, invest in training better models that bring their reasoning capabilities to whatever scaffold they're deployed in.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The training corpus is seeded from 60 questions spanning the Medium, Hard, and Dev splits of MLE-bench (Chan et al., 2024), which are run through the SandMLE multi-agent generation pipeline to produce a final corpus of 848 synthetic training tasks and 64 held-out synthetic validation tasks. For evaluation, the paper uses two separated benchmarks: MLE-Bench-Lite (22 unseen questions from the Easy split of MLE-bench) and MLE-Dojo (Qiang et al., 2025a), a curated collection of 62 additional Kaggle-derived tasks.

• Base model(s). All experiments use models from the Qwen3 family (Yang et al., 2025a): Qwen3-8B, Qwen3-14B, and Qwen3-30B-A3B-2507 (a 30B-parameter mixture-of-experts model with 3B active parameters). The paper also reports performance of Claude-4.5-Sonnet (Anthropic, 2025), DeepSeek-V3.1 (Liu et al., 2024), and Gemini-2.5-Flash (Huang & Yang, 2025) as reference points from models that are orders of magnitude larger, providing an upper bound on what is achievable. The choice of Qwen3 at three scales enables analysis of how RL training benefits scale with model capacity.

• Metrics. For MLE-Bench-Lite, performance is measured using a hierarchy of Kaggle-style success tiers: Valid Submission (percentage of tasks producing a properly formatted, parseable submission.csv), Above Median (percentage beating the 50th percentile of the human leaderboard), Bronze, Silver, Gold, and Any Medal (the union of Bronze, Silver, and Gold, serving as the primary metric). For MLE-Dojo, the paper reports Valid Submission rate and the HumanRank Score, which normalizes agent performance against human participants by computing s = 1 - p/N where p is the agent's rank among N total submissions, averaged across public and private leaderboards. More detailed definitions, including how these tiers instantiate the abstract milestone set S, are provided in Appendix A.1.

• Baselines. The paper defines four model variants per scale:

  • Base: the off-the-shelf Qwen3 model with no additional training.
  • Seed-SFT: the Base model fine-tuned via standard supervised fine-tuning on multi-turn reasoning trajectories generated by Claude-4.5-Sonnet for the 60 seed questions used in the synthetic generation pipeline. This baseline is designed to "disentangle the benefit of high-quality seed data from RL" (Section 5.1).
  • SandMLE: the Base model fine-tuned via trajectory-wise GRPO on the 848 synthetic training tasks.
  • SFT-SandMLE: the same GRPO training but initialized from the Seed-SFT checkpoint, allowing assessment of whether SFT and RL provide complementary benefits.

  For the sparse reward ablation (Section 6.2), a Sparse Reward variant is also evaluated, trained identically to SandMLE except using r = 0.1 * r_format + 0.9 * I_sgold as the reward function.

• Generation budget / compute accounting. For GRPO training, the paper uses a group size of N = 4 candidate trajectories per input, each trajectory spanning up to T_max = 20 interaction turns with a per-step execution time limit of τ_max = 90 seconds per task on a single NVIDIA H200 GPU. Training runs for 100 steps with a learning rate of 1 × 10^-6, batch size 16, generation temperature 1.0 during rollouts, and a GRPO clipping ratio of 0.28 with KL divergence penalties disabled (full hyperparameters in Appendix A.2). During evaluation, the generation temperature is set to 0.0 for deterministic outputs, and τ_max is extended to 4 hours per task to match standard MLE-bench evaluation protocols. For baseline comparisons (AIDE, AIRA, MLE-Agent), the paper adheres strictly to the configurations from the original works, with a 24-hour wall-clock cap per job.

• Cross-validation / statistical protocol. The paper does not employ cross-validation for strategy selection (unlike the reference example paper, which uses two-fold CV within difficulty bins). Instead, the synthetic tasks are generated once from the 60 seeds, filtered through execution-based verification and the sanity check (Section 4.2), producing a fixed corpus of 912 valid tasks split into 848 training and 64 validation tasks. Model selection is based on validation reward (tracked over 80 GRPO steps, shown in Appendix B.1, Figure 8), and final evaluation is conducted on the entirely separate MLE-Bench-Lite and MLE-Dojo benchmarks, which share no tasks with the training or validation sets. This is a hold-out evaluation design rather than cross-validation, but the separation between synthetic training tasks and real evaluation benchmarks provides a clean test of generalization.

Main Quantitative Results

Comparison with Baselines and Model Scaling (Table 1)

The headline result appears in Table 1, which reports performance on MLE-Bench-Lite using the ReAct framework across all model scales and training variants. The paper claims relative Any Medal rate improvements of 20.3% to 66.9% over the SFT baseline, but these percentages require careful reading. The relative improvement is computed as (SandMLE - Seed-SFT) / Seed-SFT: for Qwen3-8B, (22.7 - 13.6) / 13.6 ≈ 66.9%; for 14B, (22.7 - 18.2) / 18.2 ≈ 24.7%; for 30B, (27.3 - 13.6) / 13.6 ≈ 100.7%. This metric highlights the gap between SFT and RL, but the absolute gains are more modest: SandMLE adds 9.1 percentage points of Any Medal rate at 8B and 14B, and 13.7 points at 30B, over the Base models.

Comparing SandMLE to Base: SandMLE achieves significant absolute gains at every scale: 22.7% vs. 13.6% at 8B (+9.1 points), 22.7% vs. 18.2% at 14B (+4.5 points), and 27.3% vs. 13.6% at 30B (+13.7 points). The 30B result is notable because the Base model's Any Medal rate (13.6%) is identical to the 8B Base model—raw scaling alone provides no benefit on this metric—while SandMLE training doubles it to 27.3%.

Comparing to Seed-SFT: Seed-SFT yields essentially zero improvement over Base at 8B (both at 13.6%) and marginal improvement at 14B (18.2% for both). At 30B, Seed-SFT improves to 22.7%, but SandMLE reaches 27.3%. The paper interprets this as evidence that "behavioral cloning on expert trajectories does not transfer the iterative problem-solving behavior that trajectory-wise RL acquires through direct environment interaction" (Section 5.3). The SFT models show improvement in Valid Submission rate (72.7% vs. 68.2% at 8B, 77.3% vs. 68.2% at 30B), suggesting they learn format compliance and pipeline structure but not the higher-order reasoning needed to produce competitive submissions.

Comparing to large closed-source models: Qwen3-8B-SandMLE matches the 22.7% Any Medal rate of DeepSeek-V3.1 and Gemini-2.5-Flash. Qwen3-14B-SFT-SandMLE reaches 27.3%, closing the gap with Claude-4.5-Sonnet at 31.8%. The 14B SFT-SandMLE variant also achieves 95.5% Valid Submission (matching Claude-4.5-Sonnet's 95.5%) and 22.7% Gold (approaching Claude's 27.3%). These results suggest that RL-trained open-weight models at moderate scale can rival or approach much larger proprietary systems, though the gap on Gold medals (a stricter metric) remains substantial.

Scaling behavior within the SandMLE family: Valid Submission for pure SandMLE increases monotonically from 63.6% (8B) to 77.3% (14B) to 100% (30B), while Base models remain flat at 68–73%. Above Median rate scales from 27.3% (8B) to 36.4% (30B) for SandMLE, compared to a static 18.2% for all Base models. The paper notes that "at smaller scales, the RL policy explores aggressive strategies that do not always produce valid outputs, and that sufficient model capacity resolves this tension between exploration and output reliability" (Section 5.3). The 30B model internalizes both formatting discipline and reasoning capability through RL alone, eliminating the need for SFT initialization (100% Valid Submission for pure SandMLE vs. 90.9% for SFT-SandMLE, an inversion of the pattern seen at smaller scales).

SFT + RL complementarity: Initializing GRPO from the Seed-SFT checkpoint (SFT-SandMLE) consistently improves Valid Submission without sacrificing Any Medal rate. At 8B, SFT-SandMLE raises Valid Submission from 63.6% to 90.9% while maintaining 22.7% Any Medal. At 14B, this variant achieves the strongest overall profile: 95.5% Valid Submission and 27.3% Any Medal. The paper interprets this as SFT and RL contributing along different axes—"format compliance and pipeline construction from SFT, higher-order reasoning from RL" (Section 5.3)—and notes that the reliance on SFT initialization diminishes with scale, as the 30B pure SandMLE achieves perfect Valid Submission without SFT.

Test-Time Scaling Behavior (Figure 7)

The paper investigates whether SandMLE-trained models sustain improvement when given more interaction turns at inference time (Section 5.4, Figure 7). Using Qwen3-30B-SFT-SandMLE with the ReAct framework, the maximum allowed turns T_max is varied while keeping the compute budget fixed at 24 GPU hours per task. To handle context window overflow, the paper employs a dynamic truncation strategy: when the context window limit is reached, older messages containing failed code executions are systematically removed.

Both Any Medal and Above Median rates increase steadily as T_max grows: from 23% Any Medal at 0 turns (essentially greedy single-turn) to 27% at 5 turns, 45% at 20 turns, and peaking at 55% Any Medal and 50% Above Median at 30 turns. However, scaling beyond 30 turns causes a regression—both rates drop to around 32–36% at 40 turns—which the paper attributes to "frequent context window overflows" causing the agent to lose historical context and become "trapped in repetitive loops" (Section 5.4). This finding demonstrates that the RL-trained policy has learned trial-and-error reasoning that scales with additional compute, but that the effective context length of the underlying model becomes the bottleneck—not the reasoning capability itself.

Training Dynamics (Appendix B.1, Figure 8)

The paper tracks training and validation rewards, plus valid submission rates, over 80 GRPO steps for all three model scales. All models show clear upward trends in reward signals without signs of severe overfitting (validation rewards stabilize rather than diverge). The scaling effect is most visible in Valid Submission rates: the 8B model fluctuates substantially between 0.1 and 0.8; the 14B model reaches 1.0 only intermittently; the 30B model "rapidly climbs to near-perfect submission rates and sustains them consistently in the later stages" (Appendix B.1). Training rewards follow the expected ordering: 30B peaks near 0.67, 14B near 0.62, 8B near 0.60. This convergence across all scales validates that "the synthetic tasks provide a sufficiently diverse and well-calibrated training signal" (Section 5.5).

Framework Generalization Results (Tables 2 and 3)

The paper evaluates whether SandMLE-trained policies transfer to agent scaffolds not used during training (ReAct was the training scaffold). Table 2 reports MLE-Bench-Lite results with AIDE and AIRA for the 14B and 30B models. Table 3 reports MLE-Dojo results with MLE-Agent and AIDE.

On MLE-Bench-Lite (Table 2): Qwen3-14B-SandMLE with AIDE achieves 31.8% Any Medal, outperforming both Base (27.3%) and Seed-SFT (18.2%). With AIRA, SandMLE achieves 22.7% Any Medal vs. 9.1% (Base) and 13.6% (Seed-SFT). Qwen3-30B-SandMLE with AIRA achieves 27.3% Any Medal vs. 18.2% (Base) and 13.6% (Seed-SFT). A notable negative result: the 30B Seed-SFT model actually underperforms Base under AIDE (13.6% vs. 13.6%, tied) and AIRA (13.6% vs. 18.2%, a regression), demonstrating SFT's brittleness under scaffold shift.

On MLE-Dojo (Table 3): The generalization results are starker. Qwen3-30B-SandMLE with MLE-Agent achieves 83.9% Valid Submission and a 38.56 HumanRank score, dramatically outperforming Base (71.0%, 29.12) and Seed-SFT (17.7%, 7.34). The Seed-SFT collapse from 71.0% to 17.7% Valid Submission is the most dramatic evidence of SFT's scaffold-locking: the model fine-tuned on Claude trajectories generated with one scaffold fails almost completely when deployed in a different one. With AIDE, SandMLE achieves 77.4% Valid Submission and 28.72 HumanRank vs. Base (59.7%, 25.99) and Seed-SFT (66.1%, 28.14). At 14B with MLE-Agent, SandMLE achieves 40.3% Valid Submission and 12.55 HumanRank vs. Base (27.4%, 9.62) and Seed-SFT (3.2%, 0.65)—the Seed-SFT model nearly completely collapses.

The paper does not report MLE-Dojo results for SFT-SandMLE variants, only pure SandMLE and Seed-SFT, so the complementarity of SFT+RL is not tested in this generalization setting.

Environment Quality Results (Figures 4, 5, 6)

The paper provides three environmental statistics to validate the quality of the synthetic training corpus:

Task diversity (Figure 3): The 848 training tasks span five application domains (healthcare 25.0%, retail 18.2%, manufacturing 14.3%, IT 13.2%, other 29.3%), five data modalities (image 48.7%, tabular 24.8%, text 10.4%, multi-modal 10.3%, graph 3.5%, audio 2.3%), and multiple task formulations (classification 56.2%, regression 14.5%, ranking 2.9%, other 26.4%). The paper emphasizes that this diversity is procedurally derived from only 60 seed tasks, demonstrating the amplification capacity of the Data Strategist agent.

Task difficulty calibration (Figure 4): On a random sample of 64 synthetic tasks, pairwise comparisons of four models with known capability differences reproduce the expected ordering: Claude-4.5-Sonnet achieves a 92.9% pairwise win rate, followed by DeepSeek-V3 (39.9%), Gemini-2.5-Flash (35.6%), and GPT-4o-mini (25.5%). The fact that synthetic tasks cleanly separate models of known capability confirms that "they capture meaningful MLE task difficulty and are sufficiently challenging to serve as a reliable training signal" (Section 5.2).

Dataset scale and latency reduction (Figures 5, 6): The per-task dataset sizes cluster in the 120–150 sample range, compared to an average of ~4.09 million samples in the original MLE-bench seed tasks. This micro-scale constraint reduces average code execution time from 196.17 seconds (original MLE-bench tasks) to 14.31 seconds (SandMLE tasks), a reduction of over 13×. The latency measurement uses code implementations generated by Gemini-2.5-Flash to control for model-specific coding efficiency.

Ablation Studies and Robustness Checks

• Milestone-based vs. sparse reward (Table 4, Section 6.2): The paper ablates the dense reward function by training variants with a sparse reward that provides feedback only on format compliance and the gold threshold: r = 0.1 * r_format + 0.9 * I_sgold. This is the most important ablation in the paper and yields the strongest negative result. At 30B, sparse reward causes Any Medal rate to collapse from 27.3% (dense) to 13.6%—matching the untrained Base model. Valid Submission drops from 100.0% to 86.4%, and Above Median halves from 36.4% to 18.2%. At 8B and 14B, sparse reward fails to lift models beyond the SFT baseline (13.6% and 18.2% Any Medal, respectively). The paper interprets this as "achieving a gold standard purely through initial exploration is exceedingly rare" in long-horizon MLE tasks, and without intermediate milestones, "the policy gradient provides no signal for improvement." This ablation directly validates the necessity of the milestone-based reward design, which is the paper's primary training innovation.

• SFT vs. RL comparison (Table 1, Table 3): While not framed as a formal ablation, the Seed-SFT baseline serves as an ablation of the RL training component: it uses the same 60 seed tasks as the synthetic pipeline's input but applies behavioral cloning on expert trajectories rather than environment interaction. The result is that SFT provides minimal or zero improvement over Base on MLE-Bench-Lite (Table 1) and actively harms performance on MLE-Dojo when the scaffold changes (Table 3: Qwen3-30B-Seed-SFT collapsing to 17.7% Valid Submission vs. 71.0% for Base with MLE-Agent). This establishes that the gains from SandMLE come from RL interaction with environments, not from exposure to the seed task distribution.

• SFT + RL combination (Table 1): The SFT-SandMLE variant ablates whether SFT initialization provides complementary benefits to RL. At all scales, SFT-SandMLE achieves equal or higher Any Medal rate compared to pure SandMLE, with substantially higher Valid Submission at smaller scales (90.9% vs. 63.6% at 8B, 95.5% vs. 77.3% at 14B). The complementarity diminishes with scale: at 30B, pure SandMLE achieves 100% Valid Submission without SFT, and both variants tie at 27.3% Any Medal. This suggests that SFT provides basic operational competence (producing valid code, following format requirements) that RL alone can also learn given sufficient model capacity and training steps.

• Test-time scaling limits (Figure 7): The paper ablates the effect of maximum interaction turns, finding that performance peaks at 30 turns and regresses beyond that due to context window overflow. This is both a robustness check (the model sustains improvement over a range of T_max values) and a negative result identifying the effective context length as the scaling bottleneck. The dynamic truncation strategy partially mitigates context overflow but cannot fully compensate for lost historical information.

• Model scale scaling (Table 1, Appendix B.1): The three model sizes (8B, 14B, 30B) provide an informal scaling study. Key finding: RL benefits increase with scale, but the pattern is not uniform. The 8B to 14B jump provides a 0 percentage point gain in SandMLE Any Medal rate (both at 22.7%), while the 14B to 30B jump provides a 4.6 point gain (to 27.3%). However, Valid Submission scales monotonically (63.6% → 77.3% → 100%), and Above Median scales from 27.3% → 27.3% → 36.4%. The training dynamics (Figure 8) show that larger models start higher and converge to higher reward ceilings, with the 30B model demonstrating substantially more stable Valid Submission throughout training.

Critical Assessment

The paper claims four primary contributions: (1) a multi-agent framework for generating synthetic MLE environments that makes trajectory-wise RL practical, (2) successful application of on-policy GRPO to MLE agents for the first time, (3) significant performance gains over SFT baselines with framework-agnostic generalization, and (4) the necessity of milestone-based dense rewards for stable optimization. Each warrants careful scrutiny against the experimental evidence.

Claim 1: Synthetic environments enable practical on-policy RL for MLE. The paper provides strong evidence for the 13× speedup (Figure 6: 196.17s → 14.31s) and demonstrates that training converges on these environments (Figure 8: all three model scales show stable upward reward trends over 80 GRPO steps). However, the claim that this makes RL "practical" is not fully substantiated by cost reporting. The paper does not report the total wall-clock time or GPU hours required for the 100-step GRPO training runs. We know the environment execution time per step (14.31s), the group size (N=4), and trajectories (T_max=20), but the model inference time is omitted, and the training infrastructure cost (generating 912 synthetic environments) is mentioned (Section C.1) but not quantified in terms of compute resources. The claim of practicality is qualitative rather than quantitative—the paper shows that RL converges, but does not provide the cost comparison to SFT or to off-policy alternatives that would establish "practicality" as a meaningful advantage rather than just feasibility.

Additionally, the environment generation pipeline itself incurs substantial cost: 1,200 initial tasks, up to 5 generation attempts for ML Developer, up to 3 attempts for MLOps Engineer, with execution-based verification at each stage. The paper reports filtration numbers (1,200 → 1,119 → 1,106 → 912) but does not report the total API calls or GPU hours consumed by this pipeline. In a cost-benefit analysis, this generation cost should be amortized over the RL training runs—but if a practitioner wanted to apply SandMLE to a new domain, the upfront cost of generating 800+ synthetic tasks might outweigh the RL training savings.

Claim 2: First successful on-policy trajectory-wise RL for MLE agents. The paper demonstrates that GRPO training on SandMLE environments improves MLE-Bench-Lite performance (Table 1) across all three model scales. The training dynamics (Figure 8) show positive reward trends and stable convergence. This is the first paper to report such results, and the evidence is internally consistent.

However, the paper does not compare against the alternative approaches it critiques in Section 2.2—asynchronous GRPO (Cai et al., 2026) or step-wise RL with proxy rewards (Liu et al., 2025b). The paper's central argument is that on-policy RL provides a "high-fidelity gradient signal" superior to these off-policy alternatives, but this claim is argued theoretically (distribution shift, approximation error) rather than demonstrated empirically. The Sparse Reward ablation (Table 4) shows that reward design matters, but a direct comparison of SandMLE's on-policy GRPO against an asynchronous variant using the same synthetic environments would be the definitive test of whether on-policy training specifically—as opposed to just having fast environments—is the key enabler. Without this comparison, the paper demonstrates that fast environments enable GRPO, but not that on-policy GRPO is better than off-policy alternatives given fast environments.

Claim 3: 20.3% to 66.9% relative improvement in Any Medal rate over SFT baseline. The reported relative improvements are mathematically correct given the numbers in Table 1, but the framing warrants caution. The absolute improvements are more modest: SandMLE adds 9.1 percentage points over Seed-SFT at 8B (13.6% → 22.7%), 4.5 points at 14B (18.2% → 22.7%), and 13.7 points at 30B (13.6% → 27.3%). The large relative percentages are partly an artifact of a weak SFT baseline—Seed-SFT provides near-zero improvement over Base at 8B and 14B, making any improvement from RL look large in relative terms. The more informative comparison is SandMLE vs. Base: absolute gains of 9.1, 4.5, and 13.7 points across scales.

The framework generalization results (Tables 2 and 3) are the paper's strongest evidence and the finding most likely to be impactful beyond MLE. The contrast between SandMLE's robust cross-scaffold performance and Seed-SFT's dramatic collapse (30B Seed-SFT dropping to 17.7% Valid Submission on MLE-Dojo with MLE-Agent while SandMLE achieves 83.9%) provides unusually clean evidence that RL teaches transferable reasoning while SFT teaches scaffold-locked behavior. This finding would be even stronger with additional SFT baselines—for instance, SFT on trajectories from multiple scaffolds rather than a single one, or SFT on a larger set of expert trajectories—to distinguish whether the failure is inherent to SFT or specific to the single-scaffold, 60-trajectory SFT setup used here.

Claim 4: Milestone-based dense rewards are essential for stable optimization. The sparse reward ablation (Table 4) provides clean evidence: removing intermediate milestones causes the 30B model's performance to collapse to baseline levels. This is a strong result, but it demonstrates necessity for this specific task and training setup, not universality. The sparse reward function tested (0.1 * format + 0.9 * gold) places 90% of the reward on the most difficult achievement. An alternative sparse reward that includes execution success (e.g., 0.3 * execute + 0.7 * gold) or that uses a continuous score rather than binary thresholds might perform differently. The paper treats "sparse" as equivalent to "only gold" but there are many possible sparse reward designs—the ablation validates one specific sparse reward's failure, not the general principle that any sparse reward would fail.

What's missing. Several experiments would strengthen the paper's claims:

• Direct comparison against off-policy alternatives. Training an asynchronous GRPO variant or a proxy-reward variant on the same SandMLE environments would test whether the on-policy property specifically—rather than just environment speed—is driving the gains. This is the paper's central theoretical argument but it remains empirically untested.
• Scaling the number of training environments. The paper uses 848 synthetic training tasks from 60 seeds. An ablation varying the number of seeds or the amplification factor would reveal how much environment diversity matters for policy generalization—does performance saturate at 200 tasks? 500? Is 848 necessary or would 200 suffice?
• SFT on SandMLE-generated trajectories. The Seed-SFT baseline uses Claude-generated trajectories on the original seed tasks, not on the synthetic tasks. An SFT baseline trained on expert trajectories generated within the SandMLE environments would disentangle whether the RL advantage comes from environment interaction per se, or simply from training on a larger and more diverse set of tasks. The current setup confounds task diversity (848 synthetic tasks vs. 60 seed tasks) with training algorithm (RL vs. SFT).
• Training cost reporting. The paper does not report total GPU hours or wall-clock time for any phase: environment generation, GRPO training, or evaluation. For a paper whose core contribution is making RL "practical," the absence of cost quantification is a significant omission. The 13× speedup is relative to original MLE-bench execution time, not total training cost, and does not account for inference cost, communication overhead, or generation cost.
• Statistical significance. The MLE-Bench-Lite test set has only 22 questions. With Any Medal rates in the 13–27% range, this represents 3–6 questions achieving medals. A difference of 1–2 questions can shift the reported percentages by several points. The paper does not report confidence intervals, standard deviations across runs, or any measure of statistical reliability for these small-sample comparisons.

The genuine strength of the experimental design is the hold-out evaluation structure: training on procedurally generated synthetic tasks, testing on entirely separate real-world benchmarks (MLE-Bench-Lite and MLE-Dojo) that share no tasks with the training set. This provides a clean test of generalization from synthetic to real MLE tasks. The cross-scaffold evaluation (different scaffolds at test time than at training time) is an additional, unusually rigorous test of policy robustness that most agent training papers do not attempt. And the negative result on SFT's cross-scaffold brittleness (Tables 2 and 3) is a finding that the field should take seriously, even if the specific SFT implementation could be strengthened. These design choices make the paper's evidence more credible than if it had only reported results on a held-out split of the synthetic tasks themselves.

6. Limitations and Trade-offs

Cost of Environment Generation Is Unaccounted for in the Headline Efficiency Gains

The assumption or constraint. The paper's central efficiency claim—that SandMLE reduces execution time by over 13×, enabling practical on-policy RL (Section 1, Figure 6)—measures only the per-step code execution time during GRPO rollouts. The cost of generating the 848 synthetic training environments is explicitly excluded from this accounting. The environment generation pipeline involves: 1,200 initial candidate tasks, up to 5 generation attempts per task for the ML Developer agent, up to 3 attempts for the MLOps Engineer agent, and execution-based verification at each stage, followed by the sanity check of Section 4.2. The paper reports filtration counts (1,200 → 1,119 → 1,106 → 912) in Appendix C.1, but it does not report the total GPU hours, API calls, or wall-clock time consumed by this pipeline. The paper acknowledges the cost only implicitly: "During the generation of the training data, we allow a maximum of five attempts per task" and "we permit up to three generation attempts" (Section C.1).

The consequence. For a practitioner wanting to apply SandMLE to a new domain or to a different family of seed tasks, the upfront cost of environment generation constitutes a substantial fixed cost that must be amortized over downstream RL training runs. If a practitioner generates 912 synthetic environments but uses them for only a single RL training run (as the paper does), the total time-to-deployment includes both generation and training. The 13× speedup figure applies only to the RL loop, not to the end-to-end pipeline. For the approach to be cost-effective, the same set of synthetic environments would need to be reused across multiple training runs, model scales, or algorithm variants—but the paper never establishes whether the environments support this reuse without overfitting. Additionally, the generation pipeline depends on strong LLM agents (the Data Strategist, ML Developer, MLOps Engineer, and Technical Writer), and the paper's prompts in Appendix D.2 (Tables 6–9) are highly detailed and task-specific. Adapting these to a new domain would require prompt engineering effort that is not quantified.

What evidence exists in the paper. Figure 6 quantifies the per-step execution speedup (196.17s → 14.31s). Figure 8 demonstrates that RL training converges on the synthetic environments. However, neither figure includes the generation cost in its x-axis or cost accounting. Appendix C.1 lists filtration rates but omits compute costs. The paper makes no attempt to amortize the generation cost or to report a total cost-of-ownership comparison against SFT or off-policy alternatives.

Mitigation status. The paper does not address this limitation. The generation pipeline is presented as a one-time cost that produces a reusable training corpus, but no experiments demonstrate reusability (e.g., training multiple models or algorithm variants on the same corpus and showing that performance does not degrade from environment overfitting). The paper suggests in Section 7 that SandMLE "opens a scalable path toward training MLE agents," but does not discuss the scalability of the generation pipeline itself—whether generating environments for a new domain would require comparable expert effort, or whether the process can be automated further.

---

No Comparison Against Off-Policy or Proxy-Reward Alternatives Given Fast Environments

The assumption or constraint. The paper's primary theoretical critique of prior work is that off-policy RL (asynchronous GRPO, Cai et al., 2026) and proxy-reward RL (step-wise offline rewards, Liu et al., 2025b) "sacrifice the exploration and generalization benefits typically achieved through on-policy algorithms" (Section 1) because they introduce distribution shift or approximation error in the learning signal. The paper positions SandMLE as the solution that removes the need for these compromises by making on-policy training with true environment rewards computationally feasible (Section 2.2): "SandMLE leverages fast-executing synthetic environments to maintain a strictly on-policy training regime… providing a high-fidelity gradient signal for effective and stable learning." However, the paper never empirically tests this claim. No comparison is made between SandMLE's on-policy GRPO and an asynchronous or proxy-reward variant trained on the same SandMLE environments.

The consequence. The paper demonstrates that fast synthetic environments enable on-policy GRPO, but it does not demonstrate that on-policy GRPO is better than the off-policy alternatives—given the same fast environments. It is possible that asynchronous GRPO or proxy-reward RL would also benefit substantially from the 13× speedup, perhaps achieving comparable or even better performance by enabling more environment interactions in the same wall-clock time. If off-policy methods perform similarly, the paper's central argument—that on-policy RL is necessary and that SandMLE uniquely enables it—is weakened. The results would then primarily demonstrate the value of fast environments (regardless of the RL algorithm) rather than the value of on-policy training specifically. Without this comparison, the paper's claim that prior approaches "sacrifice" something important is an untested theoretical assertion, not an empirical finding.

What evidence exists in the paper. The paper provides extensive evidence that SandMLE's on-policy GRPO outperforms SFT baselines (Tables 1, 2, 3) and that sparse rewards fail (Table 4). It does not provide any evidence comparing on-policy vs. off-policy RL. The paper does not implement or evaluate any off-policy variant, even a simple one (e.g., GRPO with a replay buffer of trajectories from previous policy iterations). No related work baseline from Cai et al. (2026) or Liu et al. (2025b) is reproduced or approximated.

Mitigation status. Not addressed. The paper's theoretical argument for on-policy over off-policy stands or falls on the reader's acceptance that distribution shift and approximation error are inherently damaging to policy optimization in MLE tasks. The paper provides no direct evidence that these factors matter in practice for this domain, and the sparse reward ablation (Table 4) addresses reward design, not the on-policy vs. off-policy distinction. Future work explicitly called for by this gap would be comparing GRPO, asynchronous GRPO, and proxy-reward RL on identical SandMLE environments at matched wall-clock time.

---

Training-Only Generalization: No Evidence That Inference-Time Scaffold Improvements Compound with RL Training

The assumption or constraint. The paper argues that scaffolding improvements provide "brittle gains" that do not transfer across frameworks, while RL training provides "robust gains" that persist across scaffolds (Section 2.1, Section 6.1). The cross-scaffold evaluation (Tables 2 and 3) demonstrates that SandMLE-trained models outperform Base models when deployed in AIDE, AIRA, and MLE-Agent—scaffolds they were never trained with. However, the paper never tests whether the best scaffold for a Base model is also the best scaffold for the corresponding SandMLE-trained model, or whether a SandMLE-trained model deployed in a sophisticated scaffold (e.g., AIDE) can outperform a Base model deployed in that same sophisticated scaffold by a margin larger than what simple scaffold improvements alone would provide.

The consequence. The paper's cross-scaffold results (Tables 2 and 3) report performance within each scaffold separately, but they do not address the interaction between training and scaffolding. A practitioner deciding how to allocate resources has two options: (1) invest in RL training using SandMLE, then deploy with a simple scaffold, or (2) skip RL training and deploy the base model with a sophisticated scaffold. The paper shows that RL training helps in both simple (ReAct) and sophisticated (AIDE, AIRA) scaffolds, but it does not quantify whether the marginal benefit of RL training is larger or smaller than the marginal benefit of switching from a simple to a sophisticated scaffold. For example, in Table 2, Qwen3-14B-Base with AIDE achieves 27.3% Any Medal, while Qwen3-14B-SandMLE with ReAct achieves 22.7%. A practitioner with a fixed budget would choose Base+AIDE over SandMLE+ReAct based on these numbers—the scaffold improvement alone (9.1 percentage points) outweighs the training improvement (4.5 points). The paper's framework-agnostic generalization is valuable, but it does not establish that RL training is the most efficient way to improve performance, only that it is a transferable way.

What evidence exists in the paper. Tables 2 and 3 report performance within each scaffold, showing SandMLE consistently outperforms Base. The paper does not compute the marginal benefit of training vs. the marginal benefit of scaffold choice, nor does it report any combination of training and scaffolding that establishes a dominant strategy. The test-time scaling experiment (Figure 7) shows that SandMLE-trained models benefit from additional inference turns, but this is tested only with the ReAct scaffold, not with AIDE or AIRA, so we do not know whether sophisticated scaffolds amplify or diminish the test-time scaling benefits of RL training.

Mitigation status. Not addressed. The paper frames RL training and scaffold design as alternatives (Section 2.1: "the gains are contingent on specific scaffold design rather than the underlying model's reasoning capacity"), but does not explore whether they are complementary or competitive in practice. A natural extension—training with SandMLE and then deploying with AIDE, compared against Base+AIDE and SandMLE+ReAct—would resolve whether the best overall system combines both improvements, but this experiment is absent.

---

MLE-Bench-Lite Test Set Size and Statistical Reliability

The assumption or constraint. The primary evaluation benchmark, MLE-Bench-Lite, consists of only 22 unseen questions (Section 5.1: "MLE-Bench-Lite, comprising 22 unseen questions from the Easy split"). The paper reports Any Medal rates in the 13–27% range for the Qwen3 models, which corresponds to roughly 3–6 questions achieving a medal out of 22 total questions. A difference of a single question (e.g., 5 vs. 6 medals) corresponds to a 4.5 percentage point swing in Any Medal rate—which is larger than the 4.5 point absolute improvement that SandMLE provides over Base at the 14B scale (Table 1). The paper does not report confidence intervals, standard deviations, or any measure of statistical significance for any of the reported metrics.

The consequence. The headline improvements, particularly the relative improvements of 20.3% to 66.9% over SFT and the absolute gains over Base, may be sensitive to the performance on a small number of individual questions. The Any Medal metric is binary per question (medal or not), so improvements come from converting specific questions from non-medal to medal performance. If SandMLE happens to improve on questions that the Base model was already close to solving, the measured improvement could overstate the generalizability of the gain. Conversely, if a few questions happen to be particularly adversarial, the measured improvement could understate it. The paper's conclusions about scaling (e.g., the 30B model jumping from 13.6% to 27.3%) and about closing the gap with closed-source models are all based on point estimates from 22 test questions, with no quantification of uncertainty.

The situation is similar but less acute for MLE-Dojo (Table 3), which has 62 tasks and reports both Valid Submission rate and HumanRank Score (a continuous metric). The HumanRank improvements are more robust to small-sample noise, but the cross-scaffold comparisons on MLE-Dojo still involve a modest number of tasks, and the dramatic SFT collapse (30B-Seed-SFT at 17.7% valid submission vs. 71.0% for Base) could be driven by a subset of tasks where the scaffold mismatch is particularly severe.

What evidence exists in the paper. Table 1 reports point estimates only. No error bars, confidence intervals, or significance tests appear in any table or figure. The paper does not discuss sample size limitations or statistical power. The 22-question test set size is mentioned matter-of-factly in Section 5.1 without commentary on its implications for reliability.

Mitigation status. Not addressed. The paper does not report any form of statistical uncertainty quantification. Standard practices for small test sets—bootstrap confidence intervals, reporting per-question performance breakdowns, or aggregating over multiple random seeds of the training run—are absent. The cross-scaffold evaluation on MLE-Dojo (62 tasks) partially mitigates the concern by providing a second, larger test set, and the consistency of SandMLE's improvements across both benchmarks (MLE-Bench-Lite and MLE-Dojo) and across multiple scaffolds provides informal triangulation. However, formal statistical reliability is not established.

---

The Revision Model Correct-to-Incorrect Reversion Problem (and Its Absence from This Work)

The assumption or constraint. The paper's SandMLE training pipeline applies GRPO directly to a base LLM without any mechanism for the agent to selectively preserve correct work or recognize when a solution is already good. The paper does not incorporate a revision model that conditions on its own previous outputs (unlike the reference example paper, which studies iterative revision chains where the model conditions on its own prior incorrect answers to produce improvements). SandMLE's agent interacts with the environment sequentially over 20 turns, but each turn is independent from the model's perspective—the agent generates code, the environment executes it and returns observations, and the agent generates more code based on the full history. There is no explicit mechanism for the agent to "lock in" partial progress or to recognize that the current solution is already competitive and needs only refinement rather than restarting.

The consequence. Because the agent's trajectory is a flat sequence of environment interactions, the agent can "unlearn" progress: a turn that produces a strong submission (e.g., achieving a silver-medal score) can be followed by a turn that overhauls the code and produces a worse submission, and the only memory of the previous good solution is in the trajectory history. If the context window overflows (as it does in the test-time scaling experiment, Figure 7, beyond 30 turns), the good solution may be evicted and lost. The test-time scaling experiment reveals this vulnerability: the agent's performance improves up to 30 turns but regresses beyond that as "the sheer loss of historical context disrupts the agent's long-horizon memory, often trapping it in repetitive loops" (Section 5.4). The absence of a revision or checkpointing mechanism means the agent's effective performance is bounded by what it can hold in context, and scaling to very long horizons (which MLE tasks might benefit from) is bottlenecked by context length rather than reasoning capability.

What evidence exists in the paper. Figure 7 provides direct evidence for the context-overflow bottleneck: Any Medal rate peaks at 55% at 30 turns and drops to ~36% at 40 turns. The paper explicitly attributes this to "frequent context window overflows" and notes that the dynamic truncation strategy (removing older messages with failed code executions) cannot fully mitigate it. The training dynamics (Appendix B.1, Figure 8) show that at smaller scales (8B), Valid Submission rate fluctuates substantially during training, suggesting the agent does not reliably maintain its best performance throughout a trajectory.

Mitigation status. The paper acknowledges the bottleneck in Section 5.4: "test-time scaling in complex MLE tasks is ultimately bottlenecked by the model's effective context length." It implements a partial mitigation—dynamic truncation of failed code execution messages—but this is described as insufficient. The paper does not propose a revision model, checkpointing mechanism, or any architectural change to the agent that would allow it to preserve good solutions across context overflow. This is flagged as an open problem rather than a solved one. The paper's conclusion (Section 7) does not explicitly call for future work on this front, focusing instead on combining search with revisions more broadly.

---

Hardest MLE Problems Show No Improvement Regardless of Training or Compute Budget

The assumption or constraint. The paper does not explicitly stratify its evaluation results by problem difficulty (unlike the reference example paper, which bins MATH problems into five difficulty quintiles and shows that test-time compute provides zero benefit on the hardest quintile). However, the implicit difficulty selection in the paper's experimental design creates a de facto difficulty filter: the training corpus is seeded from MLE-bench's Medium, Hard, and Dev splits, while evaluation is on MLE-Bench-Lite's Easy split (Section 5.1). This means the models are trained on more difficult problems and evaluated on easier ones—a curriculum design that could mask whether the approach works on genuinely hard MLE tasks.

The consequence. The paper provides no evidence that SandMLE improves performance on hard MLE problems. The training data is drawn from harder splits, but the evaluation is restricted to the Easy split of MLE-Bench-Lite. The MLE-Dojo benchmark (62 tasks) provides additional evaluation breadth but does not provide difficulty annotations, so we cannot assess whether SandMLE's gains concentrate on easier tasks within that benchmark. A practitioner facing genuinely challenging MLE problems—where the base model's pass@1 is near zero—has no evidence from this paper that SandMLE would help. The approach might amplify existing capability (teaching the model to better solve problems it already has some chance of solving) without creating new capability (enabling the model to solve problems fundamentally beyond its reach). This would be consistent with findings from the reference example paper, where test-time compute and RL training improve performance on easy-to-medium problems but provide near-zero benefit on the hardest problem quintile.

What evidence exists in the paper. The paper does not report per-difficulty breakdowns for any evaluation benchmark. The choice of MLE-Bench-Lite (Easy split) for primary evaluation is stated in Section 5.1 without justification for why the Medium or Hard splits were not used. The training data sources (Medium, Hard, Dev splits) are stated, but there is no ablation showing whether training on easier or harder seeds affects the generalization pattern. The paper does not acknowledge this as a limitation or discuss whether the approach is expected to scale to harder problems.

Mitigation status. Not addressed. The paper does not discuss difficulty-dependent performance, does not report per-difficulty results, and does not evaluate on the harder splits of MLE-Bench. This is a significant omission for a training methodology paper, since one of the most important practical questions is whether the approach helps most on easy problems (where base performance is already non-trivial) or on hard problems (where improvement is most valuable). The paper's consistent finding that base models with 13–18% Any Medal rates can be improved to 22–27% could mean that SandMLE helps on the subset of problems where the base model already has some competence, while leaving the majority of problems (where the base model achieves no medal) untouched. Without per-difficulty analysis, this remains an open question.

7. Implications and Future Directions

How This Work Changes the Landscape

SandMLE introduces a methodological reframing rather than a paradigm shift. It does not propose a new RL algorithm, a new model architecture, or a new benchmark. Instead, it identifies a specific bottleneck—MLE execution latency driven by dataset size—and solves it through procedural environment generation, demonstrating that the right infrastructure change makes a previously infeasible training regime practical. The reframing is from "how do we adapt RL algorithms to tolerate slow environments?" to "how do we make environments fast enough that standard RL algorithms work?" This may seem like an engineering detail, but it has conceptual weight: it dissolves the tradeoff between learning signal quality (on-policy, true environment reward) and computational feasibility that prior work accepted as inevitable.

What changes in research priorities. Before SandMLE, the dominant response to slow MLE execution was to compromise on the learning algorithm—asynchronous training, proxy rewards, or abandoning RL for SFT. Each of these accepts a degraded learning signal. SandMLE demonstrates that with carefully constructed synthetic environments, you can have both fast iteration and high-fidelity on-policy gradients. This makes several research directions more attractive:

• Investing in environment design as a first-class research activity. The paper shows that environment quality—not just algorithm sophistication—determines whether RL succeeds. Generating environments with calibrated difficulty (Figure 4: synthetic tasks cleanly separate model capabilities), embedded hidden rules that require genuine ML reasoning (Appendix C), and monotonic reward landscapes (Section 4.2) is non-trivial. The paper's four-agent pipeline is one approach, but the principle—that environments should be procedurally generated, verified for logical consistency, and designed to provide dense, task-calibrated rewards—generalizes beyond MLE.

• On-policy RL for MLE moves from "infeasible" to "engineering challenge." The 13× speedup (Figure 6) makes GRPO training converge across three model scales (Figure 8), but the paper does not report total training cost or compare against off-policy alternatives given the same fast environments. The door is now open for systematic comparisons—on-policy vs. off-policy, GRPO vs. PPO, dense vs. shaped rewards—that were previously impossible because on-policy training on real MLE tasks was too slow to even attempt.

• Scaffold-centric MLE research becomes less compelling as a standalone strategy. The paper's cross-scaffold results (Tables 2 and 3) provide unusually clean evidence that scaffold-specific improvements are brittle: Seed-SFT models collapse when the scaffold changes (30B-Seed-SFT dropping from 71.0% to 17.7% valid submission on MLE-Dojo with MLE-Agent, Table 3), while RL-trained models transfer robustly. This does not mean scaffold design is unimportant—sophisticated scaffolds still improve performance (Base 14B achieves 27.3% Any Medal with AIDE vs. 9.1% with AIRA, Table 2)—but it does mean that scaffold improvements alone do not produce transferable MLE capabilities. Research that combines scaffold design with RL training (neither the paper nor prior work explores this) becomes the natural next step, and scaffold-only papers now face a higher bar to demonstrate that their gains persist across frameworks.

Reconciling prior contradictions. The paper resolves a tension that was implicit in the MLE agent literature but never directly articulated. On one side, SWE and web agent papers (DeepSWE, WebDancer) showed that trajectory-wise RL substantially improves agent performance. On the other side, MLE agent papers retreated to SFT or off-policy methods, implying that RL was infeasible for MLE. The field lacked a clear explanation for this discrepancy. SandMLE identifies the root cause: it's not that MLE tasks are inherently incompatible with RL, but that the execution latency of real MLE environments makes standard on-policy RL computationally prohibitive. This is a diagnostic contribution: it explains why MLE lagged behind SWE in adopting RL, and it provides a concrete measurement (196s vs. 14s per execution, Figure 6) that quantifies the gap. The paper also implicitly reconciles the tension between SFT and RL for agent training: SFT on expert trajectories teaches scaffold-locked behavioral patterns (the Seed-SFT collapse in Tables 2 and 3), while RL through environment interaction teaches transferable reasoning (the SandMLE cross-scaffold robustness). This provides empirical grounding for the "SFT memorizes, RL generalizes" thesis (Chu et al., 2025) in the specific context of agentic scaffolds.

What does not change. The paper does not claim that synthetic environments can fully replace real MLE tasks for evaluation. The evaluation is always on real benchmarks (MLE-Bench-Lite, MLE-Dojo). The synthetic environments are a training substrate, not a replacement for real-world testing. The paper also does not claim that test-time compute can substitute for pretraining—unlike the reference example paper, SandMLE does not conduct FLOPs-matched comparisons between RL-trained small models and larger pretrained models. The gains are demonstrated within a fixed model scale, improving the policy through RL rather than comparing across scales.

---

Follow-Up Research This Work Enables

1. On-policy vs. off-policy RL on identical fast environments. The paper's central theoretical argument is that on-policy training provides a "high-fidelity gradient signal" superior to off-policy alternatives that suffer from distribution shift or approximation error (Section 2.2). But this claim is never tested empirically. The most direct follow-up would train three variants on the same 848 SandMLE environments at matched wall-clock time: (a) standard on-policy GRPO (as in the paper), (b) asynchronous GRPO with stale trajectories (approximating Cai et al., 2026), and (c) GRPO with a learned proxy reward model trained offline on environment scores (approximating Liu et al., 2025b). The key measurement is whether the on-policy advantage persists when environment speed is no longer the bottleneck—i.e., given fast environments, does on-policy training still outperform off-policy alternatives, or was environment speed the only obstacle all along? If off-policy methods perform comparably, the paper's contribution reduces to "fast environments are useful" rather than "on-policy RL is necessary." If on-policy maintains a clear advantage, the paper's theoretical framing is validated and the field should prioritize making environments fast enough for on-policy training rather than developing more sophisticated off-policy algorithms.

2. Difficulty-stratified evaluation to identify where RL training helps (and where it doesn't). The paper evaluates on MLE-Bench-Lite's Easy split but trains on environments seeded from Medium, Hard, and Dev splits. This creates an implicit difficulty curriculum whose effects are unexplored. A natural follow-up would evaluate SandMLE-trained models on the full MLE-Bench across all difficulty splits (Easy, Medium, Hard), and report per-difficulty performance. The reference example paper found that test-time compute provides zero benefit on the hardest MATH problems—does SandMLE follow the same pattern? Specifically: do the RL gains concentrate on problems where the base model already has non-trivial performance (e.g., Easy and some Medium tasks), while providing minimal improvement on Hard tasks where the base model's pass@1 is near zero? This would reveal whether SandMLE amplifies existing capability or creates genuinely new capability. The experiment would also clarify whether training on harder synthetic tasks (seeded from Medium/Hard MLE-Bench) transfers to easier real tasks but not harder ones—an asymmetric transfer that would have practical implications for curriculum design.

3. SFT on SandMLE-generated trajectories to disentangle environment diversity from RL interaction. The current Seed-SFT baseline uses Claude-4.5-Sonnet trajectories on the original 60 seed tasks, not on the 848 synthetic tasks. This confounds two variables: task diversity (60 vs. 848) and training algorithm (SFT vs. RL). The SandMLE advantage over Seed-SFT could come from training on 14× more diverse tasks rather than from RL per se. A clean ablation would generate expert trajectories (from Claude-4.5-Sonnet or another strong model) on the 848 SandMLE environments, then SFT the base Qwen3 models on those trajectories. If SFT-on-SandMLE matches SandMLE's RL performance, the paper's contribution is primarily in environment generation (creating diverse training tasks) rather than in the RL training methodology. If SFT-on-SandMLE underperforms SandMLE RL, it strengthens the claim that environment interaction—not just environment diversity—is the active ingredient. The cross-scaffold generalization of SFT-on-SandMLE would also be informative: does training on diverse tasks via SFT produce more robust policies than training on a single scaffold's trajectories, or is the scaffold-locking problem inherent to SFT regardless of task diversity?

4. Combining SandMLE-trained policies with advanced scaffolds at inference time. The paper evaluates SandMLE models in ReAct (the training scaffold) and in AIDE/AIRA/MLE-Agent (unseen scaffolds), but never asks: what is the best combined system? Take the strongest SandMLE-trained model (30B-SFT-SandMLE) and deploy it with the strongest scaffold (AIDE, which gives the Base 14B model a 27.3% Any Medal rate vs. 18.2% with ReAct, Table 1 vs. Table 2). Does the RL training compound with the scaffold improvement, producing performance beyond what either alone achieves? Or do the benefits saturate—the scaffold already captures most of the available improvement, leaving little room for RL training to add value? The paper's test-time scaling experiment (Figure 7) shows that SandMLE models benefit from additional inference turns in ReAct; testing this in AIDE would reveal whether sophisticated scaffolds amplify or diminish the benefits of RL training. This is the experiment a practitioner would most want to see before deciding whether to invest in SandMLE training or simply use a better scaffold with the base model.

5. Scaling the number of seed tasks and the amplification factor. The paper uses 60 seed tasks to generate 848 synthetic training tasks (an amplification factor of ~14×). Two natural scaling experiments: (a) Vary the number of seed tasks (10, 30, 60, 120) while keeping the amplification factor fixed, measuring whether RL performance saturates with seed diversity. If 30 seeds produce comparable results to 60, the approach is more practical than the current setup suggests. (b) Vary the amplification factor (generate 200, 500, 848, 1500 tasks from the same 60 seeds) to identify whether environment diversity is the bottleneck or whether 848 tasks already saturate the benefit. The paper's training dynamics (Figure 8) show stable convergence without overfitting, suggesting the 848 tasks are sufficient for 80 GRPO steps, but a larger corpus might enable longer training and further improvements. Both experiments would inform the cost-benefit tradeoff of the generation pipeline: if performance saturates at 200 tasks from 30 seeds, the upfront generation cost drops dramatically, making the approach more accessible.

6. Extending the structural DNA concept to other long-horizon agent domains. The paper's DNA extraction mechanism—abstracting task structure, mutating domain, injecting noise, regenerating data—is not inherently MLE-specific. Any domain where tasks can be decomposed into structural schemas (mathematical relationships between inputs and outputs) and semantic context (the "story" wrapping those relationships) could potentially use the same approach. Obvious candidates: data science competitions beyond Kaggle (e.g., scientific discovery tasks where the agent must analyze data and formulate hypotheses), quantitative finance (where market simulation environments could be procedurally generated with embedded factor models), and scientific experiment design (where the agent must design experiments to infer hidden causal structures, with the causal graph serving as the "DNA"). A follow-up would port the four-agent pipeline to a new domain, generate synthetic training environments, and test whether RL-trained agents transfer to real benchmarks in that domain. The key question is whether the DNA abstraction captures enough structural complexity to produce non-trivial training tasks across domains, or whether MLE's particular structure (competition format, clear metrics, baseline methods) makes it uniquely suited to this approach. A negative result—finding that the DNA approach produces trivial tasks in a new domain—would clarify the boundary conditions and prevent over-claiming generality.

---

Practical Applications and Downstream Use Cases

1. Cost-efficient fine-tuning of open-weight models for internal ML engineering tasks. Organizations that maintain internal ML pipelines (e.g., for customer churn prediction, fraud detection, demand forecasting) could use SandMLE to fine-tune an open-weight model (like Qwen3-8B or 14B) on synthetic tasks structurally similar to their internal problems. The paper shows that Qwen3-8B-SandMLE matches the Any Medal rate of DeepSeek-V3.1 and Gemini-2.5-Flash on MLE-Bench-Lite (22.7%, Table 1)—closed-source models that are orders of magnitude larger and cost money per API call. A company could generate synthetic tasks modeled after their specific ML challenges (tabular classification, time-series forecasting, image-based inspection), train a small open model via SandMLE's pipeline, and deploy it internally at near-zero inference cost. The key number: the 13× execution speedup (Figure 6) means the RL training loop runs in hours rather than days on modest GPU infrastructure (single H200 per task, Section 5.1), making this feasible for teams without access to large-scale training clusters.

2. Bootstrapping data generation for self-improving MLE agents. The paper's finding that RL-trained models generalize across scaffolds (Tables 2 and 3) while SFT-trained models collapse suggests a self-improvement architecture: (1) generate synthetic environments from seed tasks using the SandMLE pipeline, (2) train a base model via GRPO on these environments, (3) deploy the trained model on real MLE tasks to collect high-quality trajectories (which the RL-trained model should produce more reliably than the base model, given the 13.7 point Any Medal gain at 30B in Table 1), (4) use those trajectories as training data for the next iteration. This creates a flywheel where each iteration's improved model generates better trajectories, which train the next iteration. The paper does not explore this (the SFT-SandMLE variant uses a fixed SFT initialization, not an iterative process), but the framework-agnostic generalization is precisely the property needed for a self-improvement loop to avoid collapse: if each generation's improvements were scaffold-locked (like Seed-SFT), the loop would diverge as the scaffold evolved. The 83.9% valid submission rate of Qwen3-30B-SandMLE on MLE-Dojo with MLE-Agent (Table 3) suggests that RL-trained models can reliably produce valid outputs across diverse environments—a prerequisite for automated data collection.

3. Training ML assistants for educational settings. MLE-Bench and MLE-Dojo consist of Kaggle-style competition tasks that resemble the assignments in applied ML courses. A SandMLE-trained model could serve as an interactive assistant that helps students debug their ML pipelines, suggest preprocessing strategies, or explain why a particular model choice underperforms—capabilities that require the trial-and-error reasoning that RL teaches (Section 6.1: "the RL policy explores aggressive strategies") but that SFT on static trajectories does not provide. The test-time scaling result (Figure 7: Any Medal rate scales from 23% at 0 turns to 55% at 30 turns) suggests that the assistant could be configured to spend more compute on harder student queries, providing deeper analysis when needed. The key practical advantage is that the model is an open-weight Qwen3 variant that can run on consumer hardware (8B and 14B are within the range of high-end consumer GPUs), making it deployable in educational settings without API costs or privacy concerns about sending student code to cloud services.

4. Verifier-controlled autonomous ML experimentation for A/B testing platforms. Many organizations run automated A/B testing and model refresh pipelines where new ML models are periodically trained and evaluated against production baselines. A SandMLE-trained agent could serve as the "auto-ML engineer" in this loop: given a dataset and a metric, it autonomously proposes, implements, and evaluates candidate models before deploying the best one. The milestone-based reward formulation is directly applicable here: the agent gets credit for producing a valid submission (execution success, 0.3 weight), beating the production baseline (median milestone, 0.1), improving substantially (bronze/silver, 0.2 each), and occasionally discovering breakthrough models (gold, 0.1). The paper's environment sanity check (Section 4.2) provides a template for how to verify that the reward landscape is monotonic before deploying the agent—critical for production systems where a misaligned reward could cause the agent to "optimize" by exploiting evaluation bugs rather than improving model quality. The 100% valid submission rate of Qwen3-30B-SandMLE (Table 1) is especially relevant here: in an automated pipeline, producing valid outputs reliably is more important than occasionally producing brilliant but broken ones.